Non-visualisation of the fetal gallbladder (NVFGB) in the second trimester is a rare condition affecting 0.1% of pregnancies.1 It might be a transient finding—the gallbladder is visible later in pregnancy or after birth in 70% of cases.2 Persistent NVFGB may be associated with benign conditions (eg, gallbladder agenesis); it may also be a manifestation of serious disorders, such as biliary atresia (BA), cystic fibrosis, or chromosomal abnormalities.3 Although chromosomal abnormalities and cystic fibrosis can be identified prenatally via chromosomal microarray (CMA) and sequencing of the cystic fibrosis transmembrane conductance regulator gene, respectively, it remains challenging to diagnose BA before birth because no diagnostic prenatal test is currently available. Biliary atresia is a devastating condition and is the leading indication for liver transplantation in childhood.4 Prenatal suspicion of BA allows prompt postnatal assessment and early diagnosis, permitting timely intervention via Kasai hepatoportoenterostomy and resulting in improved outcomes.4

The measurement of gamma-glutamyl transferase (GGT) in amniotic fluid (AF) has been suggested as a method for prenatal detection of BA. This transferase is secreted by the fetal biliary tract, passes into the intestines, and is ultimately excreted into the amniotic cavity. It becomes detectable in AF at around 14 weeks of gestation upon maturation of the intestinal villi and opening of the cloacal membrane.5 6 7 Biliary tract obstructions, such as BA, hinder the passage of GGT into the intestines and AF, leading to reduced levels of amniotic fluid gamma-glutamyl transferase (AFGGT). Muller et al5 first reported extremely low AFGGT levels in three fetuses with extrahepatic bile duct obstruction at 18 to 19 weeks of gestation; they concluded that a low AFGGT level could be a useful indicator of BA. Subsequently, Burc et al,6 Dreux et al,8 and Bardin et al9 reported similar findings. Nevertheless, existing platforms for analysis of GGT in serum or plasma samples have not been thoroughly validated for use with AF samples. Furthermore, an appropriate reference range, essential for the interpretation of AFGGT results, is difficult to establish due to the limited availability of AF samples from normal pregnancies. Thus, publications regarding gestational age–specific reference ranges for AFGGT have been scarce.

Burc et al6 established reference values for five AF enzymes (AFEs), including GGT, using the Hitachi 911 analyser (Roche Diagnostics). Despite a large sample size of 508, no separate reference range was constructed for each gestational age from 20 to 24 weeks, limiting clinical use of the findings.6 Bardin et al7 established another reference range for AFGGT, from 16 to 22 weeks, using the Integra 800 analyser (Roche). However, the numbers of samples at weeks 16, 21, and 22 were small and had relatively large standard deviations, precluding clinical application.7 Both reference ranges were derived from Caucasian populations. Because the GGT level in adult blood varies according to ethnicity,10 it is likely that AFGGT levels also vary according to ethnicity; thus, there is a need to establish a local AFGGT reference range. Furthermore, both previous reference ranges covered the 5th to 95th percentiles. A wider reference range, from the 2.5th to 97.5th percentiles, would allow greater flexibility in selecting cut-off values for clinical application.

In this study, we aimed to validate a serum/plasma matrix–based GGT assay for AF samples, establish a local gestational age–specific reference range for AFGGT (from the 2.5th to 97.5th percentiles), and evaluate the efficacy of AFGGT for predicting fetal BA in pregnancies with NVFGB using the constructed reference range.

This retrospective study used archived AF supernatant obtained from amniocentesis procedures that had been conducted to exclude fetal chromosomal abnormalities between January 2012 and December 2021. All amniocenteses were performed under ultrasound guidance with aseptic technique by fetal-maternal medicine specialists who also performed detailed ultrasound examinations to document the presence or absence of fetal structural abnormalities. All AF samples were centrifuged at 100 g for 10 minutes; 1 mL of the resulting supernatant was stored at -80℃ in plain aliquots without additives (Axygen; BioGene, Union City [CA], United States), whereas the pellet was used for either CMA and/or karyotyping by G-banding analysis. The cytogenetic laboratory information system recorded the ultrasound findings, indication for amniocentesis, gestational age at amniocentesis, and CMA and/or karyotype results. Pregnancy and fetal outcomes, including the presence of any abnormalities at birth or autopsy findings, were recorded in each patient’s electronic records.

Archived AF samples with known fetal and pregnancy outcomes, taken at 16⁺⁰ to 22⁺⁶ weeks of gestation from singleton pregnancies with a euploid fetus (confirmed by CMA or karyotype) during the period from January 2018 to December 2020, were retrieved to establish a gestational age–specific reference range for GGT. Pregnancies with fetal chromosomal, gastrointestinal, or hepatobiliary anomalies (particularly BA) polyhydramnios, or oligohydramnios were excluded to eliminate potential confounding effects of these conditions.

Non-visualisation of the fetal gallbladder was incidentally detected during fetal morphology scans in pregnancies with risk factors for fetal abnormalities because the fetal gallbladder was not assessed in low-risk routine anomaly scans, in accordance with the International Society of Ultrasound in Obstetrics and Gynecology guideline.11 It was defined as failure to visualise the fetal gallbladder on two targeted ultrasound examinations performed 1 week apart. Isolated NVFGB was defined as NVFGB in the absence of other abnormal ultrasound findings. Pregnant women were counselled regarding possible differential diagnoses and offered amniocentesis for chromosomal analysis, as well as repeated ultrasound scans until the fetal gallbladder was visible. After birth, babies with persistent NVFGB were referred to paediatricians for hepatobiliary ultrasound and liver function tests. If the gallbladder was visible during prenatal scans, paediatricians did not order further tests in the absence of clinical suspicion. We followed the progress of all babies until the time of writing, using electronic hospital records for those delivered in public hospitals and phone calls for those delivered in private hospitals. With parental consent, post-mortem examinations were arranged in pregnancies terminated for serious associated fetal abnormalities.

Amniotic fluid samples were removed from -80℃ storage in batches, thawed, and equilibrated to room temperature immediately prior to analysis. Gamma-glutamyl transferase levels were determined using an International Federation of Clinical Chemistry and Laboratory Medicine–standardised L-gamma-glutamyl-3-carboxy-4-nitroanilide (GGCNA) enzymatic colorimetric assay on a Cobas c502 analyser (Roche, Basel, Switzerland). Internal quality controls were performed before and after each batch.

Details of the AFGGT assay validation and analytical performance, including matrix effects; linearity; intra- and inter-run precision at various GGT levels; interference due to haemolysis, icterus, and lipaemia; and sample stability, are summarised in online supplementary Appendix 1.

An AFGGT reference range was developed using the Generalised Additive Models for Location (μ), Scale (υ) and Shape (σ) [GAMLSS] package in R statistical software (version 3.3.2; R Foundation for Statistical Computing, Vienna, Austria). All GGT values were transformed to their natural log equivalent before model construction; the final model balanced between percentile smoothness, goodness-of-fit, and simplicity. Model fit was assessed using the generalised Akaike information criterion and by inspection of residuals with quantile-quantile plots for all measurements, detrending of quantile-quantile plots, and comparison of empirical percentiles to fitted percentiles. Empirical percentiles were determined for comparative purposes by grouping GGT levels according to gestational age (in weeks).

The final model was used to determine reference values for the 2.5th, 5th, 50th, 95th (z = ± 1.645), and 97.5th percentiles (z = ± 1.964). Percentiles were determined using the expression μ × (1 + z_pυσ)1/υ, where z_p is the percentile of interest and μ, υ, and σ are dependent on the time covariate, gestational age. The GGT reference range was constructed using R statistical software and Microsoft Excel (Microsoft Corporation, Redmond [WA], United States).

We estimated a priori that 293 AFGGT measurements were needed to achieve a standard error of 10% of the gestational age–specific standard deviation for the 2.5th and 97.5th (z=1.96) reference percentiles, assuming that the standard error of the percentile of interest is expressed as a multiple of standard deviation using the formula , where each gestational age between 16⁺⁰ and 22⁺⁶ weeks has a minimum of 42 measurements.

Measured levels of AFGGT in pregnancies with NVFGB were transformed to their gestational age–specific percentile values using the final model. We then determined the positive predictive value (PPV) and negative predictive value (NPV) of AFGGT level <2.5th percentile for identifying BA in euploid pregnancies with NVFGB.

The performance of the GGT assay using AF is summarised in online supplementary Appendix 2. Validation studies indicated that the GGCNA enzymatic colorimetric assay for determination of GGT activity in AF had linearity, precision, recovery, interference profiles, and stability comparable to the values reported for measurement of GGT in plasma and serum samples. The verified analytical measurement range was 10 to 1200 U/L. No significant interference was observed in the presence of 0.25 g/dL haemoglobin, 103 μmol/L bilirubin, or 16.8 mmol/L triglyceride. The limits of haemolysis/icterus/lipaemia indices above which interference occurred were significantly higher than the degrees of those indices in all analysed samples.

A database search identified 518 stored amniotic samples (502 [97%] Chinese and 16 [3%] other Asian ethnicities) suitable for use in establishing a local gestational age–specific AFGGT reference range. The median number of samples per week was 65; there were 2 weeks with <42 samples (27 and 28 samples in the 16th and 19th week of gestation, respectively). The online supplementary Table lists the regressed values of AFGGT levels according to percentile for each week of gestation.

The simplest best-fit model indicated a linear relationship between natural log-transformed GGT and gestational age. Table 1 and the online supplementary Figure show the final smoothing equations for median, coefficient of variation, and skewness, along with gestational age–specific smoothed percentile curves and values of AFGGT for our local population. Residuals of the final model had a mean skewness of 0 and a variance of 1; they were almost mesokurtic, with kurtosis values close to 3.

The database search identified 32 pregnancies with NVFGB from 2012 to 2021, of which 18 had an available AF sample (four isolated NVFGB and 14 non-isolated NVFGB). There were no cases of cystic fibrosis. Nine cases were excluded from analysis, including five with chromosomal abnormalities, one with renal hypoplasia and oligohydramnios, one with hydrops and polyhydramnios, and two in which the biliary tract anatomy could not be identified. The nine remaining cases for analysis included four with isolated NVFGB and five with non-isolated NVFGB (Table 2). The median gestational age at amniocentesis was 21.0 weeks (interquartile range=20.7-22.1).

The Figure depicts the AFGGT levels in the nine pregnancies for analysis compared with our local gestational age–specific AFGGT reference range. Four cases had AFGGT level <2.5th percentile, including one with BA (AFGGT level of 27 U/L at 20⁺³ weeks) and three with transient non-visualisation (AFGGT levels of 32, 68, and 37 U/L at 20⁺⁰, 20⁺⁶, and 22⁺² weeks, respectively). Five had AFGGT level ≥2.5th percentile, including one with gallbladder agenesis and four with transient non-visualisation (Table 2). Using AFGGT level <2.5th percentile as the cut-off, the NPV and PPV for identifying fetal BA in pregnancies with NVFGB were 100% and 25% (95% confidence interval=0, 53), respectively (Table 3). Repeated analysis using AFGGT level <5th percentile or the reference ranges from Burc et al6 or Bardin et al7 yielded identical results.

Biliary atresia is a rare congenital anomaly, with a prevalence of 1 in 15 000 to 20 000 live births among Caucasian populations.12 However, BA is more common in East Asians; the prevalence is 1 in 5000 to 7000 among Chinese populations.13 14 Untreated BA is a progressive and devastating disease that can cause cirrhosis and death by 2 years of age.14 This outcome can be prevented by early intervention via palliative Kasai hepatoportoenterostomy, which is essential for re-establishing biliary drainage. If biliary drainage cannot be re-established, liver transplantation is necessary. Indeed, BA is the most common indication for liver transplantation in children, contributing to 75% of transplantations in children before 2 years of age.12 It is therefore imperative to diagnose BA early, preferably during the prenatal period. Nevertheless, prenatal diagnosis of BA is challenging because ultrasound cannot directly examine the patency of fetal bile ducts. When NVFGB is associated with a hepatic hilar cyst or heterotaxy, it is highly suggestive of BA.15 Non-visualisation of the fetal gallbladder with a hepatic hilar cyst is an indicator of cystic BA, a rare subtype representing 5% to 10% of BA cases16; therefore, this prenatal combination is uncommon. Heterotaxy is another rare condition with a prevalence of 1 in 10 000 live births,17 and concurrent BA is present in only 10.4% of left atrial isomerism cases18; therefore, this prenatal combination is even less common. Consequently, NVFGB may be the only prenatal sign indicating the possibility of BA. However, in cases of isolated NVFGB, it is difficult to differentiate between BA and gallbladder agenesis. The discovery of the association between low AFGGT levels and fetal BA has led to interest regarding the role of AFGGT in the management of NVFGB.

To our knowledge, this study is the first to validate the International Federation of Clinical Chemistry and Laboratory Medicine–standardised GGCNA enzymatic colorimetric assay on the Cobas c502 analyser, a common locally available plasma and serum analyser, for use with AF. We have demonstrated that accurate and precise measurements of GGT can be achieved with AF samples, enabling adoption in clinical settings. We have also established that the analytical measurement range is 10 to 1200 U/L. This is particularly important for an AFGGT assay because AFGGT levels in euploid pregnancies can vary across multiple orders of magnitude, whereas plasma GGT levels in healthy individuals usually remain below 100 U/L. This verification of the lower limit of quantification and linearity range improves confidence in our measurements. Furthermore, we have excluded potential interference, particularly from haemoglobin, because AF samples may sometimes contain maternal blood. We were initially concerned about GGT stability because some samples had been stored for several years; however, consistent with the World Health Organization report that GGT is stable for years in frozen serum and plasma samples,19 we found that AF stored frozen in plain bottles without additives at -20℃ or -80℃ remained stable for at least 6 months (online supplementary Appendix 1). This finding indicates that supernatants can be stored and subsequently retrieved for GGT assays, an important consideration if amniocentesis is performed in the early second trimester but an indication for AFGGT testing is identified during a mid-trimester morphology scan.

We have established a reference range for AFGGT levels at 16⁺⁰ to 22⁺⁶ weeks of gestation using a large local reference population of 518 samples (all Asian, 97% Chinese); this reference population is the largest compared with similar previous publications. The presence of an adequate sample size for each week of gestation allowed us to establish a reference range for each gestational age, thus overcoming the aforementioned limitations regarding clinical use of the two previous reference ranges.6 7 With respect to the two previous reference ranges, the 5th, 50th, and 95th percentiles in our study are similar to those reported by Bardin et al7 but higher at most gestational ages than those reported by Burc et al.6 The larger sample size in our study permitted the calculation of the 2.5th and 97.5th percentiles, such that a 95% central reference range could be established; this allows greater flexibility in selecting cut-off values for clinical application.

It has been reported that AFGGT levels are increased in oesophageal atresia and duodenal atresia, whereas they are decreased in anal atresia without fistula.20 21 Although duodenal atresia can easily be diagnosed prenatally by detection of the double bubble sign, this sign usually appears after 24 weeks of gestation.22 Prenatal diagnosis of oesophageal atresia relies on the indirect sign of non-visualisation of the stomach bubble and polyhydramnios; the sensitivities of these ultrasound findings range from 8.9% to 42%.20 Additionally, prenatal detection of anal atresia without fistula depends on the absence of the perianal muscular complex, but the anal sphincter does not fully mature until after 28 weeks of gestation.23 Further research regarding the use of AFGGT for early detection of these congenital gastrointestinal tract obstructions is valuable, and the availability of a local gestational age–specific reference range is crucial for supporting such research.

In the present study, the NPV and PPV of AFGGT level <2.5th percentile for identifying fetal BA in NVFGB were 100% and 25%, respectively (Table 3). Bardin et al9 assessed the efficacy of low AFGGT levels in predicting BA among cases of NVFGB between 17 and 22 weeks of gestation. Of the 26 cases with AFGGT level ≥5th percentile, none had BA; of the four cases with AFGGT level <5th percentile, three had BA. The corresponding NPV and PPV were 100% and 75%, respectively. The PPV in their study may have been higher because amniocentesis was performed before 22 weeks of gestation in all cases.9 In our cohort, after exclusion of the two cases in which amniocentesis was performed after 22 weeks of gestation, the PPV only marginally improved to 33.3%. Our figures are more consistent with those of Dreux et al,8 who analysed the efficacy of AFEs for predicting BA in NVFGB before and after 22 weeks of gestation. In that study, abnormal AFE was defined as GGT and/or intestinal alkaline phosphatase level <0.5 multiples of the median.8 Before 22 weeks of gestation, there were three cases of BA among seven cases with abnormal AFE and no cases of BA among 16 cases with normal AFE. The corresponding NPV and PPV were 100% and 43%, respectively. However, after 22 weeks of gestation, there was only one case of BA among six cases with abnormal AFE and four cases of BA among 56 cases with normal AFE. The corresponding NPV and PPV were 93% and 17%, respectively (Table 3). These findings confirmed the expected decrease in AFE efficacy for predicting BA after 22 weeks of gestation. By that gestational age, the passage of GGT from the intestine into the AF is impeded by mature anal sphincter muscles in normal fetuses; thus, the AFGGT level is very low after 22 weeks of gestation, and it is difficult to distinguish between a low level due to BA and a low level related to normal development.5 6 7

In our cohort, there were three fetuses without BA who had AFGGT level ≤2.5th percentile; one of these fetuses had an extremely low AFGGT level (Case 4) [Table 2]. In addition to its association with fetal BA, low AFGGT is linked to chromosomal abnormalities, cystic fibrosis, anal atresia without fistula, and polyhydramnios. However, none of the three fetuses had any of these conditions. Although one fetus (Case 2) [Table 2] had a small choledochal cyst which might have impeded biliary drainage and caused a mild decrease in AFGGT, we could not find a potential explanation for the low AFGGT levels in the other two fetuses.

Limitations of our study include its retrospective nature and small cohort size. The incidence of NVFGB is low (0.1%).1 In the largest systematic review concerning the outcomes of NVFGB, encompassing seven studies, the total number of cases was 280; among 170 cases of isolated NVFGB, only six (3.5%) had BA.2 In Hong Kong, the fetal gallbladder is not routinely assessed in low-risk routine anomaly scans, in accordance with the International Society of Ultrasound in Obstetrics and Gynecology guideline.11 24 Among the 32 cases of NVFGB in our cohort, one (3.1%) had BA; this incidence is comparable to the findings in the aforementioned review. However, only 18 cases had an AF sample available for AFGGT testing; after the exclusion of cases with abnormalities that might affect the AFGGT level, we included nine cases in the analysis. Because there was only one case of BA in our cohort and 11 more were described in the literature (Table 3), clinical application of these research findings requires caution, as well as careful pre- and post-test counselling.

In addition to the one case of BA in this cohort, two additional cases of BA were not included in this study because prenatal ultrasound scans did not indicate NVFGB. The AFGGT levels in all three cases were <30 U/L and <2.5th percentile (27, 28, and 29 U/L at 20⁺³, 21⁺⁵, and 21⁺⁶ weeks, respectively), whereas the AFGGT levels in all samples used to establish our reference range were >30 U/L. Thus, an AFGGT level <30 U/L may be a useful absolute cut-off for the prediction of BA. Notably, all three cases of extrahepatic bile duct obstruction reported by Muller et al5 also had an AFGGT level <30 U/L (20 U/L for all three cases [<1st percentile]). Further research with a larger cohort is required to confirm the efficacy of using an AFGGT level <30 U/L as the absolute cut-off for predicting BA in NVFGB.

Based on the present findings and published literature, we conclude that AFGGT testing is useful for the exclusion of fetal BA in pregnancies with NVFGB. With a consistent NPV of 100% across all published series, AFGGT level ≥2.5th percentile can provide reassurance for parents that the fetus is unlikely to have BA. However, considering its PPV of 33.3% to 75% before 22 weeks of gestation and 17% to 43% after 22 weeks of gestation, AFGGT level <2.5th percentile cannot be considered diagnostic for BA. Instead, it serves as a warning sign, indicating the need for prompt postnatal investigation of possible BA. Because NVFGB is also associated with chromosomal abnormalities, amniocentesis is recommended; the advantages of detecting underlying chromosomal abnormalities by CMA and excluding BA through AFGGT testing likely outweigh the 0.3% risk of procedure-related miscarriage.25 Follow-up prenatal ultrasound scans to visualise the fetal gallbladder should be arranged. Paediatricians should also be alerted for prompt postnatal assessment to facilitate early detection of BA. Timely performance of the Kasai operation can reduce the need for liver transplantation in childhood and improve the rate of overall survival into adulthood to 90%.26 27

Concept or design: YH Ting, DS Sahota, FCK Wong, TYT Cheung.
Acquisition of data: TYT Cheung, SR Subramaniam, FCK Wong, MHM Chan, YH Ting, NKL Wong, SL Lau, X Zhu, WT Lui, EKW Chan, YKY Kwok, KW Choy, TY Leung.
Analysis or interpretation of data: TYT Cheung, DS Sahota, FCK Wong, YH Ting, NKL Wong.
Drafting of the manuscript: YH Ting, TYT Cheung, DS Sahota, NKL Wong, FCK Wong, SR Subramaniam.
Critical revision of the manuscript for important intellectual content: DS Sahota, YH Ting, TYT Cheung, FCK Wong, NKL Wong.

All authors had full access to the data, contributed to the study, approved the final version for publication, and take responsibility for its accuracy and integrity.

All authors have disclosed no conflicts of interest.

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

This research was approved by the Joint Chinese University of Hong Kong—New Territories East Cluster Clinical Research Ethics Committee, Hong Kong (Ref No.: CRE 2020.060). Informed consent for amniocentesis in the current study and storage and use of excess amniotic fluid in future research was obtained from the patients at the time of amniocentesis.

The supplementary material was provided by the authors and some information may not have been peer reviewed. Accepted supplementary material will be published as submitted by the authors, without any editing or formatting. Any opinions or recommendations discussed are solely those of the authors and are not endorsed by the Hong Kong Academy of Medicine and the Hong Kong Medical Association. The Hong Kong Academy of Medicine and the Hong Kong Medical Association disclaim all liability and responsibility arising from any reliance placed on the content.

1. Blazer S, Zimmer EZ, Bronshtein M. Nonvisualization of the fetal gallbladder in early pregnancy: comparison with clinical outcome. Radiology 2002;224:379-82. Crossref

2. Di Pasquo E, Kuleva M, Rousseau A, et al. Outcome of non-visualization of fetal gallbladder on second-trimester ultrasound: cohort study and systematic review of literature. Ultrasound Obstet Gynecol 2019;54:582-8. Crossref

3. Ting YH, So PL, Cheung KW, Lo TK, Ma TW, Leung TY. Non-visualisation of fetal gallbladder in a Chinese cohort. Hong Kong Med J 2022;28:116-23. Crossref

4. Parolini F, Boroni G, Milianti S, et al. Biliary atresia: 20-40-year follow-up with native liver in an Italian centre. J Pediatr Surg 2019;54:1440-4. Crossref

5. Muller F, Gauthier F, Laurent J, Schmitt M, Boué J. Amniotic fluid GGT and congenital extrahepatic biliary damage. Lancet 1991;337:232-3. Crossref

6. Burc L, Guibourdenche J, Luton D, et al. Establishment of reference values of five amniotic fluid enzymes. Analytical performances of the Hitachi 911. Application to complicated pregnancies. Clin Biochem 2001;34:317-22. Crossref

7. Bardin R, Danon D, Tor R, Mashiach R, Vardimon D, Meizner I. Reference values for gamma-glutamyl-transferase in amniotic fluid in normal pregnancies. Prenat Diagn 2009;29:703-6. Crossref

8. Dreux S, Boughanim M, Lepinard C, et al. Relationship of non-visualization of the fetal gallbladder and amniotic fluid digestive enzymes analysis to outcome. Prenat Diagn 2012;32:423-6. Crossref

9. Bardin R, Ashwal E, Davidov B, Danon D, Shohat M, Meizner I. Nonvisualization of the fetal gallbladder: can levels of gamma-glutamyl transpeptidase in amniotic fluid predict fetal prognosis? Fetal Diagn Ther 2016;39:50-5. Crossref

10. Stewart SH, Connors GJ, Hutson A. Ethnicity and gamma-glutamyltransferase in men and women with alcohol use disorders. Alcohol Alcohol 2007;42:24-7. Crossref

11. Salomon LJ, Alfirevic Z, Berghella V, et al. Practice guidelines for performance of the routine mid-trimester fetal ultrasound scan. Ultrasound Obstet Gynecol 2011;37:116-26. Crossref

12. Hartley JL, Davenport M, Kelly DA. Biliary atresia. Lancet 2009;374:1704-13. Crossref

13. Hsiao CH, Chang MH, Chen HL, et al. Universal screening for biliary atresia using an infant stool color card in Taiwan. Hepatology 2008;47:1233-40. Crossref

14. Zhan J, Chen Y, Wong KK. How to evaluate diagnosis and management of biliary atresia in the era of liver transplantation in China. World J Paediatr Surg 2018;1:e000002. Crossref

15. Chalouhi GE, Muller F, Dreux S, Ville Y, Chardot C. Prenatal non-visualization of fetal gallbladder: beware of biliary atresia! Ultrasound Obstet Gynecol 2011;38:237-8. Crossref

16. Rahamtalla D, Al Rawahi Y, Jawa ZM, Wali Y. Cystic biliary atresia in a neonate with antenatally detected abdominal cyst. BMJ Case Rep 2022;15:e246081. Crossref

17. Lin AE, Ticho BS, Houde K, Westgate MN, Holmes LB. Heterotaxy: associated conditions and hospital-based prevalence in newborns. Genet Med 2000;2:157-72. Crossref

18. Gottschalk I, Stressig R, Ritgen J, et al. Extracardiac anomalies in prenatally diagnosed heterotaxy syndrome. Ultrasound Obstet Gynecol 2016;47:443-9. Crossref

19. Ehret W. Use of Anticoagulants in Diagnostic Laboratory Investigations. Geneva: World Health Organization; 1999.

20. Czerkiewicz I, Dreux S, Beckmezian A, et al. Biochemical amniotic fluid pattern for prenatal diagnosis of esophageal atresia. Pediatr Res 2011;70:199-202. Crossref

21. Muller C, Czerkiewicz I, Guimiot F, et al. Specific biochemical amniotic fluid pattern of fetal isolated esophageal atresia. Pediatr Res 2013;74:601-5. Crossref

22. The Fetal Medicine Foundation. Duodenal atresia. Available from: https://fetalmedicine.org/education/fetal-abnormalities/gastrointestinal-tract/duodenal-atresia. Accessed 19 Jan 2023.

23. Ochoa JH, Chiesa M, Vildoza RP, Wong AE, Sepulveda W. Evaluation of the perianal muscular complex in the prenatal diagnosis of anorectal atresia in a high-risk population. Ultrasound Obstet Gynecol 2012;39:521-7. Crossref

24. Chen M, Leung TY, Sahota DS, et al. Ultrasound screening for fetal structural abnormalities performed by trained midwives in the second trimester in a low-risk population—an appraisal. Acta Obstet Gynecol Scand 2009;88:713-9. Crossref

25. Salomon LJ, Sotiriadis A, Wulff CB, Odibo A, Akolekar R. Risk of miscarriage following amniocentesis or chorionic villus sampling: systematic review of literature and updated meta-analysis. Ultrasound Obstet Gynecol 2019;54:442-51. Crossref

26. Serinet MO, Wildhaber BE, Broué P, et al. Impact of age at Kasai operation on its results in late childhood and adolescence: a rational basis for biliary atresia screening. Pediatrics 2009;123:1280-6. Crossref

27. Davenport M. Biliary atresia: clinical aspects. Semin Pediatr Surg 2012;21:175-84. Crossref

Cardiovascular disease (CVD) constitutes a spectrum of diseases that affect the heart and blood vessels. Worldwide, CVD is the leading cause of death as well as a major cause of premature death and chronic disability in numerous regions.1 In 2017, CVD was responsible for an estimated 17.8 million deaths, representing 31% of all global deaths.2 Hypertension, smoking status, hyperlipidaemia, and diabetes mellitus are prominent risk factors for CVD.3 The global prevalence of CVD risk factors is increasing. Currently, an estimated 15% of the world’s population (1.13 billion people) has hypertension, and the prevalence is expected to increase to 29% by 2025.4 5 Additionally, the global prevalence of diabetes increased from 211 million in 1990 to 476 million in 2017, representing a 129.7% increase in nearly three decades.6 The prevalences of CVD and its risk factors are expected to continue to increase in the near future due to industrialisation and population ageing. Fortunately, the World Health Organization (WHO) has estimated that premature CVD is preventable in >75% of cases, and risk factor amelioration can help reduce the burden caused by CVD.7

The risk of CVD over the next 10 years for an individual can be estimated using prediction models. Cardiovascular disease risk prediction is important at the individual and population levels. At the individual level, CVD risk prediction allows primary care medical professionals to identify high-risk patients. Risk factors for CVD, such as hypertension, can be treated accordingly to reduce the patient’s future risk of CVD. At the population level, CVD risk trends allow health policy planners to make evidence-based decisions and review current public health strategies used in CVD prevention.8

Changes in CVD risk can serve as a reference to indicate changes in public health status within a population. Two studies have evaluated changes in the 10-year CVD risk in the United States (US) population. In a study by Ajani and Ford,9 risk models adopted by the National Cholesterol Education Program Adult Treatment Panel III were utilised to estimate coronary heart disease risk, rather than the more precise Framingham formulae for estimation of overall CVD risk. Notably, their analysis lacked age stratification. On the other hand, Lopez-Jimenez et al10 evaluated the changes in CVD risk between 1976 and 2004. To our knowledge, no previous study has evaluated the change in 10-year CVD risk in an Asian population, and insights are needed regarding the cardiovascular health status of the population in a more recent time period.

In this study, we aimed to calculate and compare the changes in sex- and age-specific CVD risks predicted by the Framingham model in a Hong Kong general population by using the Hong Kong Population Health Survey (PHS) 2014/15 in combination with two previous surveys conducted in 2003 to 2005, namely, PHS 2003/2004 and Heart Health Survey (HHS) 2004/2005.

The data in this study were sourced from PHS 2003/200411 and PHS 2014/15.12 Population Health Surveys are territory-wide cross-sectional surveys conducted by the Department of Health of the Hong Kong SAR Government. These surveys target the land-based non-institutional population of individuals aged ≥15 years in Hong Kong, excluding foreign domestic helpers and visitors. Systematic replicate sampling was adopted to select living quarters that were representative of the Hong Kong general population. All domestic households in the selected living quarters and household members in the target population were individually surveyed. Written consent was obtained from individuals who agreed to participate in a PHS. Participants were invited to complete a face-to-face interview, where they provided information about their socio-demographic characteristics, disease status, and daily lifestyle habits. After the interview, consenting participants aged 15 to 84 years were randomly selected to undergo a health examination that included physical measurements and biochemical testing.12 Health examinations for participants of PHS 2003/2004 were conducted as part of the HHS 2004/2005.

The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) checklist was followed for preparing this manuscript.

The main outcome of this study was the predicted risk of CVD over the next 10 years among people aged 30 to 74 years. Thus far, no specific prediction models have been developed for the Hong Kong population. In this study, we adopted the prediction model for primary care developed by the Framingham Heart Study Cohort in the general adult population aged 30 to 74 years.13 The Framingham CVD risk model was validated and can be applied to the Chinese population, but requires recalibration in men.14 A CVD event was defined as a composite of coronary heart disease, cerebrovascular events, peripheral artery disease, and heart failure in the Framingham model. The predicted CVD risk over the next 10 years was calculated for each participant using the following information: age, total cholesterol level, high-density lipoprotein cholesterol level, systolic blood pressure, use of antihypertensive medication, current smoking status, and diabetes status.

A complete-case analysis approach was utilised in this study. Descriptive statistics were used to present the characteristics of included individuals. Sex- and age-specific predicted CVD risks in 2003-2005 and 2014-2015 were calculated to summarise the change in risk according to sex and age-group. Significant differences in the above factors between the two PHSs were tested by independent t test or Chi squared test, as appropriate.

To summarise results at the population level, population weighting established by the Department of Health was applied according to age-group and sex for participants in each PHS. To compare results at the population level, age-standardised predicted CVD risk was calculated using WHO 2000-2025 world standard population data15 and the US population census data in 2000.16

All statistical analyses were performed with SPSS software (Windows version 26.0; IBM Corp, Armonk [NY], US). All significance tests were two-tailed, and P values <0.05 were considered statistically significant.

In total, 1662 and 818 participants were included in the PHS 2014/15 cohort and the PHS 2003/2004 and HHS 2004/2005 cohort, respectively. These individuals represented populations of 4 445 868 and 3 495 074, respectively. Table 1 shows the baseline characteristics of the PHS and HHS cohorts according to Framingham predictors. The mean age was similar in both cohorts (2003-2005: 48.6 years vs 2014-2015: 50.8 years) and the proportions of male and female participants were similar (female participants in 2003-2005: 54.1% vs 2014-2015: 53.0%).

Predicted CVD risk increases with age and is higher in men (Table 2). In PHS 2003/2004 and HHS 2004/2005, the mean CVD risk increased with age in both sexes, from 1.6% among women aged 30-44 years to 17.8% among women aged 65-74 years, and from 5.1% among men aged 30-44 years to 33.4% among men aged 65-74 years. Between the two surveys, there was no significant difference in overall 10-year CVD risk (2003-2005: 10.2% vs 2014-2015: 10.6%; P=0.29). After age standardisation according to WHO world standard population data, the age-standardised predicted CVD risks in the 2003-2005 cohort and the 2014-2015 cohort were 9.4% and 8.8%, respectively (Table 3). The small decrease in predicted CVD risk may be due to the decrease in smokers among men (30.5% vs 24.0%; P<0.001) [Table 1].

The risk of cardiovascular events over the next 10 years was classified as low (CVD risk <10%), medium (CVD risk ≥10% and <20%), and high (CVD risk ≥20%); the distributions of risk groups are presented in Table 4. Among participants aged 30 to 74 years, the risk group distributions were similar in both cohorts. When analysed according to sex, 29.1% of men and 5.1% of women were classified as high-risk in PHS 2014/15, whereas 28.2% of men and 6.4% of women were classified as high-risk in PHS 2003/2004 and HHS 2004/2005. When analysed according to age-group, more participants aged 65 to 74 years were classified as high-risk in PHS 2003/2004 and HHS 2004/2005 (66.8% vs 2014-2015: 53.1%; P=0.028).

Using representative samples from the Hong Kong PHS and HHS, we found that the 10-year CVD risk increases with age and is consistently higher in men (men: 15.5% vs women: 6.2%; P<0.001, in PHS 2014/15) [Table 2]. This trend is consistent with previous reports; for example, an analysis of NHANES (National Health and Nutrition Examination Survey) III data showed that men have a significantly higher CVD risk (11.8% in men vs 5.1% in women) and that CVD risk is higher in older age-groups (16.2% for participants aged 60-74 years vs 11.2% for participants aged 50-59 years).10 Thus, risk management for older adults may represent a challenge to the current health infrastructure. The burden of CVD is directly associated with increased morbidity and mortality in patients, and it translates to substantial healthcare costs. In Hong Kong, the number of people aged ≥65 years is predicted to reach 2.58 million (35.9% of the population) by 2064.17 Thus, there is a critical need to achieve a more comprehensive understanding of the aetiologies associated with CVD in older adults.

The results have extensive public health implications. Although age-standardised rates of death from CVD in Hong Kong greatly decreased from 93.4 per 100 000 standard population in 2001 to 56.0 per 100 000 standard population in 2017,18 there was no significant difference in overall 10-year CVD risk between 2003-2005 and 2014-2015 (10.2% vs 10.6%; P=0.29) [Table 2]. After age standardisation according to WHO world standard population data, a small decrease in CVD risk was observed, from 9.4% in 2003-2005 to 8.8% in 2014-2015 [Table 3]. A study of NHANES data revealed a similar trend, consisting of a small decrease in CVD risk from 1988 to 2004 (7.9% to 7.4%; P<0.001).10 The stagnation in 10-year CVD risk between 2003-2005 and 2014-2015 suggests that despite improvements in treatment, more effective prevention strategies (eg, improvements in diet and physical activity levels) are needed.19 Primary care intervention to manage modifiable risk factors, such as hypertension, dyslipidaemia, and diabetes, can complement population-based policies. Efforts to ensure access to appropriate healthcare and affordable medications will help control abnormal risk factor levels.20 Furthermore, despite the importance of targeting individual risk factors to reduce prevalence, the long-term objective should be reduced overall risk of CVD. This objective requires a holistic approach simultaneously targeting multiple risk factors.

Our results highlight the need for overall risk assessment, in addition to targeted efforts focused on specific CVD risk factors. During the past decade, numerous public health initiatives (eg, smoking cessation campaigns) have been implemented to reduce the prevalence of established risk factors for CVD. It is reasonable to expect that CVD risk would decrease over time in conjunction with the decreased prevalences of various risk factors, such as the declining number of current smokers. Surprisingly, these changes were not strongly reflected in the change in 10-year CVD risk. One explanation is that decreases in the prevalence of some risk factors are offset by increases in others. For example, population ageing in Hong Kong may shift more adults into the high-risk group. However, because the Framingham model was limited to individuals aged 30 to 74 years, the change in mean age between the two surveys is inconsequential. Other possible changes in risk factors include increases in the prevalence of diabetes and hypertension, as reflected by the increased proportion of participants receiving antihypertensive medications (Table 1). Although CVD mortality has considerably decreased, it is concerning that CVD risk has not substantially diminished over the past decade. This lack of improvement in CVD risk may lead to an increasing community burden of CVD in the near future, especially in the context of population ageing.

The overall 10-year CVD risk for participants aged 30 to 74 years in 2003-2005 after age adjustment to US census population data in 2000 was 10.7%. During a similar time period (1999-2004), the US population had a 10-year CVD risk of 7.4%.10 When Hong Kong men were stratified according to risk group in 2014-2015, 48.8% were classified as low-risk, 22.1% were classified as medium-risk, and 29.1% were classified as high-risk (Table 4). For comparison, among men in the United Kingdom population during 2012, 46.5% were classified as low-risk, 39.9% were classified as medium-risk, and 13.6% were classified as high-risk using the National Institute for Health and Care Excellence Framingham risk model.21 In terms of risk distribution, a greater proportion of Hong Kong men were classified as high-risk compared with men in the United Kingdom in the early 2010s. In terms of age-standardised risk, the Hong Kong population had a higher 10-year CVD risk than the US population in 2003.

Analysis according to age showed that the proportion of low-risk participants increased from 2003-2005 to 2014-2015, particularly in the age-groups of 55-64 and 65-74 years (2003-2005: 28.5% vs 2014-2015: 44.7% for participants aged 55-64 years, and 6.1% vs 15.4% for participants aged 65-74 years) [Table 4]. This increased proportion may be the result of primary care clinicians recognising age as a prominent risk factor for CVD, leading to more aggressive treatment of modifiable risk factors. Furthermore, there were fewer male smokers in 2014-2015 than in 2003-2005 (24.0% vs 30.5%; P<0.001) [Table 1], suggesting that the success of recent anti-smoking campaigns also contributed to this paradigm shift.

Contrary to a previous report addressing changes in several CVD risk factors,22 we used the widely validated Framingham risk model to predict the risk of CVD. By measuring the net change in cardiovascular risk, we more comprehensively estimated the impacts of prevention strategies; this is particularly important because some risk factors worsened, some improved, and some remained stable over time. To our knowledge, this is the first study to evaluate the change in 10-year CVD risk in an Asian population. The results reinforce the need for more aggressive community-wide and clinic-based preventions, with an emphasis on increased exercise hours during leisure time, dietary changes that promote higher intake of vegetables and fruits, as well as lower intake of salt and saturated fats, weight maintenance, and smoking cessation. The findings also suggest that the discovery and use of lipid-lowering drugs and antihypertensive medications does not effectively translate into overall risk reduction in an Asian population, unless these treatments are simultaneously paired with prevention strategies targeting CVD risk factors.

This study was based on two Hong Kong population health surveys, and thus the sample is highly representative of the general population. Baseline data were collected through laboratory tests and face-to-face interviews, suggesting that these data are highly reliable. The long interval between the two health surveys (2003-2005 to 2014-2015) also provides insights regarding the effectiveness of current CVD prevention strategies. Limitations of the current study involve its use of the Framingham risk prediction model and the PHS and HHS. Similar to other surveys, the PHS and HHS are susceptible to participation bias because the sample data might not provide an accurate representation of the overall population. Many PHS variables are self-reported and may lead to reporting bias. Considering the effort required to complete the long questionnaires, the collected data may be susceptible to non-response bias and recall bias. Furthermore, because the Framingham cohort primarily consisted of Caucasians, the predicted 10-year CVD risk in the Hong Kong population calculated using the Framingham model should be interpreted cautiously. The Framingham risk model was developed for people without a history of CVD; thus, a small proportion of the overall sample lacking a history of CVD might have been included in the study, causing inaccuracies in the results. Additionally, the model may overestimate the 10-year CVD risk for men in Hong Kong,14 and a recalibrated model with greater predictive power may be required. Although several limitations and assumptions may hinder the prediction of CVD risk, these potential sources of bias were present in both surveys. Therefore, the effects of such biases may be less important when comparing the change in predicted 10-year CVD risk between the two time points.

During the period from 2003-2005 to 2014-2015, the change in predicted 10-year CVD risk was not statistically significant. However, the proportion of low-risk participants within older age-groups was higher in PHS 2014/15 than in PHS 2003/2004 and HHS 2004/2005. More aggressive CVD prevention strategies and primary care interventions are needed to address CVD risk factors.

Concept or design: BMY Cheung, CLK Lam.
Acquisition of data: BYC Sung, EHM Tang, BMY Cheung, CLK Lam.
Analysis or interpretation of data: BYC Sung, EHM Tang.
Drafting of the manuscript: BYC Sung, EHM Tang.
Critical revision of the manuscript for important intellectual content: L Bedford, CKH Wong, ETY Tse, EYT Yu, BMY Cheung, CLK Lam.

All authors had full access to the data, contributed to the study, approved the final version for publication, and take responsibility for its accuracy and integrity.

As advisers of the journal, CKH Wong and EYT Yu were not involved in the peer review process. Other authors have disclosed no conflicts of interest.

The authors thank the Department of Health of the Hong Kong SAR Government for granting approval to use the data from Population Health Survey 2003/2004, Heart Health Survey 2004/2005, and Population Health Survey 2014/15 for this research. The authors also thank Dr Weinan Dong and Ms Tingting Wu from the Department of Family Medicine and Primary Care of The University of Hong Kong for their assistance in data interpretation and revising the final draft of the manuscript.

Part of this study was presented as a poster at the ESC Congress 2021 – The Digital Experience (virtual), 27-30 August 2021, and was published as an abstract (Sung BY, Tang EH, Bedford L, et al. Change in Framingham cardiovascular disease risk between 2003 and 2014 in the Hong Kong Population Health Survey [abstract]. Eur Heart J 2021;42:1).

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

The requirement for ethics approval for this research was waived by the Institutional Review Board of The University of Hong Kong/Hospital Authority Hong Kong West Cluster, Hong Kong as this study involved secondary analysis of de-identified governmental data.

1. Roth GA, Johnson C, Abajobir A, et al. Global, regional, and national burden of cardiovascular diseases for 10 causes, 1990 to 2015. J Am Coll Cardiol 2017;70:1-25. Crossref

2. GBD 2017 Causes of Death Collaborators. Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018;392:1736-88. Crossref

3. Romero CX, Romero TE, Shlay JC, Ogden LG, Dabelea D. Changing trends in the prevalence and disparities of obesity and other cardiovascular disease risk factors in three racial/ethnic groups of USA adults. Adv Prev Med 2012;2012:172423. Crossref

4. World Health Organization. Hypertension. 2019. Available from: https://www.who.int/news-room/fact-sheets/detail/hypertension. Accessed 8 Jan 2021.

5. Kearney PM, Whelton M, Reynolds K, Muntner P, Whelton PK, He J. Global burden of hypertension: analysis of worldwide data. Lancet 2005;365:217-23. Crossref

6. Lin X, Xu Y, Pan X, et al. Global, regional, and national burden and trend of diabetes in 195 countries and territories: an analysis from 1990 to 2025. Sci Rep 2020;10:14790. Crossref

7. World Health Organization. Cardiovascular diseases. 2016. Available from: https://www.who.int/health-topics/cardiovascular-diseases#tab=tab_1. Accessed 8 Jan 2021.

8. World Health Organization. Prevention of cardiovascular disease. Guidelines for assessment and management of cardiovascular risk. 2007. Available from: https://apps.who.int/iris/bitstream/handle/10665/43685/9789241547178_eng.pdf?sequence=1&isAllowed=y. Accessed 8 Jan 2021.

9. Ajani UA, Ford ES. Has the risk for coronary heart disease changed among U.S. adults? J Am Coll Cardiol 2006;48:1177-82. Crossref

10. Lopez-Jimenez F, Batsis JA, Roger VL, Brekke L, Ting HH, Somers VK. Trends in 10-year predicted risk of cardiovascular disease in the United States, 1976 to 2004. Circ Cardiovasc Qual Outcomes 2009;2:443-50. Crossref

11. Department of Health, Hong Kong SAR Government; Department of Community Medicine, The University of Hong Kong. Population Health Survey 2003/2004 2015. Available from: https://www.chp.gov.hk/files/pdf/report_on_population_health_survey_2003_2004_en.pdf. Accessed 8 Jan 2021.

12. Surveillance and Epidemiology Branch, Centre for Health Protection, Department of Health, Hong Kong SAR Government. Report of Population Health Survey 2014/2015. 2017. Available from: https://www.chp.gov.hk/files/pdf/dh_phs_2014_15_full_report_eng.pdf. Accessed 8 Jan 2021.

13. D’Agostino RB Sr, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117:743-53. Crossref

14. Lee CH, Woo YC, Lam JK, et al. Validation of the Pooled Cohort equations in a long-term cohort study of Hong Kong Chinese. J Clin Lipidol 2015;9:640-6.e2. Crossref

15. World Health Organization. Age standardization of rates: a new WHO standard. 2001. Available from: https://cdn.who.int/media/docs/default-source/gho-documents/global-health-estimates/gpe_discussion_paper_series_paper31_2001_age_standardization_rates.pdf. Accessed 6 May 2024.

16. United States Census Bureau. Introduction to Census 2000 Data Products. Available from: https://usa.ipums.org/usa/resources/voliii/pubdocs/2000/mso-01icdp.pdf. Accessed 1 May 2024.

17. Census and Statistics Department, Hong Kong SAR Government. Hong Kong population projections 2017-2066. 2017. Available from: https://www.censtatd.gov.hk/media_workers_corner/pc_rm/hkpp2017_2066/index.jsp. Accessed 8 Jan 2021.

18. Department of Health, Hong Kong SAR Government. Non-Communicable Diseases Watch. Overview of cardiovascular diseases. September 2018. Available from: https://www.chp.gov.hk/files/pdf/ncd_watch_sep_2018.pdf. Accessed 8 Jan 2021.

19. National Prevention Council, United States Government. National Prevention Strategy. America’s plan for better health and wellness. June 2011. Available from: https://www.hhs.gov/sites/default/files/disease-prevention-wellness-report.pdf. Accessed 8 Jan 2021.

20. Chow CK, Nguyen TN, Marschner S, et al. Availability and affordability of medicines and cardiovascular outcomes in 21 high-income, middle-income and low-income countries. BMJ Glob Health 2020;5:e002640. Crossref

21. Collins GS, Altman DG. Predicting the 10-year risk of cardiovascular disease in the United Kingdom: independent and external validation of an updated version of QRISK2. BMJ 2012;344:e4181. Crossref

22. Carroll MD, Lacher DA, Sorlie PD, et al. Trends in serum lipids and lipoproteins of adults, 1960-2002. JAMA 2005;294:1773-81. Crossref

The evolution of optimal axillary management for breast cancer patients has led to emphasis on the de-escalation of axillary surgery and minimisation of surgical morbidity. Favourable results from the American College of Surgeons Oncology Group (ACOSOG) Z0011 phase 3 randomised clinical trials have redefined the indications for completion axillary lymph node dissection (ALND) in patients with positive sentinel lymph nodes (SLNs). Early-stage breast cancer patients who undergo upfront breast-conserving surgery and have one or two positive SLNs can safely forgo ALND while maintaining good overall survival and disease-free survival.1 2 Consequently, the ASCO (American Society of Clinical Oncology)3 and the NCCN (National Comprehensive Cancer Network)4 have revised their clinical practice guidelines to recommend against completion ALND in this subset of patients. Although this guidance has led to a significant decline in the rate of completion ALND among ACOSOG Z0011–eligible patients,5 6 7 a similar reduction was observed among patients undergoing mastectomy.8 9 This reduction was particularly pronounced among patients with SLN micrometastases.8 Further evidence was obtained in the phase 3 IBCSG (International Breast Cancer Study Group) 23-01 randomised controlled trials, where approximately 10% of patients with SLN micrometastases underwent mastectomy; subgroup analysis demonstrated that disease-free survival among patients without axillary dissection was non-inferior to those with axillary dissection after 10 years of follow-up.10 11 Similarly, in the AMAROS trial (After Mapping of the Axilla: Radiotherapy Or Surgery?), 17% of patients with tumour (T) staging T1 to T2 primary breast cancer underwent mastectomy.12 Axillary radiotherapy led to an oncological outcome comparable to completion ALND but was associated with a lower rate of lymphoedema.

In Hong Kong, factors such as the relatively small breast sizes among Chinese women13 and more conservative cultural attitudes13 14 have contributed to a higher rate of mastectomy. The decision to perform mastectomy has prevented a substantial number of breast cancer patients from meeting the ACOSOG Z0011 criteria. Our previous study evaluated the applicability of ACOSOG Z0011 criteria in Hong Kong.15 Patients with clinical nodal (N) staging N0 breast cancer and one or more positive SLNs were stratified into eligible and ineligible groups according to the ACOSOG Z0011 criteria, with 93% of patients in the ineligible group underwent mastectomy.15 Importantly, only 24% of patients in that study met the ACOSOG Z0011 criteria and could potentially avoid ALND.15 Therefore, it is important to identify a low-risk subset of SLN-positive mastectomy patients who could benefit from this non-ALND approach. This retrospective study was conducted to compare the clinical characteristics of SLN-positive breast cancer patients in Hong Kong with the ACOSOG Z0011–eligible cohort.

This retrospective analysis of a prospectively maintained database was conducted at Queen Mary Hospital, a university-affiliated tertiary breast cancer centre in Hong Kong, from June 2014 to May 2019. Potentially eligible patients in the database were identified by an independent research assistant according to whether they met the ACOSOG Z0011 criteria, irrespective of breast surgery type. Patients were excluded if they had positive non-SLNs or positive SLNs only detected by immunohistochemical staining. Relevant data were extracted in July 2020 and missing information was verified using the Clinical Management System, a central computer system for medical records across public hospitals in Hong Kong. Recruited patients were divided into two groups, namely, the mastectomy group and the breast-conserving treatment (ie, ACOSOG Z0011–eligible) group.

All breast cancer patients underwent mammography and ultrasound of the breasts and axillae for clinical tumour and nodal staging. Sentinel lymph node biopsy (SLNB), offered to patients with clinically node-negative disease, was performed with a dual tracer of radioisotope and patent blue dye. Sentinel lymph nodes were defined as lymph nodes with ex vivo gamma probe counts exceeding 10% of the highest ex vivo reading or lymph nodes that displayed blue staining. Non-SLNs were defined as suspicious nodes that were neither hot (high gamma probe counts) nor blue-stained during SLNB, or nodes that were removed during completion ALND. During the study period, intraoperative frozen sections of SLNs or suspicious non-SLNs were routinely collected; these were analysed by standard haematoxylin and eosin staining. Immunohistochemistry was performed in cases of suspected nodal metastasis. Completion ALND was conducted if frozen or paraffin sections showed evidence of nodal metastasis. All final pathological results were reviewed in multidisciplinary meetings. The pathologies of SLNs were considered normal or containing one of the following: macrometastases (>2 mm), micrometastases (>0.2 to ≤2 mm), or isolated tumour cells (≤0.2 mm). For patients undergoing breast-conserving surgery, ‘no ink on tumour’ was regarded as an adequate resection margin16; alternatively, a second operation was performed to ensure a clear resection margin. Adjuvant treatment was administered by breast oncology specialists according to decisions made in multidisciplinary meetings.

Patient demographic characteristics and tumour characteristics were retrieved from database records; percentages were calculated. Missing information was evaluated and managed by pairwise deletion. Comparisons were made between the mastectomy and breast-conserving treatment groups in our cohort, along with the sentinel-alone arm in the ACOSOG Z0011 trial (n=436, in intention to treat).1 2 Analyses followed the per-protocol approach and calculations were performed with SPSS software (Windows version 24.0; IBM Corp, Armonk [NY], United States). Comparisons between cohorts were conducted with Student’s t test or the Chi squared test, as appropriate. Human epidermal growth factor receptor 2 (HER2) status was not assessed in the ACOSOG Z0011 study; therefore, HER2 statuses were only compared within our cohort. The Memorial Sloan Kettering Cancer Center (MSKCC) breast cancer nomogram,17 a well-validated prediction tool to assess the likelihood of non–sentinel node metastases18 19 20 (including external validation in the Chinese population19 20), was used to calculate probability through an online calculator that considered nine variables; comparisons were made between the breast-conserving treatment and mastectomy groups. P values <0.05 were considered statistically significant.

In our centre, the ACOSOG Z0011 criteria have been used to manage patients undergoing breast-conserving surgery since June 2019. From June 2014 to May 2019, 1249 breast cancer patients underwent SLNB in our institution; 171 patients (13.7%) met the study inclusion criteria of clinical T1 or T2 invasive breast cancer and one or two positive SLNs. One hundred and twelve patients (65.5%) underwent mastectomy and 59 patients (34.5%) underwent breast-conserving treatment. The median follow-up period was 58 months (range, 25-84).

Patient demographic characteristics and tumour characteristics of our mastectomy group and the sentinel-alone arm in the ACOSOG Z0011 trial are presented in Table 1. Invasive ductal carcinoma was more common in our patient population than in the ACOSOG Z0011 group. A higher prevalence of clinical T2 breast cancers (~50%) was observed in our mastectomy group (P<0.001). There were also significantly more patients with lymphovascular invasion in our cohort than in the sentinel-alone arm in the ACOSOG Z0011 trial (P<0.001). Although nearly half of the original ACOSOG Z0011 cohort had micrometastatic SLNs, approximately 70% of mastectomy patients had macrometastatic SLNs (P=0.004). These findings suggested that the clinicopathological profile was more aggressive in patients requiring mastectomy.

In our patient cohort, the mastectomy group exhibited many clinicopathological characteristics similar to the breast-conserving treatment group (Table 2). There were no statistically significant differences in terms of age, tumour grade, lymphovascular invasion status, oestrogen receptor/progesterone receptor status, or HER2 status. The mastectomy group had relatively larger tumours than the breast-conserving treatment group (mean: 2.2 cm vs 1.8 cm; P=0.005). Although the difference was not statistically significant, the mastectomy group tended to have larger proportions of patients with two metastatic SLNs (24.1% vs 13.6%; P=0.1) and SLN macrometastases (70.5% vs 57.6%; P=0.11) than the breast-conserving treatment group. Furthermore, the MSKCC probability for additional metastatic non-SLNs was slightly higher in the mastectomy group than in the breast-conserving treatment group (37.1% vs 31.4%; P=0.03) [Table 2].

Ninety-seven patients (86.6%) in the mastectomy group and 45 patients (76.3%) in the breast-conserving treatment group underwent completion ALND. Among patients who underwent mastectomy and completion ALND, 26 patients (26.8%) had additional non-SLN metastases (range, 1-18). In contrast, eight patients (17.8%) in the breast-conserving treatment group had additional non-SLN metastases (range, 1-8). There was no statistically significant difference in the rate of non-SLN metastases between the two treatment arms (P=0.24). Twenty-nine patients (17.9%) underwent SLNB alone; 15 of these patients were in the mastectomy group. Most patients with SLNB alone had micrometastatic SLNs (89.7%) and one patient had isolated tumour cells. None of the patients with SLNB alone experienced recurrence.

In the mastectomy group, 97 patients (86.6%) underwent post-mastectomy irradiation targeting the chest wall and third field regional nodes. Third field regional nodes refer to level III axillary and supraclavicular lymph node regions. None of these patients developed chest wall or axillary recurrence during the follow-up period. Among the 15 patients who did not undergo post-mastectomy irradiation, eight (53.3%) had micrometastatic SLNs and six (40.0%) had macrometastatic SLNs. There were two recurrences (13.3%). First, a 38-year-old patient with one macrometastatic SLN developed ipsilateral chest wall recurrence 4 years after the index operation; this recurrence was managed by a second operation. Second, a patient with two macrometastatic SLNs refused adjuvant systemic treatment and died of breast cancer&dash;related distant metastases. One hundred and ten patients in the mastectomy group (98.2%) received adjuvant systemic treatment: 10 patients (8.9%) required chemotherapy only, 22 patients (19.6%) required hormonal treatment only, and 78 patients (69.6%) required both of these treatments. Seven patients (6.3%) in the mastectomy group developed distant recurrence, and there were three (2.7%) breast cancer–related deaths.

In the breast-conserving treatment group, 58 of the 59 patients underwent adjuvant whole-breast irradiation; 61.0% of these patients underwent additional third field nodal irradiation. Fifty-eight patients (98.3%) in the breast-conserving treatment group received adjuvant systemic treatment involving hormonal therapy and/or chemotherapy. Three patients (5.1%) had distant recurrence; among them, one (1.7%) died at 39 months after the initial diagnosis. One patient experienced ipsilateral breast recurrence at 30 months and underwent completion mastectomy.

The favourable oncological results of the ACOSOG Z0011 trial1 2 have challenged the conventional approach of performing completion ALND in patients with SLN metastases. Patients with one or two SLN metastases who underwent breast-conserving surgery, whole-breast irradiation, and adjuvant systemic treatment could safely forgo completion ALND. This paradigm shift has led to substantial de-escalation of axillary surgery worldwide.5 A meta-analysis by Schmidt-Hansen et al,21 which involved 2020 patients and findings from the IBCSG 23-0110 11 and the AATRM (Agència d’Avaluació de Tecnologia i Recerca Mèdiques) 048/13/200022 trials, concluded that SLNB alone was sufficient for locoregional control in early breast cancer, without adverse effects on survival.

Despite widespread adoption of the ACOSOG Z0011 criteria, the study has been criticised in several ways. The low locoregional relapse rate of 1.5% indicates that the study was underpowered.23 Furthermore, significant deviation in the radiotherapy protocol, such that 18.9% of patients received ‘high tangents’ radiotherapy, has raised questions concerning the oncological safety of SLNB alone in patients without third field nodal irradiation.24 Combined with the insufficient numbers of mastectomy patients in the IBCSG 23-01,10 11 AMAROS,12 and AATRM 048/13/200022 trials, it has been unclear whether this non-ALND approach can be extrapolated to SLN-positive breast cancer patients who undergo mastectomy with or without radiotherapy.

In this study, we compared the clinicopathological characteristics among our mastectomy group, our breast-conserving treatment group, and the sentinel-alone arm in the original ACOSOG Z0011 study. Unsurprisingly, our mastectomy group exhibited more aggressive tumour characteristics than the sentinel-alone arm in the Western population; specifically, it had a larger tumour size, more frequent lymphovascular invasion, and a greater proportion of patients with SLN macrometastases. These differences in clinicopathological features have also been reported in Western populations. For example, Hennigs et al8 analysed a large German cohort that included 4093 SLN-positive mastectomy patients. Compared with the entire study cohort of 166 074 patients, T2 tumour and lymphovascular invasion were more commonly found in patients requiring mastectomy. Additionally, the study by Milgrom et al25 included 535 early-stage breast cancer patients with a positive SLNB and no ALND. In their mastectomy group, patients had significantly larger tumours and more frequently displayed multifocal/multicentric disease. However, these adverse pathological features among mastectomy patients did not justify a more aggressive axillary approach. Similarly, the low rates of local and regional failure observed in our cohort were consistent with previous reports, suggesting that axillary-specific treatment can be considered in this group of patients with low-volume SLN disease.25 26 27 28 Debate persists regarding the comparatively large proportions of patients with micrometastatic disease in the original ACOSOG Z0011 trial1 2 and other studies.25 26 Cowher et al29 published a retrospective analysis of patients who underwent mastectomy and conservative axillary regional excision (ie, removal of SLNs and other palpable nodes). Among 144 patients with pathological N1 disease, a small proportion (24%) had micrometastatic disease; only three axillary recurrences (2.1%) were reported.29 Notably, the low locoregional failure rate was not attributed to post-mastectomy irradiation25 26 27 28 29 or increased use of chemotherapy.26 27 28

In our previous study, we demonstrated differences in clinical characteristics between Asian and Western populations.15 In the present study, our breast-conserving treatment group had a higher rate of clinical T2 tumours and more frequent lymphovascular invasion compared with the Western population. Similar findings were observed in Korean30 and Japanese31 studies, which revealed larger and higher-grade tumours, increased lymphovascular permeation, and more frequent SLN macrometastases. Despite these disparities, the Korean30 and Japanese31 studies both demonstrated safe application of ACOSOG Z0011 criteria in Asia, with low incidences of disease recurrence. These intrinsic differences in tumour characteristics between Eastern and Western populations have presumably reduced the gap in clinicopathological features between patients undergoing mastectomy and those undergoing breast-conserving surgery. In the head-to-head comparison between our mastectomy cohort and our breast-conserving treatment group, the only notable difference involved the mean pathological size of the invasive focus (2.2 cm vs 1.8 cm; P=0.005); the clinical tumour stage distribution did not differ (P=0.69) [Table 2]. The small difference in mean MSKCC breast cancer nomogram probability (37.1% vs 31.4%; P=0.03) could also be related to the difference in pathological size, which is one of the nine variables considered in the nomogram. Therefore, we believe that a non-ALND approach in this low-risk subset of SLN-positive mastectomy patients is acceptable.

The primary concern regarding extrapolation of this non-ALND approach is the risk of undertreatment for patients with an extensive nodal burden. The original ACOSOG Z0011 trial revealed a non-SLN macrometastasis rate of 27.3% in the ALND group.1 2 The AMAROS trial also showed that 33% of patients in the ALND group had additional positive lymph nodes.12 Importantly, the axillary recurrence rate remained low in both of these studies. In our SLN-positive mastectomy and breast-conserving treatment groups, the proportions of patients with additional non-SLN metastases were 26.8% and 17.8%, respectively. Among patients undergoing adjuvant irradiation and adjuvant systemic treatment, it is likely that some non-SLN metastases do not progress to clinically detectable disease.

This study had several limitations. First, its retrospective design could result in recall bias and the potential for missing clinical information. Although data from the ACOSOG Z0011 trial were limited with respect to HER2 status, extracapsular extension, and multifocality, we attempted to mitigate this issue by including some of the affected variables in the comparison of our mastectomy and breast-conserving treatment groups. Second, we could not address the need for post-mastectomy irradiation among patients in this study. The value of such irradiation for breast cancer patients with <4 positive lymph nodes remains controversial. The meta-analysis by the Early Breast Cancer Trialists’ Collaborative Group,32 which included 1314 breast cancer patients with one to three positive nodes after mastectomy and ALND, suggested that radiotherapy provided oncological benefit in terms of locoregional recurrence, overall recurrence, and breast cancer mortality. However, this meta-analysis has been criticised for including some very early studies from the 1970s, in which the reported recurrence rates were much higher than rates in later studies. In 2016, a focused update by the American Society of Clinical Oncology, American Society for Radiation Oncology, and Society of Surgical Oncology acknowledged the use of post-mastectomy radiotherapy for this group of patients but recommended clinical judgement for patients with a low risk of locoregional recurrence.33 In our centre, post-mastectomy irradiation was generally administered to patients with pathological N1 disease during the study period; 86.6% of patients in the present study underwent adjuvant radiotherapy. Considering the similarities in clinicopathological features and adjuvant systemic treatment use between our SLN-positive mastectomy and breast-conserving treatment groups, we suspect that it is safe for selected low-risk SLN-positive mastectomy patients to forgo ALND through the expansion of AMAROS eligibility12 to ACOSOG Z0011–ineligible patients. Several ongoing randomised studies, such as the English POSNOC (POsitive Sentinel NOde: adjuvant therapy alone versus adjuvant therapy plus Clearance or axillary radiotherapy)34 and the Dutch BOOG 2013-07,35 are recruiting breast cancer patients who undergo mastectomy and have a maximum of two to three positive SLNs; these studies aim to compare completion axillary treatment (ALND or axillary radiotherapy) and the lack of completion axillary treatment. Additionally, the SINODAR-ONE trial36 recently published their subgroup analysis and found non-inferior overall survival and recurrence-free survival among mastectomy patients receiving SLNB and ALND. The ongoing studies are expected to provide more robust evidence concerning the optimal treatment for SLN-positive mastectomy patients.

This study demonstrated the clinicopathological similarities between SLN-positive mastectomy and breast-conserving treatment groups among breast cancer patients in Hong Kong. Cautious application of the non-ALND approach in mastectomy patients with low-volume SLN disease is reasonable, considering the low locoregional recurrence rate. However, additional research is needed to standardise the adjuvant post-mastectomy radiotherapy protocol, especially among patients who forego ALND.

Concept or design: V Man.
Acquisition of data: V Man.
Analysis or interpretation of data: V Man.
Drafting of the manuscript: Both authors.
Critical revision of the manuscript for important intellectual content: A Kwong.

Both authors had full access to the data, contributed to the study, approved the final version for publication, and take responsibility for its accuracy and integrity.

Both authors have disclosed no conflicts of interest.

This study has been presented and awarded the Young Investigator Award Best Scientific Paper in the Hong Kong Society of Breast Surgeons 5th Annual Scientific Meeting (19 September 2021, Hong Kong).

This study received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

This study was approved by the Institutional Review Board of The University of Hong Kong/Hospital Authority Hong Kong West Cluster, Hong Kong (Ref No.: HKU/HA HKW UW 09-045). Written informed consent was obtained from patients for all treatments, procedures, and publication.

1. Giuliano AE, Hunt KK, Ballman KV, et al. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA 2011;305:569-75. Crossref

2. Guiliano AE, Ballman KV, McCall L, et al. Effect of axillary dissection vs no axillary dissection on 10-year overall survival among women with invasive breast cancer and sentinel node metastasis: the ACOSOG Z0011 (Alliance) randomized clinical trial. JAMA 2017;318:918-26. Crossref

3. Lyman GH, Termin S, Edge SB, et al. Sentinel lymph node biopsy for patients with early-stage breast cancer: American Society of Clinical Oncology clinical practice guideline update. J Clin Oncol 2014;32:1365-83. Crossref

4. Gradishar WJ, Anderson BO, Balassanian R, et al. NCCN guidelines insights: breast cancer, version 1.2017. J Natl Compr Canc Netw 2017;15:433-51. Crossref

5. Camp MS, Greenup RA, Taghian A, et al. Application of ACOSOG Z0011 criteria reduces perioperative costs. Ann Surg Oncol 2013;20:836-41. Crossref

6. Delpech Y, Bricou A, Lousquy R, et al. The exportability of the ACOSOG Z0011 criteria for omitting axillary lymph node dissection after positive sentinel lymph node biopsy findings: a multicenter study. Ann Surg Oncol 2013;20:2556-61. Crossref

7. Verheuvel NC, Voogd AC, Tjan-Heijnen VC, Roumen RM. Potential impact of application of Z0011 derived criteria to omit axillary lymph node dissection in node positive breast cancer patients. Eur J Surg Oncol 2016;42:1162-8. Crossref

8. Hennigs A, Riedel F, Feißt M, et al. Evolution of the use of completion axillary lymph node dissection in patients with T1/2N0M0 breast cancer and tumour-involved sentinel lymph nodes undergoing mastectomy: a cohort study. Ann Surg Oncol 2019;26:2435-43. Crossref

9. Poodt IG, Spronk PE, Vugts G, et al. Trends on axillary surgery in nondistant metastatic breast cancer patients treated between 2011 and 2015: a Dutch population-based study in the ACOSOG-Z0011 and AMAROS era. Ann Surg 2018;268:1084-90. Crossref

10. Galimberti V, Cole BF, Zurrida S, et al. Axillary dissection versus no axillary dissection in patients with sentinel-node micrometastases (IBCSG 23-01): a phase 3 randomised controlled trial. Lancet Oncol 2013;14:297-305. Crossref

11. Galimberti V, Cole BF, Viale G, et al. Axillary dissection versus no axillary dissection in patients with breast cancer and sentinel-node micrometastases (IBCSG 23-01): 10-year follow-up of a randomised, controlled phase 3 trial. Lancet Oncol 2018;19:1385-93. Crossref

12. Donker M, van Tienhoven G, Straver ME, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981-22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol 2014;15:1303-10. Crossref

13. Sinnadurai S, Kwong A, Hartman M, et al. Breast-conserving surgery versus mastectomy in young women with breast cancer in Asian settings. BJS Open 2018;3:48-55. Crossref

14. Chan SW, Cheung C, Chan A, Cheung PS. Surgical options for Chinese patients with early invasive breast cancer: data from the Hong Kong Breast Cancer Registry. Asian J Surg 2017;40:444-52.Crossref

15. Man V, Lo MS, Kwong A. The applicability of the ACOSOG Z0011 criteria to breast cancer patients in Hong Kong. Chin Clin Oncol 2021;10:27. Crossref

16. Moran MS, Schnitt SJ, Giuliano AE, et al. Society of Surgical Oncology–American Society for Radiation Oncology consensus guideline on margins for breast-conserving surgery with whole-breast irradiation in stages I and II invasive breast cancer. Ann Surg Oncol 2014;21:704-16. Crossref

17. Memorial Sloan Kettering Cancer Center. Breast cancer nomogram: breast additional non-SLN metastases. Available from: https://nomograms.mskcc.org/breast/BreastAdditionalNonSLNMetastasesPage.aspx. Accessed 19 Mar 2024.

18. Gur AS, Unal B, Ozbek U, et al. Validation of breast cancer nomograms for predicting the non–sentinel lymph node metastases after a positive sentinel lymph node biopsy in a multi-center study. Eur J Surg Oncol 2010;36:30-5. Crossref

19. Kuo YL, Chen WC, Yao WJ, et al. Validation of Memorial Sloan-Kettering Cancer Center nomogram for prediction of non–sentinel lymph node metastasis in sentinel lymph node positive breast cancer patients: an international comparison. Int J Surg 2013;11:538-43. Crossref

20. Bi X, Wang Y, Li M, et al. Validation of the Memorial Sloan Kettering Cancer Center nomogram for predicting non–sentinel lymph node metastasis in sentinel lymph node–positive breast-cancer patients. Onco Targets Ther 2015;8:487-93. Crossref

21. Schmidt-Hansen M, Bromham N, Hasler E, Reed MW. Axillary surgery in women with sentinel node–positive operable breast cancer: a systematic review with meta-analyses. Springerplus 2016;5:85. Crossref

22. Solá M, Alberro JA, Fraile M, et al. Complete axillary lymph node dissection versus clinical follow-up in breast cancer patients with sentinel node micrometastasis: final results from the multicenter clinical trial AATRM 048/13/2000. Ann Surg Oncol 2013;20:120-7. Crossref

23. Huang TW, Kuo KN, Chen KH, et al. Recommendation for axillary lymph node dissection in women with early breast cancer and sentinel node metastasis: a systematic review and meta-analysis of randomized controlled trials using the GRADE system. Int J Surg 2016;34:73-80. Crossref

24. Jagsi R, Chadha M, Moni J, et al. Radiation field design in the ACOSOG Z0011 (Alliance) trial. J Clin Oncol 2014;32:3600-6. Crossref

25. Milgrom S, Cody H, Tan L, et al. Characteristics and outcomes of sentinel node–positive breast cancer patients after total mastectomy without axillary-specific treatment. Ann Surg Oncol 2012;19:3762-70. Crossref

26. FitzSullivan E, Bassett RL, Kuerer HM, et al. Outcomes of sentinel lymph node–positive breast cancer patients treated with mastectomy without axillary therapy. Ann Surg Oncol 2017;24:652-9. Crossref

27. Snow R, Reyna C, Johns C, et al. Outcomes with and without axillary node dissection for node-positive lumpectomy and mastectomy patients. Am J Surg 2015;210:685-93. Crossref

28. Joo JH, Kim SS, Son BH, et al. Axillary lymph node dissection does not improve post-mastectomy overall or disease-free survival among breast cancer patients with 1-3 positive nodes. Cancer Res Treat 2019;51:1011-21. Crossref

29. Cowher MS, Grobmyer SR, Lyons J, O’Rourke C, Baynes D, Crowe JP. Conservative axillary surgery in breast cancer patients undergoing mastectomy: long-term results. J Am Coll Surg 2014;218:819-24. Crossref

30. Jung J, Han W, Lee ES, et al. Retrospectively validating the results of the ACOSOG Z0011 trial in a large Asian Z0011-eligible cohort. Breast Cancer Res Treat 2019;175:203-15. Crossref

31. Kittaka N, Tokui R, Ota C, et al. A prospective feasibility study applying the ACOSOG Z0011 criteria to Japanese patients with early breast cancer undergoing breast-conserving surgery. Int J Clin Oncol 2018;23:860-6. Crossref

32. EBCTCG (Early Breast Cancer Trialists’ Collaborative Group); McGale P, Taylor C, et al. Effect of radiotherapy after mastectomy and axillary surgery on 10-year recurrence and 20-year breast cancer mortality: meta-analysis of individual patient data for 8135 women in 22 randomised trials. Lancet 2014;383:2127-35. Crossref

33. Recht A, Comen EA, Fine RE, et al. Postmastectomy radiotherapy: an American Society of Clinical Oncology, American Society for Radiation Oncology, and Society of Surgical Oncology focused guideline update. Pract Radiat Oncol 2016;6:e219-34. Crossref

34. Goyal A, Dodwell D. POSNOC: a randomised trial looking at axillary treatment in women with one or two sentinel nodes with macrometastases. Clin Oncol (R Coll Radiol) 2015;27:692-5. Crossref

35. van Roozendaal LM, de Wilt JH, van Dalen T, et al. The value of completion axillary treatment in sentinel node positive breast cancer patients undergoing a mastectomy: a Dutch randomized controlled multicentre trial (BOOG 2013-07). BMC Cancer 2015;15:610. Crossref

36. Tinterri C, Canavese G, Gatzemeier W, et al. Sentinel lymph node biopsy versus axillary lymph node dissection in breast cancer patients undergoing mastectomy with one to two metastatic sentinel lymph nodes: sub-analysis of the SINODAR-ONE multicentre randomized clinical trial and reopening of enrolment. Br J Surg 2023;110:1143-52. Crossref

Stevens–Johnson syndrome (SJS) and toxic epidermal necrolysis (TEN) are uncommon but potentially life-threatening severe mucocutaneous reactions characterised by extensive epidermal necrosis and detachment. Both entities are considered variants of a single disease continuum and are classified according to the percentage of body surface area (BSA) with epidermal detachment.1 2 Although SJS and TEN (hereafter, SJS/TEN) have been described in all ethnicities worldwide,3 studies of these reactions in Hong Kong have been limited.4 5 The incidence, clinical characteristics, aetiology, treatment regimen, morbidity, and mortality in the territory are largely unknown. This pilot study aimed to review cases of SJS/TEN over a 17-year period at a tertiary referral centre in Hong Kong, and to aid future research in Hong Kong focused on severe cutaneous adverse reactions.

This retrospective cohort study included all hospitalised patients who had been diagnosed with SJS/TEN and were treated in Prince of Wales Hospital (PWH), a major regional hospital under the New Territories East Cluster (NTEC), from 1 January 2004 to 31 December 2020.

Patients with clinical and histological diagnoses of SJS/TEN were identified from the Hospital Authority database using International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis codes and the database of the Department of Anatomical and Cellular Pathology of PWH, respectively.

Diagnoses of SJS/TEN were based on consensus guidelines.1 Patients were diagnosed with SJS, SJS/TEN overlap, and TEN when they exhibited epidermal detachment levels of <10%, 10% to 30%, and >30%, respectively, with consistent histological features (if skin biopsy was performed). Consistent histological features were regarded as partial- to full-thickness epidermal necrosis.

Patients were excluded if they had an alternative diagnosis, such as severe cutaneous adverse reactions other than SJS/TEN (eg, drug reaction with eosinophilia and systemic symptoms syndrome/acute generalised exanthematous pustulosis/generalised bullous fixed drug eruption), erythema multiforme major, autoimmune blistering disease, acute graft-versus-host disease, and infections such as staphylococcal scalded skin syndrome.

Clinical characteristics were collected from electronic records and, when available, hospital case notes. The following clinical characteristics were recorded and analysed: age at onset, sex, ethnicity, maximal BSA of detached or detachable skin, SCORTEN (Severity-of-Illness Score for Toxic Epidermal Necrolysis) prognostic score,6 mucosa involved, histology results if available, causative drugs, time from exposure to onset, time from onset to admission and treatment, treatment regimen, disease complications, mortality and its cause, and length of stay. Efforts to identify causative drugs were guided by the ALDEN (algorithm of drug causality for epidermal necrolysis) score,7 which was retrospectively calculated by two independent investigators. All clinical data were expressed as percentages or means ± standard deviations unless otherwise specified.

This article was written in compliance with the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) reporting guidelines.

Using the International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis codes for SJS/TEN, 164 potential patients with 166 cases of SJS/TEN during the period from January 2004 to December 2020 were initially identified. Six additional patients with SJS/TEN were identified from the database of the Department of Anatomical and Cellular Pathology of PWH. Forty-seven patients were excluded due to alternative diagnoses. In total, 123 patients with 125 cases of SJS/TEN were included in the study (Fig).

Among the 123 patients with SJS/TEN, 53 were men and 70 were women; the female-to-male ratio was 1.32:1 (Table 1). The mean age at onset was 51.4 years, and most patients were Chinese. Of the 125 cases, 59 were SJS, 27 were SJS-TEN overlap, and 39 were TEN. A small number of patients (n=18, 14.4%) were admitted for other medical issues but developed SJS/TEN after hospitalisation.

The mean time from disease onset to hospitalisation was 4.9 days (Table 2). Fever was present on admission in 88 cases (70.4%). The mean maximal BSA of epidermal detachment was 23%. Mucosal involvement was common; only five cases (4.0%) lacked mucosal involvement. Skin biopsy was performed in 87 cases (69.6%) and the mean SCORTEN prognostic score was 2.17.

Burns unit or intensive care unit admission was required in 40 cases (32.0%) and half of these cases required invasive mechanical ventilation. In total, 63 cases (50.4%) involved concomitant infection from various sources. Multiorgan failure or disseminated intravascular coagulation occurred in 29 cases (23.2%). The mean length of stay in the hospital was 23.9 days (Table 2).

Stevens–Johnson syndrome and TEN onset was attributed to a drug in 114 of 124 cases (91.9%); one patient developed a second case of SJS/TEN upon accidental re-exposure to the same culprit drug (paracetamol). Four cases (3.2%) were caused by infection, and no cause was identified in six cases (4.8%). The identified culprit drugs are shown in Table 3. The mean time from initiation of the culprit drug to onset of SJS/TEN was 20.5 ± 16.7 days (range, 1-87; median, 15.5).

All patients received the best supportive medical care available. In some cases, patients received additional treatment. The numbers and proportions of cases treated with different regimens are shown in Table 4. The mean time between disease onset and active treatment initiation was 7.4 ± 6.1 days (range, 1-34; median, 6).

Intravenous immunoglobulin (IVIG) was administered in 54 cases. The mean total dose of IVIG was 3.2 g/kg (range, 1.5-6; administered over 2-6 days). High-dose IVIG, defined as ≥3 g/kg, was administered in 40 cases. Systemic steroid regimens considerably varied, with daily doses of prednisolone ranging from 20 to 120 mg (or an equivalent dose). The cyclosporine regimen was 3 mg/kg/day, tapered over 20 to 30 days.

There were 27 deaths in the study cohort, and the overall mortality rate was 21.6%. The mean time from SJS/TEN onset to death was 23.6 ± 19.0 days (range, 5-76). Most patients died from multiorgan failure and/or fulminant sepsis (n=22, 81.5%); other causes of death were acute coronary syndrome (n=2), liver failure (n=1), and sudden cardiac arrest (n=2).

The observed mortality rates were 16.9%, 22.2%, and 28.2% for SJS, SJS-TEN overlap, and TEN, respectively. The SCORTEN-based predicted mortality rates were 14.1%, 28.5%, and 36.9% for SJS, SJS-TEN overlap, and TEN, respectively. Inpatient-onset SJS/TEN had a high mortality rate of 77.8%: 14 deaths among 18 patients who developed SJS/TEN after admission.

Stevens–Johnson syndrome and TEN are recognised worldwide, with several epidemiological studies conducted in Europe and the US. In the 1990s, Roujeau et al8 reported that the annual incidence of TEN in France was 1.2 cases per million; during the same period, the estimated overall annual incidences of SJS/TEN were 1.89 cases per million in Germany9 and 4.2 cases per million in the US.10 In the past decade, two large epidemiological studies in the US11 and United Kingdom12 revealed that the overall annual incidences of SJS/TEN were 12.7 and 5.76 cases per million, respectively. In contrast, the epidemiology of SJS/TEN in Asia is not well-documented.13 In Singapore, based on a small retrospective hospital-based study of 20 patients with TEN, the estimated annual incidence of TEN was 1.4 cases per million14; in Korea, a large population-based study indicated that the overall annual incidence of SJS/TEN was 4.9 to 6.5 cases per million.15 In the present study, there were 125 cases of SJS/TEN during the 17-year study period. Notably, 13 of these cases were transferred from hospitals outside of the NTEC: one was SJS, three were SJS/TEN overlap, and nine were TEN. The NTEC serves a population of 1.3 million.16 The estimated annual incidence of TEN alone and combined annual incidence of SJS, SJS-TEN overlap, and TEN were 1.36 and 5.07 cases per million, respectively; these incidences are comparable with findings from studies in other countries.

Stevens–Johnson syndrome is approximately threefold more common than TEN.15 17 However, in the current study, fewer than half of the cases (47.2%) were SJS, whereas 31.2% were TEN. This may be related to referral bias, whereby more severe cases were transferred to the study hospitals, whereas ‘milder’ cases were managed in regional hospitals where the patients were initially admitted. Prince of Wales Hospital is a tertiary referral centre and one of the few hospitals in Hong Kong with both on-site dermatologists and burns unit support. In our cohort, 31 cases (24.8%) were transferred from peripheral hospitals: 18 (14.4%) arrived from hospitals within the NTEC, whereas 13 (10.4%) arrived from hospitals outside of the NTEC.

Stevens–Johnson syndrome and TEN are most often drug-induced, and a culprit drug is identified in approximately 85% of cases.7 18 The reactions usually occur between 7 days and 8 weeks after drug ingestion.19 However, upon rechallenge with the culprit drug, SJS/TEN can develop within hours.17 19 Efforts to identify causative drugs were guided by the ALDEN score.7 In cases of SJS/TEN, the most commonly implicated high-risk medications are anticonvulsants, allopurinol, antimicrobials, and oxicam non-steroidal anti-inflammatory drugs.19 20 In the present study, SJS/TEN onset was attributed to a drug in 114 of 124 cases (91.9%). The mean time between drug initiation and SJS/TEN onset was 20.5 days. Among these 114 cases, 81.6% were caused by the high-risk medications listed above. These findings are comparable with previous reports.

Stevens–Johnson syndrome and TEN are associated with high mortality rates, with 1% to 5% in cases of SJS and 25% to 30% in cases of TEN. Survival analyses in multinational European studies (EuroSCAR [European Study of Severe Cutaneous Adverse Reactions] and RegiSCAR [Registry of Severe Cutaneous Adverse Reactions]) have indicated that the overall mortality rate in cases of SJS/TEN is approximately 22% to 23%.18 20 21 22 23 In Asia, reported overall mortality rates vary from 12.3% to 25%.24 25 26 27 Sepsis leading to multiorgan failure is the most common cause of death.21 Despite the substantial mortality, there currently is no therapeutic regimen with a clear benefit for patients with SJS/TEN.18 21 Considering the rarity of these diseases, it is difficult to conduct randomised trials. Early recognition, rapid withdrawal of offending agents, and best supportive care remain the primary components of clinical management.

In the current study, the overall mortality rate was 21.6%; in 81.5% of these cases, the patient died of fulminant sepsis or multiorgan failure. These findings are consistent with existing literature. However, the mortality rate in cases of SJS was much higher in the present study than in previous studies. In the 59 cases of SJS, there were 10 deaths; the mortality rate was 16.9%. Among the 10 patients who died, six experienced complete skin re-epithelisation before death from other medical conditions, which include massive duodenal ulcer bleeding, acute coronary syndrome, metastatic lung cancer, acute liver and renal failure due to herbs, aspiration pneumonia, and sudden cardiac arrest. The remaining four patients had inpatient-onset SJS; they were initially admitted for traumatic intracranial haemorrhage, post-hepatectomy liver failure, convulsions caused by metastatic lung cancer, and post-stroke seizure, respectively. These patients exhibited skin-specific improvements but soon died of aspiration pneumonia and acute renal failure, liver failure, metastatic lung cancer with respiratory failure and liver failure, and sudden cardiac arrest, respectively. The high mortality rate among patients with SJS in the present study could be related to referral bias (as noted in the Epidemiology subsection above); specifically, more severe cases of SJS with co-morbidities and/or complications may have been transferred to our tertiary centre for medical care, whereas less severe cases of SJS might have been managed in regional hospitals where the patients were initially admitted. Indeed, the predicted mortality rate (based on the SCORTEN prognostic score) among cases of SJS in our cohort was 14%; this rate was similar to the observed mortality rate.

In the present study, inpatient-onset SJS/TEN had a high mortality rate (77.8%). Although high mortality of inpatient-onset SJS/TEN was not previously described in the literature, we speculate that this high mortality was related to the underlying medical conditions for which patients were initially admitted. The clinical characteristics of the 14 patients who died are presented in Table 5.

In addition to high mortality, SJS/TEN were associated with high rates of burns unit/intensive care unit admission (32%) and prolonged length of stay (mean=23.9 days) [Table 2], placing a considerable burden on the public healthcare system.

As a retrospective cohort study, the present study had some intrinsic limitations. Some hospital case notes (ie, from earlier in the study period) were no longer retrievable. Clinical characteristics such as the exact date of disease onset, precise total BSA involved, and detailed drug history (including over-the-counter medications/medications prescribed by private doctors) might not have been available for some of these older cases. Skin biopsies were performed in 70% of cases and might have been omitted in cases of terminal illness. Many patients with milder cases were lost to follow-up after discharge; thus, long-term sequelae were not well-documented.

Additionally, referral bias may have been present because PWH is a tertiary referral centre. Such bias could have led to underestimation of the true incidence of SJS and overestimation of the incidence of TEN; milder cases of SJS might have been managed in regional hospitals, whereas more severe cases of TEN were transferred to our centre for better care. Similarly, there may have been overestimation of various outcome measures including length of stay, complications, and mortality.

Nonetheless, this study had several strengths. To our knowledge, this is one of the largest single-centre studies regarding SJS/TEN in Asia; it included a homogenous group of predominantly Chinese patients. The patients were managed by the same dermatology team with a consistent diagnostic and therapeutic approach throughout the study period. Data collection was adequate, and exhaustive evaluation of drug history was feasible for cases with access to both electronic records and hospital case notes. To ensure accurate identification of causative drugs, the ALDEN score was retrospectively evaluated by two independent dermatology doctors during the study.

This is the first large study in Hong Kong to provide data regarding the epidemiology, disease characteristics and clinical course, aetiology, treatment regimen, and mortality of SJS/TEN. Although uncommon, SJS/TEN is associated with substantial morbidity and mortality. Therefore, in addition to increasing awareness of SJS/TEN among patients and clinicians, efforts should be made to optimise inpatient care among public hospitals in Hong Kong by establishing dedicated multidisciplinary teams that are experienced in the management of SJS/TEN.

Concept or design: CMT Cheung, MM Chang.
Acquisition of data: All authors.
Analysis or interpretation of data: All authors.
Drafting of the manuscript: CMT Cheung, AWS Chan.
Critical revision of the manuscript for important intellectual content: All authors.

All authors had full access to the data, contributed to the study, approved the final version for publication, and take responsibility for its accuracy and integrity.

All authors have disclosed no conflicts of interest.

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

This research was approved by the Joint Chinese University of Hong Kong–New Territories East Cluster Clinical Research Ethics Committee (Ref No.: 2017.424). The requirement for informed consent was waived by the Committee due to the retrospective nature of the research.

1. Bastuji-Garin S, Rzany B, Stern RS, Shear NH, Naldi L, Roujeau JC. Clinical classification of cases of toxic epidermal necrolysis, Stevens–Johnson syndrome, and erythema multiforme. Arch Dermatol 1993;129:92-6. Crossref

2. Roujeau JC. Stevens–Johnson syndrome and toxic epidermal necrolysis are severity variants of the same disease which differs from erythema multiforme. J Dermatol 1997;24:726-9. Crossref

3. Roujeau JC, Chosidow O, Saiag P, Guillaume JC. Toxic epidermal necrolysis (Lyell syndrome). J Am Acad Dermatol 1990;23:1039-58. Crossref

4. Ying S, Ho W, Chan HH. Toxic epidermal necrolysis: 10 years experience of a burns centre in Hong Kong. Burns 2001;27:372-5. Crossref

5. Yeung CK. Intravenous immunoglobulin treatment for Stevens–Johnson syndrome and toxic epidermal necrolysis [dissertation]. Queen Mary Hospital, The University of Hong Kong; 2004.

6. Bastuji-Garin S, Fouchard N, Bertocchi M, Roujeau JC, Revuz J, Wolkenstein P. SCORTEN: a severity-of-illness score for toxic epidermal necrolysis. J Invest Dermatol 2000;115:149-53. Crossref

7. Sassolas B, Haddad C, Mockenhaupt M, et al. ALDEN, an algorithm for assessment of drug causality in Stevens–Johnson syndrome and toxic epidermal necrolysis: comparison with case-control analysis. Clin Pharmacol Ther 2010;88:60-8. Crossref

8. Roujeau JC, Guillaume JC, Fabre JP, Penso D, Fléchet ML, Girre JP. Toxic epidermal necrolysis (Lyell syndrome). Incidence and drug etiology in France, 1981-1985. Arch Dermatol 1990;126:37-42. Crossref

9. Rzany B, Mockenhaupt M, Baur S, et al. Epidemiology of erythema exsudativum multiforme majus, Stevens–Johnson syndrome, and toxic epidermal necrolysis in Germany (1990-1992): structure and results of a population-based registry. J Clin Epidemiol 1996;49:769-73. Crossref

10. Chan HL, Stern RS, Arndt KA, et al. The incidence of erythema multiforme, Stevens–Johnson syndrome, and toxic epidermal necrolysis. A population-based study with particular reference to reactions caused by drugs among outpatients. Arch Dermatol 1990;126:43-7. Crossref

11. Hsu DY, Brieva J, Silverberg NB, Silverberg JI. Morbidity and mortality of Stevens–Johnson syndrome and toxic epidermal necrolysis in United States adults. J Invest Dermatol 2016;136:138797. Crossref

12. Frey N, Jossi J, Bodmer M, et al. The epidemiology of Stevens–Johnson syndrome and toxic epidermal necrolysis in the UK. J Invest Dermatol 2017;137:1240-7. Crossref

13. Lee HY, Martanto W, Thirumoorthy T. Epidemiology of Stevens–Johnson syndrome and toxic epidermal necrolysis in Southeast Asia. Dermatologica Sinica 2013;31:217-20. Crossref

14. Chan HL. Toxic epidermal necrolysis in Singapore, 1989 through 1993: incidence and antecedent drug exposure. Arch Dermatol 1995;131:1212-3. Crossref

15. Yang MS, Lee JY, Kim J, et al. Incidence of Stevens–Johnson syndrome and toxic epidermal necrolysis: a nationwide population-based study using national health insurance database in Korea. PLoS One 2016;11:e0165933. Crossref

16. Hospital Authority, Hong Kong SAR Government. New Territories East Cluster biennial report 2018-2020. Available from: https://www3.ha.org.hk/ntec/clusterreport/clusterreport2018-20/index.html. Accessed 8 Feb 2024.

17. Schwartz RA, McDonough PH, Lee BW. Toxic epidermal necrolysis: Part I. Introduction, history, classification, clinical features, systemic manifestations, etiology, and immunopathogenesis. J Am Acad Dermatol 2013;69:173. e1-13; quiz 185-6. Crossref

18. Creamer D, Walsh SA, Dziewulski P, et al. UK guidelines for the management of Stevens–Johnson syndrome/toxic epidermal necrolysis in adults 2016. J Plast Reconstr Aesthet Surg 2016;69:e119-53. Crossref

19. Roujeau JC, Kelly JP, Naldi L, et al. Medication use and the risk of Stevens–Johnson syndrome or toxic epidermal necrolysis. N Engl J Med 1995;333:1600-7. Crossref

20. Mockenhaupt M, Viboud C, Dunant A, et al. Stevens–Johnson syndrome and toxic epidermal necrolysis: assessment of medication risks with emphasis on recently marketed drugs. The EuroSCAR study. J Invest Dermatol 2008;128:35-44. Crossref

21. Schwartz RA, McDonough PH, Lee BW. Toxic epidermal necrolysis: Part II. Prognosis, sequelae, diagnosis, differential diagnosis, prevention, and treatment. J Am Acad Dermatol 2013;69:187.e1-16; quiz 203-4. Crossref

22. Roujeau JC, Stern RS. Severe adverse cutaneous reactions to drugs. N Engl J Med 1994;331:1272-85. Crossref

23. Sekula P, Dunant A, Mockenhaupt M, et al. Comprehensive survival analysis of a cohort of patients with Stevens–Johnson syndrome and toxic epidermal necrolysis. J Invest Dermatol 2013;133:1197-204. Crossref

24. Barvaliya M, Sanmukhani J, Patel T, Paliwal N, Shah H, Tripathi C. Drug-induced Stevens–Johnson syndrome (SJS), toxic epidermal necrolysis (TEN), and SJS-TEN overlap: a multicentric retrospective study. J Postgrad Med 2011;57:115-9. Crossref

25. Roongpisuthipong W, Prompongsa S, Klangjareonchai T. Retrospective analysis of corticosteroid treatment in Stevens–Johnson syndrome and/or toxic epidermal necrolysis over a period of 10 years in Vajira Hospital, Navamindradhiraj University, Bangkok. Dermatol Res Pract 2014;2014:237821. Crossref

26. Suwarsa O, Yuwita W, Dharmadji HP, Sutedja E. Stevens–Johnson syndrome and toxic epidermal necrolysis in Dr Hasan Sadikin General Hospital Bandung, Indonesia from 2009-2013. Asia Pac Allergy 2016;6:43-7. Crossref

27. Lee HY, Fook-Chong S, Koh HY, Thirumoorthy T, Pang SM. Cyclosporine treatment for Stevens–Johnson syndrome/toxic epidermal necrolysis: retrospective analysis of a cohort treated in a specialized referral center. J Am Acad Dermatol 2017;76:106-13. Crossref

Intensive care treatments are primarily intended to improve patient outcomes. Considering the high operating costs of intensive care units (ICUs), a reliable, decision-supporting, risk stratification system is needed to predict patient outcomes and facilitate cost-effective use of ICU beds. Several disease severity scoring systems, such as the Acute Physiology and Chronic Health Evaluation (APACHE) system and the Simplified Acute Physiology Score system, are currently used to objectively assess outcomes and recovery potential in this complex and diverse group of patients.1 2

The APACHE system, one of the most commonly used benchmark severity scoring systems worldwide, can measure disease severity and predict hospital mortality among ICU patients. In the 40 years since its initial development, the APACHE system has undergone multiple revisions to improve statistical power and discrimination performance by modifying the numbers and weights of included variables.3 4 5 6 The underlying statistical principle is multivariable logistic regression based on data from an American population. The results are easy to interpret and allow robust outcome prediction for individuals with characteristics similar to the original population. However, the APACHE system has limited capacity to manage non-linear relationships between predictor and outcome variables, interactions between variables, and missing data. Although the value of the APACHE system for mortality prediction has been established, especially in Western countries, its discrimination performance and calibration are inconsistent when applied outside of the US.7 8 9 10 Since 2008, the Hospital Authority in Hong Kong has utilised the APACHE IV model to assess outcomes in critically ill patients. Nevertheless, the APACHE II model remains the most extensively validated version; it is widely used for research and reference purposes.11

In the early 1990s, artificial neural networks (ANNs), a type of machine learning algorithm, were proposed as alternative statistical techniques to logistic regression–based method. Similar to the organisation and data processing configurations in human brains, these networks consist of input and output layers with at least one or more intermediate (hidden) layers for pattern recognition. Each layer contains several ‘artificial neurons’, known as nodes, for data extraction; these nodes are connected with each other through variable ‘weights’.12 Artificial neural networks identify representative patterns from input data and observed output data within a training set, then fine-tune the variable weights; thus, they can predict outcomes when provided novel information. This method has considerable advantages in terms of managing non-linear relationships and multivariable interactions.13

A review of 28 studies comparing ANN and regression-based models showed that ANN outperformed regression-based models in 10 studies (36%), was outperformed by regression-based models in four studies (14%), and had similar performance in the remaining 14 studies (50%).14 Multiple recent studies also demonstrated that the integration of machine learning with electronic health records provided more accurate and reliable predictive performance compared with conventional prognostic models.15 16

This study was conducted to compare ANN performance with the performances of extensively validated and benchmark scoring systems—APACHE II and APACHE IV—in terms of predicting hospital mortality among critically ill patients in Hong Kong.

This retrospective analysis included all patients aged ≥18 years with first-time admissions to the ICU of Pamela Youde Nethersole Eastern Hospital between 1 January 2010 and 31 December 2019. The hospital is a 2000-bed tertiary care regional hospital that provides comprehensive services except for cardiothoracic surgery, transplant surgery, and burn management. The ICU is a 24-bed, closed, mixed medical-surgical unit with an average of 1600 patients admitted annually.

Demographic characteristics and hospital mortality data were retrospectively recorded. The worst value of each physiological parameter during the first 24 hours after ICU admission was used to generate an APACHE score. The predicted mortality risk was calculated based on published methods.3 5 Included parameters were age, sex, systolic and diastolic blood pressures, temperature, heart rate, respiratory rate, glucose level, blood urea nitrogen level, serum sodium level, creatinine level, haematocrit level, white cell count, albumin level, bilirubin level, pH, fraction of inspired oxygen, partial pressures of carbon dioxide and oxygen, bicarbonate, and urine output during the first 24 hours after ICU admission. For patients who had multiple ICU admissions during a single hospital stay, only the first admission was included. Patients were excluded if they died or were discharged from the ICU within 4 hours after admission.

Instances of incomplete data were resolved by multiple imputation using the Markov chain Monte Carlo algorithm (ie, fully conditional specification). This method fits a univariate (single dependent variable) model using all other available variables in the model as predictors, then imputes missing values for the dependent variable. The method continues until the maximum number of iterations is reached; the resulting imputed values are saved to the imputed dataset.

Neural network models were constructed with SPSS software (Windows version 25.0; IBM Corp, Armonk [NY], US) using the same parameters as in the APACHE IV model (online supplementary Fig); SPSS software was also used to examine model precision. The multilayer perceptron procedure, a class of feed-forward learning model, consists of ≥3 layers of nodes: input, hidden, and output.17 Automatic architecture building, which computes the best number of units in a hidden layer, was performed with SPSS software. Each hidden unit is an activation function of the weighted sum of the inputs; the values of the weights are determined by an estimation algorithm. In this study, the hidden layer consisted of 12 units (nodes). A hyperbolic tangent activation function was also employed for the hidden layers. Softmax activation and cross-entropy error functions were used for the output layer. The multilayer perceptron procedure utilised a backpropagation technique for supervised training. Learning occurred in the recognition phase for each piece of data via changes to connection weights based on the amount of error in the output compared with the expected result (gradient descent method).18

The training process was terminated when no further decreases in calculated error were observed. Subsequently, network weights were identified and used to compute test values. The importance of an independent variable was regarded as a measure of the extent to which network model–predicted values differed from observed values. Normalised importance, expressed as a percentage, constituted the ratio between the importance of each predictor variable and the largest importance value. Model stability was assessed by tenfold cross-validation. Oversampling of minority classes was performed via duplication to manage imbalances in outcome data.

Categorical and continuous variables were expressed as numbers (percentages) and medians (interquartile ranges). The Chi squared test or Fisher’s exact test was used for comparisons of categorical data; the Mann-Whitney U test was used for comparisons of continuous data. The performances of ANN, APACHE II, and APACHE IV models were evaluated in terms of discrimination and calibration power. Discrimination, which constitutes the ability of a predictive model to separate data into classes (eg, death or survival), was evaluated using the area under the receiver operating characteristic curve (AUROC). The AUROCs of the models were compared using the DeLong test. Calibration, which represents the closeness of model probability to the underlying probability of the study population, was evaluated using the Brier score, Hosmer–Lemeshow statistic, and calibration curves.19 All P values were two-sided, and values < 0.05 were considered statistically significant. All analyses were performed with SPSS software and MedCalc statistical software (version 19.6.1).

In total, 14 503 patients were included. The demographic characteristics and hospital mortality data of the study cohort were shown in Table 1, while the physiological and laboratory parameters required to generate an APACHE score were presented in Table 2. Among the recruited patients, 4.93% had at least one missing data point, and the overall rate of missing data was 0.48%. Furthermore, 1400 (9.7%) of the recruited patients were randomly assigned to the validation set; the remaining patients (n=13 103, 90.3%) were assigned to the model development set. With respect to the ANN model, 70% and 30% of the development set were used for training and testing purposes, respectively. The median age was 67 years (interquartile range [IQR]=54-78), median APACHE II score was 18 (IQR=13-25), and median APACHE IV score was 66 (IQR=46-91). The overall hospital and ICU mortality rates were 19.3% (n=2799) and 9.6% (n=1392), respectively.

The baseline co-morbidities, source of admission, disease category, APACHE II score, and APACHE IV score were similar in the test and validation sets (Table 1). More patients in the validation set received continuous renal replacement therapy (18.3% vs 16.1%; P=0.04). Concerning the worst physiological and laboratory parameters within the first 24 hours (Table 2), there were almost no significant differences between the development and validation sets; notably, the haemoglobin level was lower in the validation set (11.3 g/dL vs 11.5 g/dL; P=0.02).

In the development set, the ANN model (AUROC=0.89, 95% confidence interval [CI]=0.88-0.92, Brier score=0.10; P in Hosmer–Lemeshow test=0.34) outperformed the APACHE II model (AUROC=0.80, 95% CI=0.79-0.81, Brier score=0.15; P<0.001) and APACHE IV model (AUROC=0.84, 95% CI=0.83-0.85, Brier score=0.12; P<0.001) for prediction of hospital mortality. The cross-validation accuracy ranged from 0.98 to 1 (mean=0.99), indicating that our ANN model had good stability. There was no statistically significant difference between our ANN model and an ANN model created by oversampling of minority classes (AUROC=0.89, 95% CI=0.89-0.90; P=0.103).

In the validation set, the ANN model (AUROC=0.88, 95% CI=0.86-0.90, Brier score=0.10, P in Hosmer–Lemeshow test=0.37) was superior to the APACHE II model (AUROC=0.85, 95% CI=0.80-0.85, Brier score=0.14; P<0.001 for both comparisons of AUROCs and Brier scores) but similar to the APACHE IV model (AUROC=0.87, 95% CI=0.85-0.89, Brier score=0.11; P=0.34 for comparison of AUROCs, and P=0.05 for comparison of Brier scores) [Fig 1].

The calibration curve for the validation set showed that the ANN model (Fig 2a) outperformed the APACHE IV model (Fig 2b) and the APACHE II model (Fig 2c).

The importances of the predictor variables in predictions of hospital mortality using the ANN model were evaluated. Within 24 hours of ICU admission, the highest sodium level was the most important variable, followed by the highest bilirubin level and the lowest white cell count. Details regarding the normalised importance of each covariate are presented in online supplementary Tables 1 and 2.

To our knowledge, this is the first study in Asia to assess the performance of ANN and compare it with the performances of two extensively validated and benchmark scoring systems—APACHE II and APACHE IV—in terms of predicting hospital mortality among critically ill patients. We found that the ANN model provided better discrimination and calibration compared with the APACHE II model. However, the difference between the ANN and APACHE IV models was less prominent. Calibration was slightly better with the ANN model, but discrimination was similar between the ANN and APACHE IV models.

Conventional logistic regression–based APACHE systems often lose calibration over time and require regular updates to maintain performance.6 11 20 21 22 The original APACHE II model was developed over 30 years ago using data from 13 different hospitals in the US; it was validated in the country before clinical application.2 Studies in Hong Kong7 and Singapore23 have shown that the APACHE II model has good discrimination but poor calibration for ICU patients in Asia. Calibration remained suboptimal regardless of customisation as demonstrated by Lew et al,23 indicating the need for a new prognostic prediction model. Wong and Young24 showed that the APACHE II model had equivalent performance status compared with an ANN model that had been trained and validated using the original APACHE II data. In a medicalneurological ICU in India, an ANN model trained on an Indian population (with or without redundant variables) demonstrated better calibration compared with the APACHE II model.25 The authors speculated that this finding was partly related to differences in standards of care and resources between American and Indian ICUs.25 Overall, differences in case mix, advances in medical technology, and the use of more recent data may explain the superiority of our ANN model compared with the APACHE II model.

Compared with ICU patients in the US, it is fivefold more common for Hong Kong ICU patients to begin renal replacement therapy.26 More than 50% of critically ill patients in Hong Kong require mechanical ventilation, compared with 28% in the US.26 27 A recent population-based study of all patients admitted to adult ICUs in Hong Kong between 2008 and 2018 showed that the APACHE IV standardised mortality ratio decreased from 0.81 to 0.65 during the study period, implying a gradual decline in the performance of the APACHE IV model.26 This model, which was established using data derived from >100 000 ICU patients in 45 US hospitals between 2002 and 2003,5 also tends to overestimate hospital mortality among ICU patients in Hong Kong. In contrast to our study population, where Asian ethnicities were most common, 70% of the patients in APACHE IV reference population were Caucasian.5 The subtle differences in performance between our ANN model and the APACHE IV model could be related to differences in timing during the development of the models. Nevertheless, our ANN model trained on a Hong Kong population was better calibrated for prediction in such a population, compared with the APACHE IV model. This improved calibration could be related to differences in target population (Asian vs Caucasian), epidemiology, and disease profile.

The selection of appropriate variables is a key aspect of model development. The inclusion of additional predictor variables does not necessarily improve a model’s overall performance. Redundant variables may result in overfitting and produce a complicated predictive model without additional benefits. A recently published large national cohort study from Sweden showed that a simplified ANN model with eight parameters outperformed the Simplified Acute Physiology Score III model in terms of discrimination and calibration.28 Among the eight parameters, age and leukocyte count were the most and least important variables, respectively. Notably, leukocyte count was the most important variable in terms of predicting mortality among patients on continuous renal replacement therapy.29 Similar to the present study, Kang et al29 found that age was the 12th most important variable. The overall performance of an ANN model trained with APACHE II parameters in an Indian population could be maintained with the 15 highest information gain variables, including serum sodium level and leukocyte count.25

Among the 53 parameters in our ANN model, the highest sodium level, highest bilirubin level, and lowest white cell count within 24 hours of ICU admission were the top three most important predictor variables (online supplementary Table 1). The association between acquired hypernatraemia and increased hospital mortality among critically patients has consistently been demonstrated in multiple studies.30 31 Hyperbilirubinaemia, another complication in patients with sepsis, was associated with the onset of acute respiratory distress syndrome.32 Sepsis and gastrointestinal/hepatobiliary diseases caused ICU admission in approximately 40% of our patients, possibly explaining the importance of hyperbilirubinaemia in our ANN model. Although the importance of leukocyte count has been demonstrated in other mortality prediction models, the previous models did not specify whether the count was high or low.25 28 29 In the present study, the lowest white cell count was more important than the highest white cell count. Another intriguing observation was that age constituted the 11th most important predictor in our ANN model (online supplementary Table 1). Age is a predictor of survival in many prognostic models.3 5 28 Increasing biological age is often associated with multiple co-morbidities and a progressive decline in physiological reserve, leading to increased mortality. However, a recently published systematic review of 129 studies showed large variations in ICU and hospital mortality rates among older ICU patients, ranging from 1% to 51% in single-centre retrospective studies and 6% to 28% in multicentre retrospective studies.33 These results could be related to differences in admission policies, premorbid functional status, and the intensity of provided to older critically ill patients.

Our ANN model was trained and internally validated on a large number of representative data samples that included most patients admitted to a tertiary ICU in Hong Kong over the past decade. This approach addressed the small sample size limitation that was common in previous studies.24 25 34 All data were automatically collected by a computer system, eliminating the risk of human error during data extraction. Healthcare system digitalisation and advances in information technology have enabled effortless generation of abundant clinical data (eg, physiological parameters, laboratory results, and radiological findings), which can facilitate data collection and development of a new risk prediction model via machine learning.35 36 We hope that generalisability to other ICUs in Asia can be achieved through external validation studies.

This study had some limitations. Although the sample size was large, all data were collected from a single centre; in contrast, data for the APACHE scoring system were derived from multiple large centres. Because the primary objective of the present study was comparison of performance between our ANN model and the APACHE II and APACHE IV models using identical parameters, we did not attempt to determine the optimal subset of parameters that would maintain high ANN performance.25 28 Furthermore, our ANN model may not be applicable to other centres with different case mixes and medical approaches. The lack of external validation may lead to concerns about overfitting, which is a common challenge in ANN model development. Because mortality prediction among ICU patients is a dynamic process, other limitations include the use of static data and the lack of a fixed time point for mortality assessment.

Mortality prediction among critically patients is a challenging endeavour. Our ANN model, which was trained with representative data from a Hong Kong population, outperformed the internationally validated APACHE II model with respect to critically ill patients in Hong Kong. In contrast to the APACHE IV model, our ANN model demonstrated better calibration but similar discrimination performance. External validation studies using data from other hospitals are recommended to confirm our findings. Future studies should explore the feasibility of reducing the number of variables while preserving the discrimination and calibration power of the ANN model. The widespread use of computerised information systems, rather than paper records, in ICU and general ward settings has led to increased data availability. Artificial neural networks, along with other machine learning techniques, are valuable tools that can enhance real-time prognostic prediction.

Concept or design: S Lau, HP Shum, CCY Chan.
Acquisition of data: S Lau, HP Shum, CCY Chan.
Analysis or interpretation of data: S Lau, HP Shum, CCY Chan.
Drafting of the manuscript: S Lau.
Critical revision of the manuscript for important intellectual content: MY Man, KB Tang, KKC Chan, AKH Leung, WW Yan.

All authors had full access to the data, contributed to the study, approved the final version for publication, and take responsibility for its accuracy and integrity.

All authors have disclosed no conflicts of interests.

Part of the research was presented at the 34th Annual Congress of the European Society of Intensive Care Medicine (3-6 October 2021, virtual) and the Annual Scientific Meeting 2021 of Hong Kong Society of Critical Care Medicine (12 December 2021, virtual).

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

The study protocol complies with the Declaration of Helsinki and was approved by the Hong Kong East Cluster Research Ethics Committee of Hospital Authority, Hong Kong (Ref No.: HKECREC-2021-024). The requirement for patient consent was waived by the Committee due to the retrospective nature of the study.

The supplementary material was provided by the authors and some information may not have been peer reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by the Hong Kong Academy of Medicine and the Hong Kong Medical Association. The Hong Kong Academy of Medicine and the Hong Kong Medical Association disclaim all liability and responsibility arising from any reliance placed on the content.

1. Keegan MT, Gajic O, Afessa B. Severity of illness scoring systems in the intensive care unit. Crit Care Med 2011;39:163-9. Crossref

2. Breslow MJ, Badawi O. Severity scoring in the critically ill: part 1—interpretation and accuracy of outcome prediction scoring systems. Chest 2012;141:245-52. Crossref

3. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818-29. Crossref

4. APACHE III study design: analytic plan for evaluation of severity and outcome [editorial]. Crit Care Med 1989;17:S169-221.

5. Zimmerman JE, Kramer AA, McNair DS, Malila FM, Shaffer VL. Intensive care unit length of stay: benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV. Crit Care Med 2006;34:2517-29. Crossref

6. Zimmerman JE, Kramer AA. Outcome prediction in critical care: the Acute Physiology and Chronic Health Evaluation models. Curr Opin Crit Care 2008;14:491-7. Crossref

7. Tan IK. APACHE II and SAPS II are poorly calibrated in a Hong Kong intensive care unit. Ann Acad Med Singap 1998;27:318-22.

8. Gupta R, Arora VK. Performance evaluation of APACHE II score for an Indian patient with respiratory problems. Indian J Med Res 2004;119:273-82.

9. Choi JW, Park YS, Lee YS, et al. The ability of the Acute Physiology and Chronic Health Evaluation (APACHE) IV score to predict mortality in a single tertiary hospital. Korean J Crit Care Med 2017;32:275-83. Crossref

10. Ghorbani M, Ghaem H, Rezaianzadeh A, Shayan Z, Zand F, Nikandish R. A study on the efficacy of APACHEIV for predicting mortality and length of stay in an intensive care unit in Iran. F1000Res 2017;6:2032. Crossref

11. Ko M, Shim M, Lee SM, Kim Y, Yoon S. Performance of APACHE IV in medical intensive care unit patients: comparisons with APACHE II, SAPS 3, and MPM0 III. Acute Crit Care 2018;33:216-21. Crossref

12. Xie J, Su B, Li C, et al. A review of modeling methods for predicting in-hospital mortality of patients in intensive care unit. J Emerg Crit Care Med 2017;1:18. Crossref

13. Tu JV. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 1996;49:1225-31. Crossref

14. Sargent DJ. Comparison of artificial neural networks with other statistical approaches: results from medical data sets. Cancer 2001;91(8 Suppl):1636-42. Crossref

15. Meiring C, Dixit A, Harris S, et al. Optimal intensive care outcome prediction over time using machine learning. PLoS One 2018;13:e0206862. Crossref

16. Rajkomar A, Oren E, Chen K, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med 2018;1:18. Crossref

17. Norgaard M, Ravn O, Poulsen NK, Hansen LK. Neural Networks for Modelling and Control of Dynamic Systems: A Practitioner’s Handbook. London: Springer; 2000.

18. Ludermir TB, Yamazaki A, Zanchettin C. An optimization methodology for neural network weights and architectures. IEEE Trans Neural Netw 2006;17:1452-9. Crossref

19. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128-38. Crossref

20. Lam KW, Lai KY. Evaluation of outcome and performance of an intensive care unit in Hong Kong by APACHE IV model: 2007-2014. J Emerg Crit Care Med 2017;1:16. Crossref

21. Paul E, Bailey M, Van Lint A, Pilcher V. Performance of APACHE III over time in Australia and New Zealand: a retrospective cohort study. Anaesth Intensive Care 2012;40:980-94. Crossref

22. Mann SL, Marshall MR, Holt A, Woodford B, Williams AB. Illness severity scoring for intensive care at Middlemore Hospital, New Zealand: past and future. N Z Med J 2010;123:47-65.

23. Lew CC, Wong GJ, Tan CK, Miller M. Performance of the Acute Physiology and Chronic Health Evaluation II (APACHE II) in the prediction of hospital mortality in a mixed ICU in Singapore. Proc Singapore Healthc 2019;28:147-52. Crossref

24. Wong LS, Young JD. A comparison of ICU mortality prediction using the APACHE II scoring system and artificial neural networks. Anaesthesia 1999;54:1048-54. Crossref

25. Nimgaonkar A, Karnad DR, Sudarshan S, Ohno-Machado L, Kohane I. Prediction of mortality in an Indian intensive care unit. Comparison between APACHE II and artificial neural networks. Intensive Care Med 2004;30:248-53. Crossref

26. Ling L, Ho CM, Ng PY, et al. Characteristics and outcomes of patients admitted to adult intensive care units in Hong Kong: a population retrospective cohort study from 2008 to 2018. J Intensive Care 2021;9:2. Crossref

27. Wunsch H, Angus DC, Harrison DA, Linde-Zwirble WT, Rowan KM. Comparison of medical admissions to intensive care units in the United States and United Kingdom. Am J Respir Crit Care Med 2011;183:1666-73. Crossref

28. Holmgren G, Andersson P, Jakobsson A, Frigyesi A. Artificial neural networks improve and simplify intensive care mortality prognostication: a national cohort study of 217,289 first-time intensive care unit admissions. J Intensive Care 2019;7:44. Crossref

29. Kang MW, Kim J, Kim DK, et al. Machine learning algorithm to predict mortality in patients undergoing continuous renal replacement therapy. Crit Care 2020;24:42. Crossref

30. O’Donoghue SD, Dulhunty JM, Bandeshe HK, Senthuran S, Gowardman JR. Acquired hypernatraemia is an independent predictor of mortality in critically ill patients. Anaesthesia 2009;64:514-20. Crossref

31. Olsen MH, Møller M, Romano S, et al. Association between ICU-acquired hypernatremia and in-hospital mortality: data from the medical information mart for intensive care III and the electronic ICU collaborative research database. Crit Care Explor 2020;2:e0304. Crossref

32. Zhai R, Sheu CC, Su L, et al. Serum bilirubin levels on ICU admission are associated with ARDS development and mortality in sepsis. Thorax 2009;64:784-90. Crossref

33. Vallet H, Schwarz GL, Flaatten H, de Lange DW, Guidet B, Dechartres A. Mortality of older patients admitted to an ICU: a systematic review. Crit Care Med 2021;49:324-34. Crossref

34. Clermont G, Angus DC, DiRusso SM, Griffin M, Linde-Zwirble WT. Predicting hospital mortality for patients in the intensive care unit: a comparison of artificial neural networks with logistic regression models. Crit Care Med 2001;29:291-6. Crossref

35. Kim S, Kim W, Park RW. A comparison of intensive care unit mortality prediction models through the use of data mining techniques. Healthc Inform Res 2011;17:232-43. Crossref

36. Bulgarelli L, Deliberato RO, Johnson AE. Prediction on critically ill patients: the role of “big data”. J Crit Care 2020;60:64-8. Crossref