Inter-observer agreement of vertebral fracture assessment with dual-energy x-ray absorptiometry equipmentю

Together with Leiden University Medical Center, a research team at ImageBiopsy Lab investigated the time and effort needed to perform vertebral morphometry and inter-observer agreement to identify #vertebral fractures on vertebral fracture assessment images.

Share this article

Inter-observer agreement of vertebral fracture assessment with dual-energy x-ray absorptiometry equipmentю

Together with Leiden University Medical Center, a research team at ImageBiopsy Lab investigated the time and effort needed to perform vertebral morphometry and inter-observer agreement to identify #vertebral fractures on vertebral fracture assessment images.

Authors: Jacob M. Mostert, Stephan Romeijn, Petra Dibbets-schneider, Daphne D. D. Rietbergen, Lenka M. Pereira Arias-Bouda, Christoph Götz, Matthew DiFranco, Hans Peter Dimai & Willem Grootjans

Abstract

Purpose

To investigate the time and effort needed to perform vertebral morphometry, as well as inter-observer agreement for identification of vertebral fractures on vertebral fracture assessment (VFA) images.

Methods

Ninety-six images were retrospectively selected, and three radiographers independently performed semi-automatic 6-point morphometry. Fractures were identified and graded using the Genant classification. The time needed to annotate each image was recorded, and reader fatigue was assessed using a modified Simulator Sickness Questionnaire (SSQ). Inter-observer agreement was assessed per-patient and per-vertebra for detecting fractures of all grades (grades 1–3) and for grade 2 and 3 fractures using the kappa statistic. Variability in measured vertebral height was evaluated using the intraclass correlation coefficient (ICC).

Results

The per-patient agreement was 0.59 for grades 1–3 fracture detection, and 0.65 for grades 2–3 only. The agreement for per-vertebra fracture classification was 0.92. Vertebral height measurements had an ICC of 0.96. The time needed to annotate VFA images ranged between 91 and 540 s, with a mean annotation time of 259 s. Mean SSQ scores were significantly lower at the start of a reading session (1.29; 95% CI: 0.81–1.77) compared to the end of a session (3.25; 95% CI: 2.60–3.90; p < 0.001).

Conclusion

Agreement for detection of patients with vertebral fractures was only moderate, and vertebral morphometry requires a substantial time investment. This indicates that there is a potential benefit for automating VFA, both in improving inter-observer agreement and in decreasing reading time and burden on readers.

Data availability

The dataset from this study is not publicly available due to data protection regulations.

Code availability

Statistical analyses were performed using open-source python modules Scikit-learn 0.20.3, Statsmodels 0.10.1, and Pingouin 0.3.8.

Download and read the full paper you can here


References
  1. Kanis JA on behalf of the World Health Organization Scientific Group (2007) Assessment of osteoporosis at the primary health-care level. Technical Report. World Health Organization Collaborating Centre for Metabolic Bone Diseases, University of Sheffield, UK. 2007
  2. O’neill TW, Felsenberg D, Varlow J, Cooper C, Kanis JA, Silman AJ, European Vertebral Osteoporosis Study Group (1996) The prevalence of vertebral deformity in European men and women: the European Vertebral Osteoporosis Study. J Bone Miner Res 11(7):1010–1018. https://doi.org/10.1002/jbmr.5650110719
  3. CAS Article PubMed Google Scholar
  4. Felsenberg D, Silman AJ, Lunt M et al (2002) Incidence of vertebral fracture in Europe: results from the European Prospective Osteoporosis Study (EPOS). J Bone Miner Res 17(4):716–724. https://doi.org/10.1359/jbmr.2002.17.4.716
  5. CAS Article PubMed Google Scholar
  6. Wade SW, Strader C, Fitzpatrick LA, Anthony MS, O’Malley CD (2014) Estimating prevalence of osteoporosis: examples from industrialized countries. Arch Osteoporos 9(1):182. https://doi.org/10.1007/s11657-014-0182-3
  7. CAS Article PubMed Google Scholar
  8. Gullberg B, Johnell O, Kanis JA (1997) World-wide projections for hip fracture. Osteoporos Int 7(5):407–413. https://doi.org/10.1007/pl00004148
  9. CAS Article PubMed Google Scholar
  10. Ross PD, Ettinger B, Davis JW, Melton L, Wasnich RD (1991) Evaluation of adverse health outcomes associated with vertebral fractures. Osteoporos Int 1(3):134–140. https://doi.org/10.1007/bf01625442
  11. CAS Article PubMed Google Scholar
  12. Melton LJ, Atkinson EJ, Cooper C, O’Fallon WM, Riggs BL (1999) Vertebral fractures predict subsequent fractures. Osteoporos Int 10(3):214–221. https://doi.org/10.1007/s001980050218
  13. Article PubMed Google Scholar
  14. Black DM, Arden NK, Palermo L, Pearson J, Cummings SR (1999) Prevalent vertebral deformities predict hip fractures and new vertebral deformities but not wrist fractures. Study of Osteoporotic Fractures Research Group. J Bone Miner Res 14(5):821–828. https://doi.org/10.1359/jbmr.1999.14.5.821
  15. CAS Article PubMed Google Scholar
  16. Cauley JA, Hochberg MC, Lui LY, Palermo L, Ensrud KE, Hillier TA, Nevitt MC, Cummings SR (2007) Long-term risk of incident vertebral fractures. JAMA 298(23):2761–2767. https://doi.org/10.1001/jama.298.23.2761
  17. CAS Article PubMed Google Scholar
  18. Lindsay R, Silverman SL, Cooper C, Hanley DA, Barton I, Broy SB, Licata A, Benhamou L, Geusens P, Flowers K, Stracke H, Seeman E (2001) Risk of new vertebral fracture in the year following a fracture. JAMA 285(3):320–323. https://doi.org/10.1001/jama.285.3.320
  19. CAS Article PubMed Google Scholar
  20. Ismail AA, Cockerill W, Cooper C et al (2001) Prevalent vertebral deformity predicts incident hip though not distal forearm fracture: results from the European Prospective Osteoporosis Study. Osteoporos Int 12(2):85–90. https://doi.org/10.1007/s001980170138
  21. CAS Article PubMed Google Scholar
  22. Genant HK, Jergas M (2003) Assessment of prevalent and incident vertebral fractures in osteoporosis research. Osteoporos Int 14(Suppl 3):S43-55. https://doi.org/10.1007/s00198-002-1348-1
  23. Article PubMed Google Scholar
  24. Roux C, Baron G, Audran M, Breuil V, Chapurlat R, Cortet B, Fardellone P, Trémollières F, Ravaud P (2011) Influence of vertebral fracture assessment by dual-energy X-ray absorptiometry on decision-making in osteoporosis: a structured vignette survey. Rheumatology (Oxford) 50:2264–2269. https://doi.org/10.1093/rheumatology/ker225
  25. Article Google Scholar
  26. Lewiecki EM, Laster AJ (2006) Clinical applications of vertebral fracture assessment by dual-energy x-ray absorptiometry. J Clin Endocrinol Metab 91(11):4215–4222. https://doi.org/10.1210/jc.2006-1178
  27. CAS Article PubMed Google Scholar
  28. Malgo F, Hamdy NAT, Ticheler CHJM, Smit F, Kroon HM, Rabelink TJ, Dekkers OM, Appelman-Dijkstra NM (2017) Value and potential limitations of vertebral fracture assessment (VFA) compared to conventional spine radiography: experience from a fracture liaison service (FLS) and a meta-analysis. Osteoporos Int 28(10):2955–2965. https://doi.org/10.1007/s00198-017-4137-6
  29. CAS Article PubMed PubMed Central Google Scholar
  30. Shuhart CR, Yeap SS, Anderson PA, Jankowski LG, Lewiecki EM, Morse LR, Rosen HN, Weber DR, Zemel BS, Shepherd JA (2019) Executive summary of the 2019 ISCD position development conference on monitoring treatment, DXA cross-calibration and least significant change, spinal cord injury, peri-prosthetic and orthopedic bone health, transgender medicine, and pediatrics. J Clin Densitom 22(4):453–471. https://doi.org/10.1016/j.jocd.2019.07.001
  31. Article PubMed Google Scholar
  32. Genant HK, Wu CY, Van Kuijk C, Nevitt MC (1993) Vertebral fracture assessment using a semiquantitative technique. J Bone Miner Res 8(9):1137–1148. https://doi.org/10.1002/jbmr.5650080915
  33. CAS Article PubMed Google Scholar
  34. Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG (1993) Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 3(3):203–220. https://doi.org/10.1207/s15327108ijap0303_3
  35. Article Google Scholar
  36. Randolph JJ (2005) Free-Marginal Multirater Kappa (multirater K [free]): An alternative to Fleiss' fixed-marginal multirater kappa. Paper presented at the Joensuu University Learning and Instruction Symposium 2005, Joensuu, Finland, October 14–15th, 2005
  37. Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163. https://doi.org/10.1016/j.jcm.2016.02.012
  38. Article PubMed PubMed Central Google Scholar
  39. Davies KM, Recker RR, Heaney RP (1989) Normal vertebral dimensions and normal variation in serial measurements of vertebrae. J Bone Miner Res 4(3):341–349. https://doi.org/10.1002/jbmr.5650040308
  40. CAS Article PubMed Google Scholar
  41. Pearson D, Horton B, Green DJ, Hosking DJ, Goodby A, Steel SA (2006) Vertebral morphometry by DXA: a comparison of supine lateral and decubitus lateral densitometers. J Clin Densitom 9(3):295–301. https://doi.org/10.1016/j.jocd.2006.03.011
  42. Article PubMed Google Scholar
  43. Bazzocchi A, Spinnato P, Fuzzi F, Diano D, Morselli-Labate AM, Sassi C, Salizzoni E, Battista G, Guglielmi G (2012) Vertebral fracture assessment by new dual-energy X-ray absorptiometry. Bone 50(4):836–841. https://doi.org/10.1016/j.bone.2012.01.018
  44. Article PubMed Google Scholar
  45. Van Dort MJ, Romme EAPM, Smeenk FWJM, Geusens PPPM, Wouters EFM, van den Bergh JP (2018) Diagnosis of vertebral deformities on chest CT and DXA compared to routine lateral thoracic spine X-ray. Osteoporos Int 29(6):1285–1293. https://doi.org/10.1007/s00198-018-4412-1
  46. Article PubMed PubMed Central Google Scholar
  47. Fuerst T, Wu C, Genant HK, Von Ingersleben G, Chen Y, Johnston C, Econs MJ, Binkley N, Vokes TJ, Crans G, Mitlak BH (2009) Evaluation of vertebral fracture assessment by dual X-ray absorptiometry in a multicenter setting. Osteoporos Int 20(7):1199–1205. https://doi.org/10.1007/s00198-008-0806-9
  48. CAS Article PubMed Google Scholar
  49. Ferrar L, Jiang G, Schousboe JT, DeBold CR, Eastell R (2008) Algorithm-based qualitative and semiquantitative identification of prevalent vertebral fracture: agreement between different readers, imaging modalities, and diagnostic approaches. J Bone Miner Res 23(3):417–424. https://doi.org/10.1359/jbmr.071032
  50. Article PubMed Google Scholar
  51. Damiano J, Kolta S, Porcher R, Tournoux C, Dougados M, Roux C (2006) Diagnosis of vertebral fractures by vertebral fracture assessment. J Clin Densitom 9(1):66–71. https://doi.org/10.1016/j.jocd.2005.11.002
  52. Article PubMed Google Scholar
  53. Lee JH, Lee YK, Oh SH, Ahn J, Lee YE, Pyo JH, Choi YY, Kim D, Bae SC, Sung YK, Kim DY (2016) A systematic review of diagnostic accuracy of vertebral fracture assessment (VFA) in postmenopausal women and elderly men. Osteoporos Int 27(5):1691–1699. https://doi.org/10.1007/s00198-015-3436-z
  54. Article PubMed Google Scholar
  55. Cooper C, Atkinson EJ, O’Fallen WM, Melton LJ (1992) Incidence of clinically diagnosed vertebral fractures: a population based study in Rochester, Minnesota, 1985–1989. J Bone Miner Res 7:221–227. https://doi.org/10.1002/jbmr.5650070214
  56. CAS Article PubMed Google Scholar
  57. Delmas PD, van de Langerijt L, Watts NB, Eastell R, Genant HK, Grauer A, Cahall DL (2005) Underdiagnosis of vertebral fractures is a worldwide problem: the IMPACT study. J Bone Miner Res 20(4):557–563. https://doi.org/10.1359/jbmr.041214
  58. Article PubMed Google Scholar
  59. Yang J, Cosman F, Stone PW, Li M, Nieves JW (2020) Vertebral fracture assessment (VFA) for osteoporosis screening in US postmenopausal women: is it cost-effective? Osteoporos Int 31(12):2321–2335. https://doi.org/10.1007/s00198-020-05588-6