Introduction
The number of radiological imaging procedures carried out using all modalities in clinical imaging has continued to increase markedly over recent decades. In Australia, annual use of diagnostic imaging increased by 138% from 2000 to 2019 [1]. This has resulted in a steady increase in the amount of radiation dose to the world’s population averaged over the entire population [2]. Computed tomography (CT) procedures had the largest contribution (62.6%) to the collective effective dose to the world population [2]. In addition, more individuals are being given unusually high doses as many interventional procedures are becoming more complex and widely available. This is especially the case in well-resourced countries and areas, such as New South Wales (NSW), Australia. This increase in population dose may cause an increased risk of radiation induced tissue and stochastic effects. Optimization in imaging aims to achieve desired diagnostic outcomes with a reduction in radiation dose to the persons being imaged. To achieve optimization, a reproducible readily available measure of the radiation doses delivered during X-ray imaging, using any ionizing radiation for imaging, is a necessary starting point. Although different metrics are used for different modalities, these metrics can all be effectively utilised to estimate the risk to a population undergoing particular examinations, or procedures in the clinical setting. Clinical doses can be compared to some standard or reference dose that is widely accepted to give diagnostic quality imaging for a particular diagnostic procedure. These diagnostic reference doses are usually accumulated as a function of the radiation doses delivered for a defined procedure as carried out at a range of different practices within a defined country, region, local district, or imaging facility.
Diagnostic reference levels (DRLs), as defined by the International Commission on Radiological Protection as a form of investigation level, are used as a tool to aid in optimization of radiation doses and image quality during the medical exposure of patients for diagnostic and interventional procedures [3]. Local DRLs (LDRLs) (also known as facility reference levels) allow individual sites to monitor and compare their current state of practice in terms of the amount of radiation used for specific clinical imaging procedures to some national diagnostic reference level (NDRL). Additionally, establishment of LDRLs fulfils the departmental accreditation requirement, as established by Diagnostic Imaging Accreditation Scheme, to review ionizing radiation doses delivered during imaging [4]. Australian Radiation Protection and Nuclear Safety Agency (ARPANSA) has run nationwide surveys in Australia since August 2011 and NDRLs for eight procedures are currently available [5].
Dose management software (DMS) systems have recently become more widely available, to allow for the storage and analysis of radiation dose structured reports (RDSRs) generated by radiological imaging equipment in a Digital Imaging and COmmunications in Medicine (DICOM) compliant format. The RDSR standard first became available in 2007 and has now been implemented by X-ray unit manufacturers [6]. The DMS stores and allows analysis of the RDSRs. It also allows presentation of the acquisition parameters and other relevant dose related information in a format that allows ready identification of the sources of potentially high doses.
Dose audits can be broadly categorised by the method of data collection, that being manual, or software based. Manual methods involve individually querying the picture archiving and communication systems (PACS) for a set sample size of patient medical imaging dose data for a given protocol. Software based methods automatically collect dose data on a local or cloud-based database. Methods have been described for development of DRLs and these were generally followed in the methods of this study [7].
The aim of this research is to implement a DMS system for recording and analysing radiation doses delivered by CT scanners in Sydney, NSW, and to compare two methods of establishing LDRLs; a manual method using a limited well defined selected sample of patients, and a DMS system-based method where a larger less uniform population of people examined using the protocol of interest are included. We also explored the advantages and disadvantages of each method. This will be achieved by generating LDRLs using both manual and DMS based methods, for the eight procedures in CT for which NDRLs are available [5] and investigating the impact of each method on the establishment of DRLs.
Materials and Methods
The study investigates two methods of conducting dose audits for the purpose of establishing LDRLs namely manual and automated methods. All scanners included in the study held valid NSW Environment Protection Agency (EPA) compliance certificates, indicating scanner performance and safety complied with standards described in “NSW EPA radiation standard 6 part 5 computed tomography” [8]. The flow diagram for carrying out the LDRL determinations is detailed in Fig. 1. The detailed steps required in each of the methods investigated are given in the flow diagram.
1. Manual Method
The manual method followed the procedures described in [9]. The manual method results for LDRLs were obtained from three CT scanners across three medical imaging departments within the NSW Local Health District (LHD). This was done as part of the departments’ annual CT dose data submissions to the annual ARPANSA NDRL program survey for 2022.
Eight CT procedures defined by anatomical scan regions indicated by the ARPANSA NDRL program [5] were intended for manual data collection. Data from a scanner were not included if it did not perform a minimum of 10 studies for that scanner and CT procedure. Manual methods were performed by radiographers were some may filter procedures by modality and protocol name on PACS from the last calendar year, while others use the radiological information system (RIS). Nine CT procedures and examples of local CT protocol names used for each scan region are provided in the Table 1 as CT chest with and without contrast were included.
Patient selection criteria included individuals aged 15 years and above with a standard size range of between 70 kg and 80 kg. Patient weight was verified using the electronic medical records database. The goal was to collect data from each scanner for at least 10 and up to 20 patients per procedure. Dose data for 352 CT procedures were collected across three departments for the manual method.
For each suitable patient, the data collection process involved recording the volumetric computed tomography dose index (CTDIvol) and dose length product (DLP) from PACS, along with patient weight, age, and gender. For multiphase exams, users recorded the largest helical CTDIvol and its associated DLP. All data were entered into the ARPANSA NDRL service portal. The NDRL program also requested users to enter protocol-specific technical parameters such as tube voltage (kVp), starting or reference tube charge (mAs), tube rotation time, set pitch, use of contrast, use of dose modulation, number of phases, image acquisition type (helical/axial), detector configuration, iterative reconstruction used, reconstructed slice width, reconstruction algorithm/kernel, scan field of view, beam shaping filter, and set noise index.
The manual method of data collection involved minimal data cleaning during analysis because the data cleaning was intrinsic to the selection of standard-sized patients and age, as well as the selection of procedures consistent with the clinical indication and scan regions recommended in the NDRL program.
After the submission of site data, a facility report was generated and made available to the user, providing a comparison of the site facility DRL, or LDRL, and the NDRL.
2. Automated Methods
Automated methods utilized the recently introduced DMS, which stored dose data along with exposure factors for every medical imaging procedure sent to the DMS. The procedures involved in the implementation process of integrating DMS into the clinical workflow were found to be a critical aspect of making full and efficient use of the DMS.
1) Implementation of dose management software
The DMS named Sectra DoseTrack, developed by Sectra Medical, was introduced into clinical service in 2022 as part of the transition to a new PACS. The multi-stage implementation process involved the following events in chronological order:
(1) Creation of the DMS implementation team and schedule of planning meetings. The implementation team consisted of product application specialists and developers, the district rollout project manager, the Department Information Technology (IT) manager, the lead radiographer, RIS/PACS managers, and medical physicists.
(2) The PACS team provided the implementation teams with a list of imaging stations in the district that were already generating and exporting RDSRs to PACS. Many modern stations were already pre-configured to produce and send RDSRs, whereas older stations required the option to generate and send RDSRs to be activated. This was observed to be a major cause of delay in getting stations online.
(3) Individual application entity (AE) titles were created on DMS for each imaging station.
(4) RDSRs of test patients were sent from each station to PACS and DMS to test connectivity between stations, PACS, and DMS, and to confirm any changes to network configuration. A staged approach was taken, where each department sent test patient RDSRs. Priority was given to those stations that were already producing RDSRs so that data collection could commence promptly [10].
(5) Verification and validation of accurate dose information transfer to DMS were performed for each unit by a medical physicist [10].
(6) After confirmation of steps three to five, the station was then configured to auto-push RDSRs to DMS without user input.
(7) Super user accounts were created for RIS/PACS managers, and user accounts were created for medical physicists and radiographers. Sessions of end-user training were provided by the vendor.
2) Fully automated and semi-automated methods
The automated method of establishing LDRLs was further divided into two methods: fully automated and semi-automated. Both automated methods use the dose data stored in the DMS; however, the semi-automated method was developed to include data cleaning procedures specific to CT exposures, which were found to be critical in accurately determining LDRLs.
3) Fully automated method
The fully automated method solely utilized the Sectra DoseTrack DMS, along the sorting, filtering, and analysis tools it provides. Using the search filter feature of the DMS, nine CT procedures were individually selected: CT brain, CT spine cervical, CT neck with contrast, CT chest, with and without contrast, CT abdomen pelvis with contrast, CT renal tract, CT chest abdomen pelvis with contrast, and CT lumbar spine. Eight of the nine protocols align with current ARPANSA NDRLs. An additional protocol CT chest without contrast was also included in the analysis, due to its frequent use and availability of data.
CT procedures (n=11,956) across five CT scanners and four departments were collected and analyzed solely using DMS. CT scanners A, D, and E were Canon Aquilion Prime (160 slice) and scanner B and C were Canon Aquilion One (320 slice). For each procedure, the patient age of 15 years and above was selected. The data collection occurs automatically using the fully automated method, as dose data are immediately sent from the imaging station to PACS, then to DMS after a procedure. The studies were filtered for a period of 12 months from Q2 2022 to Q2 2023, and the CT acquisition type was filtered to only include spiral (helical) acquisitions.
Using the available analysis and graphing tools, the median (2nd quartile) of the max spiral CTDIvol and max spiral DLP was presented. The data in table format allowed for the total number of studies included per scanner to be known. The DMS also features visualization of the dose metrics using common graph types such as histograms, bar charts, and XY scatter plots. This was observed to be useful as an efficient method of visualizing the spread of doses across the selection period and population.
4) Semi-automated method
The semi-automated method begins with the same processing steps as the fully automated method. Data collection for both fully and semi-automated methods is performed on DMS. The process then differs beyond that point. Using the search filters, each individual scanner was selected by its unique AE title. The data were also filtered to a 12-month period. The entire dose dataset was then exported as bulk data from DMS in a comma-separated value (.csv) file for each CT scanner. Bulk export data were arranged by exposure, with each row representing an exposure and multiple rows constituting one CT procedure, containing different types of CT acquisitions. Only the helical acquisitions were chosen to be included in LDRL calculation. CT procedures (n=13,783) were selected across five scanners and nine CT protocols.
The cleaning and filtering of bulk data were performed using Microsoft Excel. Patient selection involved filtering out the non-adult population, ages 0–14 years. Protocol selection involved choosing one of the nine CT procedures, labelled identically as those in the fully automated method. Using Excel, exposures belonging to the same examination were grouped as they shared the same accession number and study date and time. It is important to regroup based on study time, as this handles the possibility of a patient receiving two examinations separated by a brief period of time. The scan range was then addressed, where any acquisition less than 2 cm was discarded, as those were not considered to be helical diagnostic acquisitions. This allowed the removal of acquisitions such as localizers, data acquisition scan modes, bolus tracking, and similar types of acquisitions from the dataset, as these can contain a large CTDIvol.
The semi-automated method differed significantly in the data cleaning and processing steps, as the additional steps were deemed necessary to ensure multiphase exams could be accurately analysed. Based on ARPANSA guidelines, multi-series exams covering multiple anatomical coverages had their CTDIvol averaged and DLP summed.
The research was approved by the South Eastern Sydney LHD Human Research Ethics Committee; the project was deemed to meet the requirements of the National Statement on Ethical Conduct in Human Research 2007.
Results
The results were collected for the manual method and for the automated method during the year from April 2022 to April 2023, from five CT scanners in a single LHD in NSW. A total number of 26,091 CT procedures between the manual and automated methods were collected. The study count per method of LDRL calculation for each CT scanner and protocol is given in Table 2. The blank data for the manual method was not available from two scanners (A and B), due to the department not participating in the NDRL program. The three methods of establishing CTDIvol and DLP are given in Tables 3 and 4, including the current ARPANSA NDRLs [5].
Two protocols were chosen to demonstrate the differences in the three methods as graphs. The 25th, 50th (median) and 75th percentiles of CT head and CT abdomen pelvis protocol CTDIvol and DLP, from the two automated methods are shown in Figs. 2 and 3.
The benefit of DMS can be demonstrated in Fig. 4, where the doses delivered from each scanner for each of the nine CT protocols can be represented together. Because of the large data set, the spread in the CTDIvol and DLP can also be graphed to better illustrate comparison with other CT scanners and NDRL.
Large datasets, which are a positive feature of using DMS, allow data to be presented as in Fig. 5, which shows the spread of CTDIvol values over a year for CT head procedures. The visualization may lead to awareness of increased number of outliers over time or between scanners. It can also assist in identifying procedures that may require optimization based on its large deviation from the indicated LDRL.
Discussion
1. Differences in Manual Method Procedures between Sites
The steps taken in the manual method were observed to differ between personnel undertaking the dose audit task. A major contributor to these variations was in the patient selection process and mainly how the patients included were distributed over time. One site selected the last 20 patients of standard size from the time of data collection, which ultimately resulted in patients being included solely from the last month or last quarter of the year. In contrast, another site spread out their collection period over the calendar year and collected small numbers of samples each quarter. The advantage of collecting samples over an entire calendar year allowed for seasonal changes in practice to be incorporated. In contrast, samples from a short time period, such as weeks or a quarter, may be influenced by external biases such as staff rotation, equipment performance changes due to service practices, or temporary drifts in imaging performance. The manual method therefore inherently exhibits user bias. Discussions with users on how they chose which particular exams to include in their sample of 20 involved the dismissal of outliers. One user would not include procedures where they found the dose metric to be significantly low or high. There were also discrepancies observed in how multiphase exam dose data were handled. One site took the largest CTDIvol and DLP value of an individual acquisition within a CT procedure, while another site’s protocol used single acquisitions throughout the entire scan region and quoted one dose metric from those single acquisitions.
Strategies have been identified to minimize user bias. These strategies include the establishment of standardized data collection and FRL calculation procedures that align with the guidelines outlined in the ARPANSA NDRL survey user guide for facilities participating in the ARPANSA NDRL service. Data collection procedures should explicitly specify that the data collection period is distributed evenly throughout the calendar year. Additionally, they should define a narrow weight range that reflects standard-sized patients to eliminate subjectivity regarding what constitutes a standard size. Furthermore, instructions should be provided on how to handle CTDIvol and DLP measurements for multiphase exams, discussed in the ‘Special considerations for multiphase CT exams’ section.
2. Comparison of Methodologies for Establishing LDRLs
Overall, the differences among the three methods show an acceptable agreement, ranging between 0% and 20%. This suggests that the automated method of establishing DRLs is viable, given the following considerations: The largest difference between the three methods was found to be between the manual and fully automated methods, with a difference of 20.4% related to abdomen pelvis procedures. This is thought to be due to the large range of patient size and imaging protocols, which give rise to shifts in LDRLs derived from fully automated methods compared to the 20 standard patients that the manual methods included in their ARPANSA submissions.
The discrepancy in study count is related to whether the system is connected to DMS and sending RDSRs. The manual method is acknowledged as a laborious task that requires a significant amount of time. The ability to filter out non-spiral CT acquisition types is an important feature of DMS, and for those systems that do not allow such filtering, it can lead to the inclusion of other acquisition types such as scout and bolus tracking, which can cause drifts in LDRL.
The fully automated method is as yet unable to appropriately process multiphase exams; therefore, either each phase of the procedure is taken as an individual procedure, or the maximum CTDIvol and DLP are taken.
The manual method offers the advantage of requiring fewer resources compared to the automated method. However, it is important to acknowledge its limitations, including the potential for user bias and data bias. These biases can arise due to variations in individual practices, subjective selection and interpretation of data, and human error at any point during the auditing and calculation process. Consequently, the manual method may not consistently provide accurate and unbiased results.
It is important to understand specifically what dose metric data are being used for dose calculations. If non-spiral CTDIvol values are used, this can lead to differences in the final LDRL.
The fully automated method provides a comprehensive solution for performing dose audits and these can be setup as dashboards to give the required data outputs when required during the year. The semi-automated method requires data processing tools such as database management software, and knowledge and skills in data cleaning and processing. Using readily available software such as Microsoft Excel, the initial setup is seen as the brunt of the work, and macros can be set up to further automate some of the manual data processing steps.
The software based method requires purpose-built software. It often involves complex implementation processes that require collaborative efforts from stakeholders such as vendors, department IT and PACS teams, radiographers, and medical physicists to fully implement and integrate the audit findings into clinical practice.
Despite these challenges, the automated method offers several advantages. It is significantly faster compared to manual audits. It can process large volumes of data in a shorter timeframe, enabling a comprehensive evaluation of radiation doses. This allows for large-scale dose audits, facilitating comparison and optimization for sites irrespective of their size in terms of imaging equipment.
The software based method offers a more inclusive and accurate dose audit due to its larger sample size. Utilizing software for data collection, processing, and analysis enables automation and ensures consistency, which reduces the impact of inter-user variability. This comprehensive approach improves the reliability of the audit results, providing a more complete representation of radiation dose distribution in medical imaging practices.
Manual methods in the past have meant that only the published protocols are examined (although not necessarily), but in the DMS system, other exams can readily and easily be included in the gathering and analysis process. This leads to the added benefit of automation with DMS systems.
One minor disadvantage related to the use of DMS for ARPANSA NDRL submissions is that there are many additional items of information that ARPANSA requests as part of their survey. Some of these data will not be available on a DMS that is simply receiving RDSR information from the imaging units. It is impractical to acquire this information for each of the over 100 exams, but for a select 20, it can be achievable.
3. Special Considerations for Multiphase CT Exams
A crucial factor to note is that extra care is required for multi-series CT exams, as the CTDIvol can vary significantly between different series, especially in cases where the anatomy is markedly different between series. A chest abdomen pelvis exam is one such case, where the CTDIvol for each anatomical section can differ from the others. The CT chest section will have a CTDIvol much lower than that of the CT abdomen or pelvis. The automated method was observed to only consider the largest CTDIvol and DLP values, whereas the semi-automated method, which performs a check for multi-series exams, calculates LDRLs based on ARPANSA recommendations. The difference in accounting for the variation in CTDIvol in multiphase exams did not appear to have a significant impact on CTDIvol LDRLs; however, the DLP was found to vary by as much as 50%. This is due to the requirement that the total DLP be summed and the CTDIvol averaged.
4. Limitations of the Study
The comparison between manual and automated datasets is offset by 3 months due to delays in the rollout and implementation of the DMS, particularly in the configuration of stations to generate RDSRs and come online. The unavailability of patient weight data in the DICOM header for the automated methods resulted in reduced analysis and sub-selection of standard patients. Although the DMS could not store patient weight data at the time of data collection, the vendor advised that this feature was planned and has since been added. The DMS now stores patient weight data, which can be utilised in the future for size stratified analysis which is particularly important for dose assessment in paediatric populations.
It may also be possible to extend these findings for use in other imaging modalities, such as fluoroscopic imaging, general radiology, and mammography, as all these modalities can send RDSRs to DMS systems. This was not confirmed in the current study, but future work is planned to confirm these findings for other imaging modalities, particularly those which deliver high patient doses (such as interventional procedures), significant population doses (such as mammographic screening), and those for which NDRLs are available (such as diagnostic coronary angiography).
Conclusion
The implementation and assessment of using a DMS for defining LDRLs, demonstrated that the use of a DMS facilitates the definition of LDRLs. These LDRLs compared meaningfully with the NDRLs published by ARPANSA. This approach is inclusive of a large fraction of the studies carried out using any particular protocol and can provide a relatively quick assessment of routine imaging practices within hospitals.
Dose audits of CT imaging examinations were investigated using two primary methods: manual methods and semi-automatic software based methods. While the manual method for radiation dose audits requires fewer resources, it is susceptible to user and data bias. In contrast, the automated software based method, though requiring a large initial investment in acquiring and commissioning specialised software and collaboration from various stakeholders, offers the benefits of greater speed, improved accuracy, and comprehensiveness.
The correct establishment of LDRLs is crucial, as it will trigger future efforts to change imaging procedures and optimize protocols, improving image quality and reducing the radiation dose delivered to patients. However, if the LDRL is improperly established or biased as a consequence of the methods used, it may lead to optimization efforts being based on misleading information. Considering this, the automated method demonstrated here offers a favourable approach for conducting CT radiation dose audits in medical imaging examinations, allowing for more efficient and standardized radiation management practices.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation 
Print 
