Trial Review

Technical difficulties have been reported by some users of the search function and is being investigated by technical staff. Thank you for your patience and apologies for any inconvenience caused.

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12624000615583
Ethics application status
Approved
Date submitted
28/04/2024
Date registered
10/05/2024
Date last updated
10/05/2024
Date data sharing statement initially provided
10/05/2024
Type of registration
Retrospectively registered

Titles & IDs
Public title
Evaluating the ability of machine learning to predict hospital admissions from emergency department triage at St John of God Midland Hospital using data from 2016 to 2023.
Scientific title
Evaluating the performance of machine learning in predicting hospital admissions from emergency department triage, addressing concept drift and incremental learning at SJOG Midland Hospital using 2016 to 2023 data.
Secondary ID [1] 312028 0
None
Universal Trial Number (UTN)
Trial acronym
Linked study record

Health condition
Health condition(s) or problem(s) studied:
Health service research 333637 0
Condition category
Condition code
Emergency medicine 330322 330322 0 0
Other emergency care
Public Health 330323 330323 0 0
Health service research

Intervention/exposure
Study type
Interventional
Description of intervention(s) / exposure
A machine learning model (both a neural network (NN) and a extreme gradient boosting (XGB) machine) will be trained on the first year of data of all presentations to the SJOG Midland Emergency Department (2016). It will then use this data to prospectively move through chronologically from 2017 to 2023 to predict admission based off data recorded from patient triage. This will be recorded as area under the curve, sensitivity, specificity and accuracy The second phase of the trial will implement two self learning algorithms, one to the NN and one to the XGB, to assess whether this can improve triage based admission prediction accuracy. All patients with all medical conditions will be included. The specific exclusion criteria is only patients who passed away in the emergency department and patients who left the ED against medical advice/"did not wait". Given that all data has been totally de-identified and is not leaving the hospital's network for analysis, consent from every individual patient has not been requested by the ethics committee.
Intervention code [1] 328474 0
Early detection / Screening
Comparator / control treatment
No control group of patients. Though, a model without a self learning feature may be considered the 'control model' while the self updating model would be the 'intervention model'. All models will be trained, validated and tested on the same set of patient data.
Control group
Active

Outcomes
Primary outcome [1] 338074 0
Performance of the model is the primary outcome of predicting hospital admission between 2017 and 2023. Performance in machine learning comprises four measures: accuracy, sensitivity, specificity and area under the curve
Timepoint [1] 338074 0
Initial training Jan 1st 2016 to Dec 31 2016, no accuracy will be measured at this time. Testing will occur from Jan 1st 2017 to Dec 31 2023 data, with the primary outcome measure being measured continuously as a weighted moving average over time. It may be reported on any time frame, but we suspect it will be best reported either monthly or quarterly.
Secondary outcome [1] 434389 0
The secondary outcome measure is also a composite of: accuracy, sensitivity, specificity and area under the curve. However, the secondary outcome measure is comparing this composite of outcomes between the 'base' model that doesn't update over time, compared to the self updating/learning model between 2017 to 2023.
Timepoint [1] 434389 0
Initial training Jan 1st 2016 to Dec 31 2016, no accuracy will be measured at this time. Testing will occur from Jan 1st 2017 to Dec 31 2023 data, with the primary outcome measure being measured continuously as a weighted moving average over time. It may be reported on any time frame, but we suspect it will be best reported either monthly or quarterly. The comparison will be made at each timepoint.

Eligibility
Key inclusion criteria
All presentations to the SJOG Midland Emergency Department between January 1st 2016 and 31st December 2023. SJOG Midland ED is a mixed adult and paediatric department so patients of all ages will be participants.
Minimum age
No limit
Maximum age
No limit
Sex
Both males and females
Can healthy volunteers participate?
No
Key exclusion criteria
The only patients excluded from the dataset are patients who did not wait or passed away in the emergency department.

Study design
Purpose of the study
Diagnosis
Allocation to intervention
Non-randomised trial
Procedure for enrolling a subject and allocating the treatment (allocation concealment procedures)
Methods used to generate the sequence in which subjects will be randomised (sequence generation)
Masking / blinding
Open (masking not used)
Who is / are masked / blinded?



Intervention assignment
Parallel
Other design features
Phase
Not Applicable
Type of endpoint/s
Statistical methods / analysis

Recruitment
Recruitment status
Recruiting
Date of first participant enrolment
Anticipated
Actual
1/04/2024
Date of last participant enrolment
Anticipated
31/05/2024
Actual
Date of last data collection
Anticipated
Actual
Sample size
Target
550000
Accrual to date
500000
Final
Recruitment in Australia
Recruitment state(s)
WA
Recruitment hospital [1] 26473 0
St John of God Midland Public Hospital - Midland
Recruitment postcode(s) [1] 42460 0
6056 - Midland

Funding & Sponsors
Funding source category [1] 316379 0
University
Name [1] 316379 0
University of Notre Dame, Australia
Country [1] 316379 0
Australia
Primary sponsor type
Individual
Name
Dr Ethan Williams
Address
Country
Australia
Secondary sponsor category [1] 318561 0
University
Name [1] 318561 0
University of Notre Dame, Australia
Address [1] 318561 0
Country [1] 318561 0
Australia

Ethics approval
Ethics application status
Approved
Ethics committee name [1] 315187 0
Scientific Review Sub-Committee (SRC) of the St John of God Health Care (SJGHC) Human Research Ethics Committee
Ethics committee address [1] 315187 0
ethics@sjog.org.au
Ethics committee country [1] 315187 0
Australia
Date submitted for ethics approval [1] 315187 0
07/11/2022
Approval date [1] 315187 0
24/11/2022
Ethics approval number [1] 315187 0
2018

Summary
Brief summary
The purpose of this study is to build machine learning and AI models to predict admissions to hospital from just information available at emergency department traige. We look to address current gaps in the literature by exploring the effect of concept drift and will attempt to address concept drift to try to make these models more applicable to the real clinical environment.
Trial website
Trial related presentations / publications
Public notes

Contacts
Principal investigator
Name 133930 0
Dr Ethan Williams
Address 133930 0
Notre Dame Fremantle, School of Medicine, 32 Mouat St, Fremantle WA 6160
Country 133930 0
Australia
Phone 133930 0
+61434032522
Fax 133930 0
Email 133930 0
20160347@my.nd.edu.au
Contact person for public queries
Name 133931 0
Ethan Williams
Address 133931 0
Notre Dame Fremantle, School of Medicine, 32 Mouat St, Fremantle WA 6160
Country 133931 0
Australia
Phone 133931 0
+61434032522
Fax 133931 0
Email 133931 0
ethan.lw1998@gmail.com
Contact person for scientific queries
Name 133932 0
Ethan Williams
Address 133932 0
Notre Dame Fremantle, School of Medicine, 32 Mouat St, Fremantle WA 6160
Country 133932 0
Australia
Phone 133932 0
+61434032522
Fax 133932 0
Email 133932 0
ethan.lw1998@gmail.com

Data sharing statement
Will individual participant data (IPD) for this trial be available (including data dictionaries)?
No
No/undecided IPD sharing reason/comment
At this stage, the publication of unidentified patient data of every patient to the emergency department has not been agreed upon by the ethics committee, hospital or patient groups. This is for many reasons, including the sheer volume (over 600,000 presentations) and also concerns regarding it's use in data mining by large AI models.


What supporting documents are/will be available?

No Supporting Document Provided



Results publications and other study-related documents

Documents added manually
No documents have been uploaded by study researchers.

Documents added automatically
No additional documents have been identified.