Trial Review

Technical difficulties have been reported by some users of the search function and is being investigated by technical staff. Thank you for your patience and apologies for any inconvenience caused.

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12616001339448
Ethics application status
Approved
Date submitted
28/08/2016
Date registered
26/09/2016
Date last updated
11/02/2021
Date data sharing statement initially provided
11/02/2021
Date results information initially provided
11/02/2021
Type of registration
Retrospectively registered

Titles & IDs
Public title
Embedding simulation in clinical training in occupational therapy
Scientific title
A multi-centre randomised controlled trial, with embedded economic analysis, of the effect of a simulated clinical placement in comparison to a traditional clinical placement on occupational therapy student learning outcomes.
Secondary ID [1] 289742 0
Nil
Universal Trial Number (UTN)
U1111-1185-6699
Trial acronym
Linked study record

Health condition
Health condition(s) or problem(s) studied:
vocational rehabilitation 299588 0
physical rehabilitation 299589 0
Mental health 299590 0
Condition category
Condition code
Physical Medicine / Rehabilitation 299556 299556 0 0
Occupational therapy
Mental Health 299557 299557 0 0
Other mental health disorders

Intervention/exposure
Study type
Interventional
Description of intervention(s) / exposure
This project will assess a Simulated Clinical Placement (SCP) that aims to fully substitute 40-hours (within one-week) of Traditional Clinical Placement ( TCP). The SCP is designed to be situated in the foundation years of an occupational therapy program where the focus of learning is on the development of professional behaviours, self-management, communication and information gathering skills that are applicable in all professional practice placement/work settings.

The SCP will be designed to address the criteria stipulated by the Occupational Therapy Council for fieldwork replacement simulation-based learning hours with the aim of being eligible to contribute to the 1000 hours of clinical placement mandated within the Occupational Therapy Accreditation Standards (Occupational Therapy Council (Aust & NZ), 2013). The five criteria are:
1. Designed with a high level of authenticity for occupational therapy practice.
2. Designed with a high level of complexity requiring student engagement and interaction.
3. Delivered with immediacy to interaction with a real client (who may be portrayed by a standardised patient) and to occupational therapy clinical placements.
4. Designed and assessed with respect to meeting occupational therapy clinical placement objectives.
5. No one simulation modality can be used as a ‘stand-alone’ alternative to clinical training time (2010, p. 6).

Standardised scenarios and training materials have been developed to support SCPs targeting three types of units of study within Occupational Therapy degree programs: adult physical health and productivity, adult mental health and adult community based rehabilitation. Multiple case-scenarios (including scripts, case files and DVD films / web based resources) are developed for each of these settings and diversity of learning experiences.

Standardisation procedures. A research protocol and simulation implementation manual with DVDs for actors’ training has been developed. The SCP will be provided to students by academics and practice educators. All staff involved in implementing the SCP will receive training to ensure quality of SCP and implementation of RCT according to standard procedures.

Students attend the SCP on-site at the University in which they are enrolled and engage in a range of activities using a variety of media: face to face meetings with supervisors, engagement with on-line learning material, interviews with actor-clients, telephone interviews with actor-professionals, off-site work-site or community visit pertinent to the case-scenario, small group meetings, mock case conference, and student-led presentations of outcomes.

Each student experiences the SCP once for 40 hours in a one-week block over 5 days. This constitutes the total duration of the placement. Adherence is assessed using attendance logs.
Intervention code [1] 295380 0
Other interventions
Comparator / control treatment
The control condition will be an early year’s TCP experience of equal duration and distribution of hours to the SCP. That is, 40-hours within a one-week block of professional practice experience.
Control group
Active

Outcomes
Primary outcome [1] 299034 0
Student competencies: assessed using the Student Practice Evaluation Form – Revised (SPEF-R)
Timepoint [1] 299034 0
End of placement (after 40 hours)
Primary outcome [2] 299035 0
Students clinical reasoning and relevant professional knowledge will be assessed using a purpose designed Professional Practice Examination. Three examinations have been developed for the three clinical practice areas: vocational rehabilitation, mental health and physical rehabilitation. Each student will sit only one examination - the one pertinent to the practice area in which their placement occurred.

Development of each examination was undertaken using a team of experts to ensure validity, and examination marking will be undertaken by trained assessors, blind to student group allocation, who are trained in reliable marking. A structured marking rubric and sample correct answers are used to increase the reliability of the marking process.
Timepoint [2] 299035 0
Within 4 weeks of completing the placement.
Secondary outcome [1] 325933 0
Student self-confidence assessed using the Student Level of Confidence Questionnaire, a measure developed specifically for occupational therapy students by Derdall et al. (2002),
Timepoint [1] 325933 0
Assessment will occur within 1 week of the placement start (pre-placement assessment) and within 1 week of placement completion (post placement assessment).
Secondary outcome [2] 325934 0
Clinical educators perception of placement effectiveness assessed using a survey specifically designed for the study. Educators will also be invited to contribute to a focus group to provide more in-depth information about their perceptions of the placement experience.
Timepoint [2] 325934 0
Post placement assessment of educators' experiences will be collected within 1 week of placement completion by survey, and within 4 weeks of placement completion by focus group.
Secondary outcome [3] 325935 0
Student perception of placement effectiveness assessed using a survey specifically designed for the study. Students will also be invited to contribute to a focus group to provide more in-depth information about their perceptions of the placement experience.
Timepoint [3] 325935 0
Post placement assessment of students' experiences will be collected within 1 week of placement completion by survey, and within 4 weeks of placement completion by focus group.
Secondary outcome [4] 325936 0
Economic outcomes 1: Cost outcomes
The cost analysis to answer this question will be designed around ‘pathway analysis’ where each comparator in the trial is clearly specified in terms of activities (‘who teaches what to who, when, where and how often’). Other components of the cost analysis will include: i) the specification of costs by expenditure category (e.g. staff, capital; consumables; overheads; etc.); ii) the specification of attribution principles used where joint/common costs are involved; and iii) the clear specification of all costs included or excluded from the analysis.

Timepoint [4] 325936 0
Cost data is collected before (including time used and materials developed in preparation for the SCP and time and resources used to source and pay for TCP). The time period for the pre-placement data collection is dependent on University specific activity but will generally occur within the 6 months prior to placement. Cost data is also collected during the placement (both SCP and TCP) including costs of resources (including staff time, materials used, rooms rented) to implement both SCP and TCP.
Secondary outcome [5] 327889 0
Economic analysis 2
If SCP is not inferior to TCP but it costs more to deliver SCP than TCP, longer term benefits will be evaluated using cost-effectiveness ratios that have pedagogical and policy meaning, for example, time to complete the occupational therapy degree, participation in workforce. Desirably a decision threshold value is specified in terms of what constitutes ‘value-for-money’ with the chosen cost-effectiveness ratios. For example, with health sector interventions, ‘<$50,000 per quality adjusted life year gained’ is a common decision threshold.
Timepoint [5] 327889 0
If this analysis is required, data will be collected up to 3 years post completion of the placement.

Eligibility
Key inclusion criteria
All occupational therapy students who are enrolled in the identified unit of study in which the SCP and TCP are situated are eligible to participate. This includes:

1. 1st year undergraduate students (at end of year after 36 weeks of OT coursework, equivalent to 3 semesters of learning) from the University of Newcastle;

2. 2nd year undergraduate students from the Australian Catholic University; Curtin University; University of Sydney; University of South Australia; Deakin University

3. 1st year Graduate Entry Masters’ (GEM) students from the University of South Australia
Minimum age
18 Years
Maximum age
No limit
Sex
Both males and females
Can healthy volunteers participate?
Yes
Key exclusion criteria
Students repeating the unit of study in which the trial is embedded will be excluded from the trial.

Study design
Purpose of the study
Educational / counselling / training
Allocation to intervention
Randomised controlled trial
Procedure for enrolling a subject and allocating the treatment (allocation concealment procedures)
Once consent has been obtained, allocation to groups will be undertaken using an investigator concealed procedure to randomly allocate students within programs, to either the SCP or TCP group.
Methods used to generate the sequence in which subjects will be randomised (sequence generation)
Random Allocation Software will be used to generate the random allocation sequence of students to the SCP or TCP. This software is freely available from the internet and is described in peer-reviewed literature (Saghaei, 2004) 19. It is accessed at; http://random-allocation- software.software.informer.com/2.0/. This software enables control of different random allocation sequence attributes and produces lists for parallel group trials.
Masking / blinding
Blinded (masking used)
Who is / are masked / blinded?


The people assessing the outcomes
Intervention assignment
Parallel
Other design features
Multi-site
Phase
Not Applicable
Type of endpoint/s
Efficacy
Statistical methods / analysis
Sample size estimates:
The sample size was calculated based on our determination of an appropriate effect size to demonstrate non-inferiority. The primary outcome used for this calculation was the post placement examination (score range 0-100%). Non-inferiority was defined as a mean score difference between SCP and TCP groups of =7%. This value was based on the standard deviation of student examination scores obtained within the unit of study where the SCP was piloted at ACU in 2013. Our experience has been that more than 95% of students who undertake a TCP or the ACU-SCP meet acceptable professional competencies on the placement. An attrition rate of 20% was included in the sample size calculation to ensure that the analysis is adequately powered (power of at least 0.8, alpha level 0.05) to detect inferiority of the SCP, if present. Using standard power estimate criteria 120 students per group are required (total sample = 240). To account for clustering according to university site, where an average cluster size of 50-60 students is anticipated, and an estimated ICC of 0.007 an additional 100 subjects will be required (n = 340). Finally, to allow for a 20% attrition rate (340/0.8) a total sample of 425 is required.

All quantitative data will be cleaned and assessed for missing data. Demographic data will be summarised descriptively and displayed to demonstrate the adequacy of randomisation to distribute students equally according to key demographics (GPA, age). Where important differences between groups are identified, these independent variables will be included as covariates in the primary outcome analyses.

Question 1: To evaluate whether students who attend an SCP achieve non inferior outcomes to those who attend a TCP the following analyses will be undertaken:

a. Differences in odds ratios between SCP and TCP groups in predicting overall pass/fail grades on the SPEF-R will be evaluated using simple binary logistic regression. Independent variables included in the analyses as possible covariates will include sex, age, academic achievement, group allocation, university, broad placement type and any demographic variables identified in baseline analyses as important.
b. Differences between SCP and TCP groups in achievement of a pass/fail grade on each of the four completed sections of the SPEF-R will be assessed using chi-square analysis..
Differences between SCP and TCP groups in achievement on (i) the Post Placement Examination and (ii) overall Subject/Unit Grades, both scored on a 0-100 continuous scale, will be evaluated using Analysis of Covariance (ANCOVA). A mean difference of = 7 points on the examination and subject/unit grade between students will be considered an educationally meaningful difference.

Question 2: To evaluate whether students and clinical educators report similar professional practice learning opportunities during a SCP compared to a TCP, the following analyses will be undertaken:

a. The differences in frequencies of ‘not applicable’ and insufficient observation’ items checked in the total SPEF-R by (i) students and (ii) clinical educators will be compared using an independent samples t-test, using 95% confidence intervals of the estimated mean difference between groups to identify meaningful differences.

Question 3: To evaluate whether students who experience a SCP report non inferior levels of self- confidence during placement outcomes compared to those who experience a TCP, the difference between change scores on the level of self-confidence during placement questionnaire between SCP and TCP groups will be calculated using an independent samples t-test. Between group analyses of mean total score differences on this questionnaire, both pre and post SCP and TCP, will use independent samples t-tests. Within group differences will use paired sample t-tests.

Question 4: To assess whether students, clinical educators and university staff are satisfied with the placement experience, data will be extracted from the SPEF-R student review of professional practice placement survey, Student Evaluation of Placement questionnaire, and Clinical Educator Evaluation of Placement questionnaire, summarised and reported descriptively. The 5 point Likert- style item responses on each questionnaire will be reported graphically. Differences in item ratings between groups of participants will be assessed using independent samples t-tests.

Additional analysis (subgroup analysis and adjusted analysis)
Qualitative analyses of focus group data will be undertaken to explore themes arising from the experience of undertaking an early years SCP or TCP.

Sub group analysis will be undertaken on outcomes of those who withdraw prematurely from the trial. For participants who withdraw from the study prior to study completion, all data available on the participant will be used for analysis. No interim analysis will be undertaken in this study. There are no stopping guidelines in this study as the trial is not going to produce risk of harm to participants.

Question 5. Economic analyses
From a health sector perspective and from the perspective of participating universities, is the Simulated Clinical Placement (SCP) cheaper than the Traditional Clinical Placement (TCP).
To address this question, the method of economic evaluation is a cost-minimisation/cost-consequences analysis and the time horizon is equal to the duration of the RCT.

Cost data collection is to be undertaken to meet the requirements of economic evaluation conducted from two perspectives: the ‘health sector perspective’ and the ‘university perspective’. Only costs incurred by the universities or health care organisations are included under this scenario. The cost of successful clinical/simulation placement per occupational therapy student for each placement modality in each of the study sites will be estimated. This study will focus on: i) any differences in ‘average cost per student’ across modalities within the same site (which is what the research question is about); and ii) any differences in ‘average cost per student’ within the same placement modality, but across the study sites.

If costs of SCP are not cheaper than TCP, then a subsequent economic research question follows:

Question 6. Economic analysis
From a health sector perspective and from the perspective of participating universities, are there offsetting benefits that make the SCP a cost-effective alternative, even though SCP is more expensive than TCP?

The assumptions corresponding to Research Question 2 are that:
non-inferiority in the outcomes is demonstrated in the course of the RCT; but
1. non-inferiority in the outcomes is demonstrated in the course of the RCT; but
2. that the cost of SCP was more than the cost of TCP during the observation period.

In this scenario the economic appraisal will be taken to the next level of efficiency test, examining whether the long-term improved outcomes/cost reductions offset the increased costs. The time horizon for the economic analysis will be extended beyond the “within-trial” observation period to consider the long-term implications of SCP. The possible benefits that fall outside the observation period of RCT can be the improved time to completion of degree, improved enrolment into post-graduate occupational therapy courses; improved workforce participation of the occupational therapy graduates etc. This requires more complex forms of economic analysis and probable addition of the allocative efficiency (value-for-money) research objective. As this assessment must occur towards the end of the study when the trial results are in, this component of the economic appraisal will be indicative, rather than comprehensive.

Recruitment
Recruitment status
Completed
Date of first participant enrolment
Anticipated
Actual
14/12/2015
Date of last participant enrolment
Anticipated
28/10/2016
Actual
28/10/2016
Date of last data collection
Anticipated
5/12/2016
Actual
5/12/2016
Sample size
Target
425
Accrual to date
Final
570
Recruitment in Australia
Recruitment state(s)
NSW,QLD,SA,WA,VIC

Funding & Sponsors
Funding source category [1] 294124 0
Government body
Name [1] 294124 0
Australian Department of Health
Country [1] 294124 0
Australia
Primary sponsor type
Individual
Name
Christine Imms
Address
School of Allied Health,
Australian Catholic University,
Level 2, Daniel Mannix Building,
17 Young Street, FITZROY, 3065, VIC.
Country
Australia
Secondary sponsor category [1] 292952 0
Individual
Name [1] 292952 0
Eli Mang Yee Chu
Address [1] 292952 0
School of Allied Health,
Australian Catholic University
Level 2, Daniel Mannix building,
17 Young Street, FITZROY, VIC 3065
Country [1] 292952 0
Australia
Secondary sponsor category [2] 292953 0
Individual
Name [2] 292953 0
Stephen Guinea
Address [2] 292953 0
Faculty of Health Sciences,
Australian Catholic University
250 Victoria Street, FITZROY, VIC 3065
Country [2] 292953 0
Australia
Secondary sponsor category [3] 292954 0
Individual
Name [3] 292954 0
Loretta Sheppard
Address [3] 292954 0
School of Allied Health,
Australian Catholic University
Level 2, Daniel Mannix building,
17 Young Street, FITZROY, VIC 3065
Country [3] 292954 0
Australia
Secondary sponsor category [4] 292955 0
Individual
Name [4] 292955 0
Elspeth Froude
Address [4] 292955 0
School of Allied Health,
Australian Catholic University
Level 9, 33 Berry Street
NORTH SYDNEY, NSW
Country [4] 292955 0
Australia
Secondary sponsor category [5] 292956 0
Individual
Name [5] 292956 0
Rob Carter
Address [5] 292956 0
Deakin Health Economics,
Deakin University
Building BC, Room BC3.113,
221 Burwood Highway, Burwood, Victoria 3125 Australia
Country [5] 292956 0
Australia

Ethics approval
Ethics application status
Approved
Ethics committee name [1] 295539 0
Australian Catholic University Human Research Ethics Committee
Ethics committee address [1] 295539 0
Office of the Deputy Vice Chancellor (Research)
Australian Catholic University
Ethics committee country [1] 295539 0
Australia
Date submitted for ethics approval [1] 295539 0
30/10/2014
Approval date [1] 295539 0
20/01/2015
Ethics approval number [1] 295539 0
2014 310V
Ethics committee name [2] 295540 0
Curtin University HREC
Ethics committee address [2] 295540 0
Curtin University
PERTH
Western Australia
Ethics committee country [2] 295540 0
Australia
Date submitted for ethics approval [2] 295540 0
Approval date [2] 295540 0
11/06/2015
Ethics approval number [2] 295540 0
HR106/2015
Ethics committee name [3] 295543 0
Deakin University HREC
Ethics committee address [3] 295543 0
Burwood Campus
Postal: 221 Burwood Highway
Burwood Victoria 3125 Australia
Ethics committee country [3] 295543 0
Australia
Date submitted for ethics approval [3] 295543 0
Approval date [3] 295543 0
16/03/2015
Ethics approval number [3] 295543 0
2015-054
Ethics committee name [4] 295544 0
University of Sydney HREC
Ethics committee address [4] 295544 0
Level 6, Jane Foss Russell
The University of Sydney
NSW 2006 Australia
Ethics committee country [4] 295544 0
Australia
Date submitted for ethics approval [4] 295544 0
Approval date [4] 295544 0
04/09/2015
Ethics approval number [4] 295544 0
2015/664
Ethics committee name [5] 295545 0
University of South Australia
Ethics committee address [5] 295545 0
GPO Box 2471
Adelaide, South Australia 5001
Australia
Ethics committee country [5] 295545 0
Australia
Date submitted for ethics approval [5] 295545 0
Approval date [5] 295545 0
17/06/2015
Ethics approval number [5] 295545 0
0000034486
Ethics committee name [6] 295823 0
University of Newcastle HREC
Ethics committee address [6] 295823 0
The University of Newcastle
Callaghan NSW 2308
Ethics committee country [6] 295823 0
Australia
Date submitted for ethics approval [6] 295823 0
31/08/2015
Approval date [6] 295823 0
12/11/2015
Ethics approval number [6] 295823 0
H-2015-0289

Summary
Brief summary
Background:
Clinical placements are a critical component of the training for professional health workers such as occupational therapists. However, with increasing student enrolments and workload pressures on practitioners, it is becoming increasingly difficult to find sufficient suitable placements for all students to satisfy the mandatory 1000 hours of professional practise. The professional accreditation body for occupational therapy in Australia now allows up to 200 of the 1000 professional practice hours to be completed via simulation-based learning activities. The aim of this study is to provide evidence about the use of a simulated clinical placement (SCP) as a partial substitution for a traditional clinical placement (TCP).

Objective:
This project will evaluate whether ‘early years’ students from entry-level Occupational Therapy programs in Australia can obtain non-inferior professional competency outcomes at ‘acceptable’ costs through a 40 hour (1 week) Simulated Clinical Placement (SCP), when compared to similar students who attend an equal-length Traditional Clinical Placement (TCP). Early years students are defined as those in the first or second year of a four-year undergraduate degree program or first year of a two-year graduate-entry program.

Methods:
The aim will be met through the conduct a pragmatic, non-inferiority, single-blind, multicentre, two-group randomised controlled trial with nested economic appraisal.

We study will evaluate students’ professional occupational therapy competencies and perceptions of their placement experience after completing a SCP and compare these to the same outcomes for students who attend a TCP. It is hypothesised that students attending SCP will achieve non-inferior outcomes when compared to outcomes of students who attend a TCP. Secondary outcomes will include measurement of clinical supervisors’ and educators’ perceptions of the simulated experience, as well as a cost analysis and economic analysis of the delivery of SCPs and TCPs.

Trial website
Trial related presentations / publications
Imms, C., Chu, E., Guinea, S., Sheppard, L., Froude, E., Carter, R., Darzins, S., Ashby, S., Gilbert-Hunt, S., Gribble, N., Nicola-Richmond, K., Penman, M., Gospodarevskaya, E., Mathieu, E., & Symmons, M. (2017). Effectiveness and cost effectiveness of embedded simulation in occupational therapy clinical practice education: study protocol for a randomized controlled trial. BMC Trials. 18: 345. DOI 10.1186/s13063-017-2087-0.

Chu, E. M. Y., Sheppard, L., Guinea, S., & Imms, C. (accepted). Placement replacement: A conceptual framework for designing simulated clinical placements in occupational therapy. Nursing and Health Science Journal.

Public notes

Contacts
Principal investigator
Name 67642 0
Prof Christine Imms
Address 67642 0
University of Melbourne
Department of Paediatrics | MDHS
Level 3, West Building, Royal Children’s Hospital
50 Flemington Road, Parkville, Victoria 3052 Australia
Country 67642 0
Australia
Phone 67642 0
+61393454953
Fax 67642 0
Email 67642 0
christine.imms@unimelb.edu.au
Contact person for public queries
Name 67643 0
Prof Christine Imms
Address 67643 0
University of Melbourne
Department of Paediatrics | MDHS
Level 3, West Building, Royal Children’s Hospital
50 Flemington Road, Parkville, Victoria 3052 Australia
Country 67643 0
Australia
Phone 67643 0
+61393454953
Fax 67643 0
Email 67643 0
christine.imms@unimelb.edu.au
Contact person for scientific queries
Name 67644 0
Prof Christine Imms
Address 67644 0
University of Melbourne
Department of Paediatrics | MDHS
Level 3, West Building, Royal Children’s Hospital
50 Flemington Road, Parkville, Victoria 3052 Australia
Country 67644 0
Australia
Phone 67644 0
+61393454953
Fax 67644 0
Email 67644 0
christine.imms@unimelb.edu.au

Data sharing statement
Will individual participant data (IPD) for this trial be available (including data dictionaries)?
No
No/undecided IPD sharing reason/comment
Not included in original consenting process.


What supporting documents are/will be available?

Doc. No.TypeCitationLinkEmailOther DetailsAttachment
10585Study protocolImms, C., Chu, E., Guinea, S., Sheppard, L., Froude, E., Carter, R., Darzins, S., Ashby, S., Gilbert-Hunt, S., Gribble, N., Nicola-Richmond, K., Penman, M., Gospodarevskaya, E., Mathieu, E., & Symmons, M. (2017). Effectiveness and cost effectiveness of embedded simulation in occupational therapy clinical practice education: study protocol for a randomized controlled trial. BMC Trials. 18: 345. DOI 10.1186/s13063-017-2087-0https://DOI 10.1186/s13063-017-2087-0 
10586Ethical approval  elspeth.froude@acu.edu.au


Results publications and other study-related documents

Documents added manually

Documents added automatically
No additional documents have been identified.