Trial Review

Technical difficulties have been reported by some users of the search function and is being investigated by technical staff. Thank you for your patience and apologies for any inconvenience caused.

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12619000412134
Ethics application status
Approved
Date submitted
20/02/2019
Date registered
13/03/2019
Date last updated
26/02/2020
Date data sharing statement initially provided
13/03/2019
Type of registration
Retrospectively registered

Titles & IDs
Public title
Building Capacity for Quality Teaching in Australian Schools - The effects of the Quality Teaching Rounds professional development intervention on student literacy, numeracy and science outcomes.
Scientific title
Building Capacity for Quality Teaching in Australian Schools: A Randomised Controlled Trial assessing the effects of the Quality Teaching Rounds professional development intervention on student literacy, numeracy and science.
Secondary ID [1] 297038 0
Nil known
Universal Trial Number (UTN)
Trial acronym
Linked study record

Health condition
Health condition(s) or problem(s) studied:
Educational outcomes (literacy, numeracy and science ability) 312012 0
Condition category
Condition code
Other 309669 309669 0 0
Research that is not of generic health relevance and not applicable to specific health categories listed above

Intervention/exposure
Study type
Interventional
Description of intervention(s) / exposure
Quality Teaching Rounds is a professional development process in which a form of instructional "rounds" (Elmore, 2007) are undertaken by a group of teachers in a professional learning community (PLC) (Lave & Wenger, 1991), with analysis and discussion guided by a pedagogical model, the Quality Teaching model (NSW Department of Education and Training [NSW DET], 2003). Groups of four (or more) teachers working in a PLC (not necessarily from the same school) undertake a set of rounds together over a period of 3-6 months. The teachers within the PLC are responsible for administering the rounds.

A "round" is comprised of sequential sessions that occur on a single day, involving:
1. Reading discussion: Designed to support the group in developing a shared theoretical basis for professional conversations and build a sense of professional community (typically one hour);
2. Observation: One PLC member teaches a lesson that is observed by all other members of the PLC (a full lesson length, typically 30-80 minutes); and
3. Coding and discussion: Individual coding of the observed lesson, including coding by the observed teacher, using the Quality Teaching framework (NSW DET, 2003), is followed by discussion whereby all PLC members contribute (typically one to two hours).

Lessons are coded using the Quality Teaching framework (NSW DET, 2003), which is made up of 18 elements of teaching practice (see below), within three dimensions of teaching and learning. Each element has a set of 5 descriptors, with elements coded from 1 to 5 depending on the agreement of the lesson with the element descriptor. The elements form the topics of professional discussion during each of the rounds undertaken.

Dimension 1. Intellectual Quality
Deep knowledge
Deep understanding
Problematic knowledge
Higher order thinking
Metalanguage
Substantive communication

Dimension 2. Quality Learning Environment
Explicit quality criteria
Engagement
High expectations
Social support
Students’ self-regulation
Student direction

Dimension 3.Significance
Background knowledge
Cultural knowledge
Knowledge integration
Inclusivity
Connectedness
Narrative

As a group of teachers undertake each round within a "set" of rounds, an implementation fidelity checklist is completed and submitted online. The fidelity checklist consists of:
1- a professional reading session was conducted
2- PLC members were present throughout the lesson
3- PLC members individually coded all Quality Teaching elements prior to the lesson discussion
4- the post-lesson discussion was at least 60 min in duration
5- the host teacher was included in the discussion
6- PLC members were present throughout the discussion
7- PLC members provided their codes and justification using evidence from the lesson for each QT element
8- the Quality Teaching Classroom Practice Guide was a consistent point of reference throughout the discussion

Further to teacher fidelity checks, a member of the research team attends a round for each of the PLCs within the study during the intervention period to cross check teacher understanding of the rounds protocols.

In Quality Teaching Rounds, at least one lesson is observed for every member of the PLC, and PLC members stay together for an entire set of rounds. The intent of QTR is to focus on the relationship between classroom practice and student learning and to show respect for the teacher and the teaching-learning process by watching a whole lesson each time (Bowe & Gore, 2017). During the intervention period teachers participating in the intervention will carry out a full "set" of Quality Teaching Rounds, with each teacher in a PLC being observed across the study period. Schools are funded for the teachers involved to be "released" from class for the four days they are undertaking QTR.

Training workshop: Prior to commencing rounds, two teachers from each PLC are funded release from class to participate in a two-day QTR workshop to prepare them for conducting QTR within their school. The training workshops will provide background information on the Quality Teaching framework and Quality Teaching Rounds, highlighting the intention and importance of each component of the approach (i.e. PLCs, readings, observation, individual coding, group discussion). Teachers will be given opportunities to practice the QT coding process and participate in simulated rounds using sample video-recorded lessons. This training is delivered by a member of the research team.

As the QTR intervention is designed to modify the teaching practice of those teachers involved, students in this study receive the intervention via the modified teaching practice of their classroom teacher.
Intervention code [1] 313320 0
Behaviour
Comparator / control treatment
There are three control arms in this study: 1) Trainer led-QTR group; 2) Peer-observation time-equivalent active wait-list control; and 3) Professional-development-as-usual wait-list control.

1) Trainer led-QTR group
This group receives the QTR intervention, however the training for teachers is delivered by a trainer that has been trained by the members of the research team. Teachers in this group are funded for the equivalent amount of teacher release as the intervention group to undertake QTR in 2019.

QTR Trainers were exemplary teachers targeted and recruited by members of the research team after responding to advertising from the New South Wales Department of Education (NSWDoE) and The University of Newcastle (UoN). Trainers undertake 28 days of training in the theory, and practical delivery of QTR two-day training workshops.

2) Peer-observation time-equivalent active wait-list control
Teachers in this group participate in an alternative form of Professional Development called Peer Observation. This method is endorsed by the Australian Institute for Teaching and School Leadership, the regulatory body for accreditation of Australian teachers. As with the intervention, two of the four teachers in a PLC are funded for two days of training in this method, and the four teachers involved in the PLC are funded for the equivalent number of release days as the intervention teachers for 2019

Teachers undertake peer observation in pairs, with the key components:
1- Pre-observation meeting: Identify focus of the observation, agree date/time and duration
of observation, and provide background and context for the lesson that will be observed.
2- Classroom observation: Observer records data – what the teacher and students say, do, make and write. Discreet interaction with observed teacher and/or students may occur if appropriate.
3- Post-observation debrief: Observer shares data collected relevant to the identified focus, observer and teacher share reflections
looking at connections between data collected and the teaching and student learning in relation to the teacher’s focus, observer poses questions to prompt further development, observer reflects on observations in
relevance to their own practice, and plan next steps – action to be taken using shared reflections to improve practice.

3) Professional-development-as-usual wait-list control
Schools in this group will continue their in-school professional development as usual, with no additional funding provided for professional development in 2019.

In 2020, teachers in the wait-list control groups will receive training in Quality Teaching Rounds (trainer-led version). This will not involve any data collection, and is purely to ensure that the benefits of QTR are made available to all schools participating in the research project.
Control group
Active

Outcomes
Primary outcome [1] 318642 0
Mathematics achievement.
Australian Council for Educational Research's Progressive Achievement Tests (PATs) for Mathematics (PAT-M 4), Each of these tests has multiple iterations which are developmentally appropriate for different school year levels (e.g. Year 2, Year 3). Analysis will utilize the test scaled score (interval scale from 0 to 100).
Timepoint [1] 318642 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)
Primary outcome [2] 319432 0
Reading achievement.
Australian Council for Educational Research's Progressive Achievement Tests (PATs) for Reading (PAT-R 5). Analysis will utilize the test scaled score (interval scale from 0 to 100).
Timepoint [2] 319432 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)
Primary outcome [3] 319433 0
Science achievement.
Australian Council for Educational Research's Progressive Achievement Tests (PATs) for Science (PAT-Science). Analysis will utilize the test scaled score (interval scale from 0 to 100).
Timepoint [3] 319433 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)
Secondary outcome [1] 365565 0
Quality of teaching
Quality of teaching will be based on observations of lessons. Quality of teaching will be measured using the scoring instrument in the Quality Teaching Classroom Practice Guide (NSW DET, 2003), which has been demonstrated to show high validity in previous studies (Ladwig et al., 2007).

This scoring instrument measures 18 elements of classroom practice grouped within three dimensions of classroom practice (Intellectual Quality, Quality Learning Environment, and Significance). Scores from 1 to 5 are generated in response to descriptors within each element for each of the 18 elements of the Quality Teaching model.

The coding process produces a score for each element that will be averaged within each of the three dimensions to produce QT dimension scores. Scores for the 18 elements will also be averaged to produce a QT overall score.
Timepoint [1] 365565 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)
Secondary outcome [2] 365567 0
Teacher outcomes.
Engaged Teachers Scale - Klassen, R. M., Yerdelen, S., & Durksen, T. L. (2013). Measuring Teacher Engagement: Development of the Engaged Teachers Scale (ETS). Frontline Learning Research, 1(2), 33-52. Addressing emotional engagement, cognitive engagement, social engagement toward students, and social engagement toward colleagues.
Timepoint [2] 365567 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)
Secondary outcome [3] 365575 0
Teacher outcomes.
Teachers’ Sense of Efficacy Scale - Tschannen-Moran, M., & Woolfolk Hoy, A. (2001). Teacher efficacy: Capturing and elusive construct. Teaching and Teacher Education, 17, 783-805 (SHORT FORM)
Timepoint [3] 365575 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)
Secondary outcome [4] 368161 0
Teacher outcomes.
Teacher Morale - Hart, P. M., Wearing, A. J., Conn, M., Carter, N. L., & Dingle, R. K. (2000). Development of the School Organisational Health questionnaire: a measure for assessing teacher morale and school organisational climate. British Journal of Educational Psychology, 70(2), 211–228. http://dx.doi.org/10.1348/000709900158065.
Timepoint [4] 368161 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)
Secondary outcome [5] 368162 0
Teacher outcomes.
Collective efficacy scale - Goddard, R. D., Hoy, W. K., & Hoy, A. W. (2004). Collective Efficacy Beliefs:Theoretical Developments, Empirical Evidence, and Future Directions. Educational Researcher, 33(3), 3-13. doi:10.3102/0013189x033003003
Timepoint [5] 368162 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)
Secondary outcome [6] 368163 0
Student outcomes.
Quality of School Life - Australian Council of Educational Research, Quality of School Life Questionnaire. 27 Questions addressing student perceptions of: 1) achievement, 2) general satisfaction, 3) adventure, 4) relevance, and 5) teachers.
Timepoint [6] 368163 0
Baseline (Term 1 2019, 11th Feb - 29th March)
7-month post intervention start (Term 4 2019, 4th Nov - 14th Dec)

Eligibility
Key inclusion criteria
In this study, the effects of QTR are sought in a sample of Stage 2 students (school years 3-4, Age 8-10 years), when the QTR intervention is delivered to their teachers. One of the four (preferably 2/4) eligible teachers recruited per school must be teaching Stage 2 students during the intervention year. An example of this structure is outlined below:

School 1:
- Teacher 1 (Stage 1)
- Teacher 2 (Stage 2) - Students invited
- Teacher 3 (Stage 3)
- Teacher 4 (Stage 2) - Students invited

Schools with any number of Stage 2 classes are eligible to participate.

Teachers who are available for the full period of the study, have regular access to a class of students, and have not undertaken QTR previously are eligible to participate.

Students of eligible Stage 2 teachers will be eligible to participate.
Minimum age
7 Years
Maximum age
11 Years
Sex
Both males and females
Can healthy volunteers participate?
Yes
Key exclusion criteria
Schools with no Stage 2 classes, or with no eligible Stage 2 teachers are ineligible

Teachers who have previously participated in QTR will not be eligible to participate.

Students with funded learning difficulties will be excluded from the study.

Study design
Purpose of the study
Educational / counselling / training
Allocation to intervention
Randomised controlled trial
Procedure for enrolling a subject and allocating the treatment (allocation concealment procedures)
Randomisation occurs after baseline assessment by an external researcher to maintain allocation concealment.
Methods used to generate the sequence in which subjects will be randomised (sequence generation)
Simple randomisation using a random number generator created by computer software.
Masking / blinding
Blinded (masking used)
Who is / are masked / blinded?


The people assessing the outcomes
Intervention assignment
Parallel
Other design features
This trial is designed to investigate the outcomes for students when their teachers undertake QTR professional development, or an alternate form of professional development. The study is designed around the primary outcome of academic achievement (Maths, Reading and Science) among Stage 2 students in Primary schools. QTR requires four teachers to be involved in the process, and the majority of schools do not have more than two Stage 2 classes. As a result, only the students of eligible Stage 2 teachers are invited to participate in the study, with teachers from other stages eligible to participate without their students being invited. The example below highlights the structural design and the measures undertaken by participants:

School 1:
- Teacher 1 (Stage 1) - No students invited: Teacher survey only
- Teacher 2 (Stage 2) - Students invited: Teacher survey and observation; Student achievement and survey
- Teacher 3 (Stage 3) - No students invited: Teacher survey only
- Teacher 4 (Stage 2) - Students invited; Teacher survey and observation; Student achievement and survey
Phase
Not Applicable
Type of endpoint/s
Efficacy
Statistical methods / analysis
Power calculations were conducted to determine the sample size required to detect changes in the primary outcomes in students (i.e. Progressive Achievement Test scores). As the goal of this study is to detect an effect for those receiving the Researcher-led QTR intervention (Group 1) compared with those in the business-as-usual wait-list control (Group 4), calculations are based on comparison of these groups, with sampling extended to the additional arms of the study (Trainer-led QTR and Peer observation control). For the two-group solution (QTR vs control), a total of 3988 students from 100 schools will be needed to achieve an effect size of d = 0.27 at 80% power with alpha 0.05. This sample is extended across the four arms of the study for a total of 7976 students from 400 teachers’ classes at 200 schools.

Outcomes analysis will involve multi-level linear mixed models assessing the impact of allocation (QTR, QTR-Trainer, Peer observation and Usual behavior), time (treated as categorical with levels baseline, and 7-months), and group-by-time interaction. The models will be specified to adjust for the clustered nature of the data (i.e., teachers located within schools) using random intercepts. Mixed model analyses are consistent with the intention-to-treat principle, assuming the data are missing at random (White, Carpenter, & Horton, 2012).

Moderators of intervention effects will be explored using linear mixed models with interaction terms for the following: (a) SES (based on school Index of Community Socio-Educational Advantage (ICSEA) values), (b) geographic location of school (urban versus rural), (c) years of teaching experience, and (d) sex of teacher. Subgroup analyses will be conducted if significant interaction effects (p < .1) are identified (Assmann, Pocock, Enos, & Kasten, 2000).

Fidelity checks will be conducted to identify PLCs that meet at least six of eight pre-specified standards, and per-protocol analyses will be performed on this sample.

Qualitative analyses using NVivo will also be conducted to explore responses from the teacher interviews.

Recruitment
Recruitment status
Recruiting
Date of first participant enrolment
Anticipated
Actual
16/12/2018
Date of last participant enrolment
Anticipated
13/03/2020
Actual
Date of last data collection
Anticipated
14/12/2020
Actual
Sample size
Target
200
Accrual to date
170
Final
Recruitment in Australia
Recruitment state(s)
NSW

Funding & Sponsors
Funding source category [1] 301607 0
Charities/Societies/Foundations
Name [1] 301607 0
Paul Ramsay Foundation
Country [1] 301607 0
Australia
Funding source category [2] 301628 0
Government body
Name [2] 301628 0
Australian Research Council
Country [2] 301628 0
Australia
Primary sponsor type
University
Name
Teachers and Teaching Research Centre, The University of Newcastle
Address
University Drive
Callaghan NSW 2308
Country
Australia
Secondary sponsor category [1] 301826 0
None
Name [1] 301826 0
Address [1] 301826 0
Country [1] 301826 0

Ethics approval
Ethics application status
Approved
Ethics committee name [1] 302335 0
The University of Newcastle’s Human Research Ethics Committee
Ethics committee address [1] 302335 0
University Drive
Callaghan NSW 2308
Ethics committee country [1] 302335 0
Australia
Date submitted for ethics approval [1] 302335 0
03/09/2018
Approval date [1] 302335 0
14/09/2018
Ethics approval number [1] 302335 0
H-2018-0340
Ethics committee name [2] 302710 0
NSW Department of Education
Ethics committee address [2] 302710 0
105 Phillip St, Parramatta NSW 2150
Ethics committee country [2] 302710 0
Australia
Date submitted for ethics approval [2] 302710 0
22/08/2018
Approval date [2] 302710 0
19/11/2018
Ethics approval number [2] 302710 0
2018458

Summary
Brief summary
This project seeks to understand the effects of various forms of teacher professional development on teachers' practice and student outcomes. This research involves a 4-arm Randomised Controlled Trial to test the efficacy of Quality Teaching Rounds (trained in two modes), alongside an alternative (Peer Observation) professional development approach and a usual professional development practice control condition. Data will be collected from participating teachers via surveys, lesson observations and interviews. Data will be collected from participating students in the form of Progressive Achievement Tests (PATs) and surveys.

We hypothesise that students taught by a teacher who has undertaken QTR will display greater positive achievement growth in mathematics, reading and science, and hold more positive perceptions of their own academic performance, their relationships with teachers, and the relevance of their schooling, compared with students whose teachers undertook alternate methods of professional development. We also hypothesise that teachers undertaking QTR will display gains in teaching quality, and improvements in morale, engagement, individual efficacy, and collective efficacy above that of teachers undertaking alternate methods of professional development.
Trial website
Trial related presentations / publications
Public notes

Contacts
Principal investigator
Name 89862 0
Prof Jenny Gore
Address 89862 0
University of Newcastle,
School of Education,
University Drive
Callaghan NSW 2308
Country 89862 0
Australia
Phone 89862 0
+61 02 4921 6709
Fax 89862 0
Email 89862 0
jenny.gore@newcastle.edu.au
Contact person for public queries
Name 89863 0
Prof Jenny Gore
Address 89863 0
University of Newcastle,
School of Education,
University Drive
Callaghan NSW 2308
Country 89863 0
Australia
Phone 89863 0
+61 02 4921 6709
Fax 89863 0
Email 89863 0
jenny.gore@newcastle.edu.au
Contact person for scientific queries
Name 89864 0
Dr Drew Miller
Address 89864 0
University of Newcastle,
School of Education,
University Drive
Callaghan NSW 2308
Country 89864 0
Australia
Phone 89864 0
+61 02 43484214
Fax 89864 0
Email 89864 0
andrew.miller@newcastle.edu.au

Data sharing statement
Will individual participant data (IPD) for this trial be available (including data dictionaries)?
No
No/undecided IPD sharing reason/comment
Sharing student or teacher data publicly is not in line with NSW Department of Education research policy.


What supporting documents are/will be available?

No Supporting Document Provided



Results publications and other study-related documents

Documents added manually
No documents have been uploaded by study researchers.

Documents added automatically
No additional documents have been identified.