These methods get results!

Hi everyone!

Shyam here again, coordinator of the Alcoa Champions of Maths program.

If you’ve been following our blog this year you will have heard some inspiring stories, seen some great tasks you can use in your classroom, and learned some powerful techniques for teaching mathematics effectively. Our eight teachers worked hard this year, training with me intensively, in developing approaches to teaching through problem solving, discussion and reasoning. I applied several levels of measurement to this project to evaluate its success and am proud to say we have some outstanding results. I’d like to share with you a summary here. Perhaps most significantly, the academic outcomes for the students in these teacher’s classes were well above average for the year. Not only do these methods achieve deeper thinking, collaboration, communication and engagement in a maths class, they also support better student performance.

Changes in teacher belief:

If you take a look back to our first blog post, I described the concept of teacher orientations taken from the paper Effective Teachers of Numeracy by Askew et al (1997). It was shown in this paper that teachers who hold a strongly connectionist orientation get high student gains. These are teachers who believe being numerate means being both accurate and efficient, that numeracy is best learnt by being exposed to the connections between different mathematical ideas, and that it is best taught through dialogue between teacher and student, building on and extending students’ own understanding. Discovery orientated teachers believe students should develop their own approaches in their own time and focus on providing materials, while Transmission orientated teachers believe students should replicate set routines that the teacher explains verbally and repeatedly if required. Teachers who have strongly transmission or discovery orientations achieve low student gains.

Based on this information, teachers in the program were introduced to an understanding of the orientations and encouraged to adopt techniques that were consistent with a connectionist approach. The connectionist approach is evident in the Alcoa Champions of Maths approach in that we start the students with problems and use their understandings to form the basis of rich, well-orchestrated discussions. We deeply explored many aspects of how to focus the learning on concepts and connections and how to generate powerful dialogue for understanding.

We measured the effect of this work on teachers’ belief by using a 14-question survey based on the aspects of orientation described in the Effective Teachers of Numeracy paper. The survey was given to teachers at the beginning and end of the year. Teachers could achieve a score from 0-14 for how strongly connectionist their orientation was.

These are the results:

An average shift of 28% toward connectionist orientation. Given that the scale caps at 14, an average score of 12.4 indicates a very strong connectionist group of teachers. This is an extremely satisfying result. Given that teacher’s beliefs significantly changed during the program and that connectionist teachers get higher student gains we would expect these classes to achieve significantly higher gains this year. Before we look at that, let’s consider some other measurements to corroborate the story that teachers have become more connectionist during the year.

Changes in teacher behaviour:

I used two measures of behaviour for this program, one in the first half of the year and the second in the second half. The first measure we used was observational and we called it extension moves. This is any move that a teacher makes that extends the dialogue pattern with a student beyond the basic Initiate-Respond-Evaluate dialogue pattern. An example of this would be:

Teacher: “What is 6 times 3?” (initiate)

Student: “18” (respond)

Teacher: “Correct.” (evaluate)

With an extension move a teacher will not evaluate after a student responds, but will ask, say or do something to get the student to extend their thinking by clarifying, generalising, adding on etc. A typical generic extension move would be to ask, “How do you know?” but more specific and sophisticated moves are possible. For example:

Students have been investigating the pattern in creating 4 arm tile shapes like: 

Student: The number of tiles is always four times the number in an arm, plus one more.

Teacher: Ok, great observation, what would happen if we removed an arm? (Extension move – teacher is prompting student to consider another case)

Student: It would be three times the number in each arm, plus one more.

Teacher: Right, could you explain what happens with any number of arms? (Extension move – teacher is prompting student to generalise relationship)

Student: I think it would be however many arms, times how many in each arm, plus one.

The teacher has made two extension moves here.

When doing our coaching rounds both myself and a second teacher were observing a teacher delivering a lesson. We would simply keep a tally of extension moves we believed we witnessed (only during whole class discussion). At the end of the lesson we would confer on our counts. I believe that teachers with a more connectionist orientation and hence more focussed on discussion, would be using more of these types of moves in their discussions. This is what we observed in the first half of the year, during which we had four coaching rounds:

We saw a rapid and significant increase in the first couple of sessions. In the fourth session we saw a slight drop-off which is unsurprising. Between round 3 and 4 I introduced the idea of talk moves (as described in Intentional Talk) with the teachers and they were encouraged to focus on using these in their discussions to generate more interaction with the whole class during class discussions. So, they were reducing the amount of times they were redirecting dialogue back at a specific student because sometimes they would be directing discussion toward the whole class instead. Once again these are very encouraging results supporting the story that teachers are adopting a more connectionist approach.

The second measurement was based on the talk moves themselves. Since there are so many, it was too difficult to count them all while observing a class. Instead teachers were surveyed on their confidence with the moves at mid-year and at end of year.

Blue = Mid-Year. Orange = End of year.

In all talk moves, the teacher’s confidence has increased significantly (except for wait time, which they all felt fairly confident with to begin). We also have hours of footage of the teacher’s demonstrating adept use of these talk moves during class. This is further evidence of their effectiveness at leading powerful discussions, an important characteristic of a connectionist teacher.

Changes in teacher confidence:

As the content of the program focussed very specifically on problem solving and reasoning in maths class, we asked teachers to rate their agreement with two statements on a Likert scale from Strongly Disagree to Strongly Agree, taking Strongly Disagree as a score of 0 and Strongly Agree as a score of 4, we saw these changes in their response from beginning to end of year:

I have a high level of confidence in teaching problem solving in mathematics. 34% average increase.

I have a high level of confidence in assessing reasoning. 34% average increase.

A strong shift in the teachers’ confidence in these two areas supports the idea that the program is meeting it’s intended aims of increasing teacher capacity in teaching through problem solving and reasoning. The large collection of lesson plans and previous blog posts constitutes a large body of evidence demonstrating their aptitude with teaching problem solving lessons and reasoning skills.

Effect on student performance:

Finally, the punchline. The most convincing data in today’s environment are student test results and we did not shy away from collecting this data during our program. All teachers participating in the program used the 4^th edition PAT-Maths tests. These are a standardised test that assess general knowledge across the whole curriculum for each year level. The tests are normalised across year levels, so we can identify expected scores and growth from beginning to end of year. Based on these norms, and a model developed by Tierney Kennedy, I have been able to obtain an estimate of gains beyond expected growth in years of additional gain. These are the key results for 8 classes:

The growth in scores for Class 7 is an additional 1.6 years above the expected growth. The cohort on average gained an additional 0.6 years of learning in 2019. These are fairly extraordinary results. All classes achieve above expected growth for the year. Those classes who showed greatest additional gains were the classes with teachers that had adopted the approaches of the program most thoroughly.

For most teachers, we do not have comparative data (i.e. growth in scores of their classes for previous years, or growth in scores for their students in previous years.) We do have this in a couple of cases. Where we were able to make this comparison (in two classes), we found growth of the students this year compared to last year was four times the growth. In the one case where we were able to compare the teacher’s class growth to their previous year’s class growth, we saw 1.5 times the growth.

All of these results strongly indicate that these methods support highly effective mathematics teaching resulting in large student gains.

I’m looking forward to another year of powerful maths learning and teaching in 2020 when we run the program again with a new cohort. Keep following for inspiration and learning from the Alcoa Champions of Maths Program.

Keep thinking and enjoy the struggle!

Shyam

Find out more about the Alcoa Maths Enrichment Program or read the next article.

Would you like to continue a discussion on this topic with other educators? Join our Facebook group!

If you are interested in joining the Alcoa Maths Enrichment Program please fill out the enquiry form below and a Scitech representative will be in touch with you.