Technologically and linguistically adventurous EFL teacher, trainer, writer and manager

Posts tagged ‘assessment’

IH Academic Managers and Trainers (AMT) conference 2019

One of my two favourite conferences each year (along with IATEFL!) happened at Devonport House in Greenwich from 10th-12th January 2019. I was surprised to realise that this year’s AMT was my 6th – time flies! I’ve blogged about some of them: 2014, 2015, 2016. If you want to see photos from this year’s conference, take a look at the IH World page. You can also watch the video here:

I decided not to live tweet this time as we were given a beautiful notebook and my iPad is getting quite old and tired! Instead here’s a summary of the things from my notes which I think were most useful and/or thought-provoking.

In the classroom

Although the conference is aimed at teachers and trainers, there are always some sessions which are directly related to what happens in the classroom.

Engagement

Sarah Mercer spoke to us about the differences between motivation and engagement, and how to keep learners’ (and teachers’!) attention in a world full of distractions. She suggested looking at how video games do it, and taking some of those principles into our classrooms. We should make sure lessons are CLARA:

  • Challenging
  • Learner-centred
  • Active (what is the learner doing?)
  • Relevant/Valuable
  • Autonomy-rich

and that we incorporate GOSCH:

  • Goals (including interim goals)
  • Options
  • Surprise (through variety, promoting curiosity)
  • Challenge
  • Hooks (emotional, through storylines, and ensuring personal relevance)

Both of these acronyms incorporate the idea that in video games you can make the choices – you are the agent, not the audience – and there are easy wins at the start, with challenge building and immediate feedback throughout. Storylines in games create curiosity and there is a clear sense of progress.

I also agree with Sarah’s observation that teachers who’ve built good relationships with students have dramatically lower levels of discipline problems.

Sarah is continuing her research in the area of engagement, and I look forward to seeing more of her findings – there are certainly lots of ideas to explore here.

Assessment

Gordon Stobart has a UK state school background. He spoke to us about assessment for learning within the UK school system, and how it could be applied to language schools. A key ingredient is clear success criteria, answering the question:

What will it look like when I’ve done it?

If students don’t know that, it’s hard for them to even start working (definitely something I remember from Delta days!) Having clear success criteria means we can help students to work out which work best meets the criteria, give them guided practice to work towards achieving it, and give them clear feedback on how many of the criteria they have met and what to do to meet the others. These criteria can, of course, be negotiated with students – they don’t have to come from the teacher.

He mentioned Geoff Petty’s ideas of giving medals and missions which I like as a way of really boiling down feedback. To push higher level students, Gordon suggested missions like ‘What would you do if you only had half of this material?’ or ‘Argue the other side.’ The goal of all of this is self-regulating learners who can think for themselves.

In an aside, Gordon mentioned that he had one group who he used to jokingly start lessons with by saying ‘Previously in this course, we’ve looked at…’ in the style of a TV series. The learners said it really helped as they had often forgotten!

Autonomy

Katie Harris blogs about language learning at joy of languages. Her talk described what learning languages has taught her about teaching. I have to admit that I wasn’t sure about attending this, as I kind of felt like I’ve written about that a lot myself, but I’m glad I did. In the first half of her talk Katie covered what independent language learners like herself (and me!) do to learn, and in the second half she talked about a different way of approaching lessons that she has come up with as a result, which I definitely want to experiment with. Her suggestion is that for some or all of every class (depending on what else you have to do) you let students work on things which they are passionate about, for example TV programmes, books, or whatever else it might be. Here’s how a typical lesson might look:

  • Students share what they did and show each other the new words/grammar they found. Teacher circulates, answers questions and gives feedback.
  • Flexible productive tasks, such as mind maps, creative tasks (change the story, add a character etc), writing a diary entry from the perspective of a character, changing the language to a different register, I’m an expert on (for other learners to ask questions), etc.
  • Deal with emergent language.
  • Learner training.

The learner training is a key component, as you have to show students how to do things like access learner dictionaries and record new language. If you want to give them more structured homework, beyond just watching/reading more, you can give them questions like ‘Can you find examples of the structure XXX we studied last lesson?’ or ‘Can you find examples of new grammar which you think you’ve never seen before?’

The whole idea is that learners can follow what they are interested in, but that a qualified, professional teacher can help them get there faster than they would be able to alone. By doing this in a group with other people, they can share their interests and learn from each other.

Katie has done a webinar for Macmillan on the same topic if you want to see her talk about these ideas for yourself – I’d recommend it. I really want to experiment with this structure with one of my groups this year who I think would really benefit from it. I’ll speak to them about it in our next lesson, and will report back if I try it out!

Determination

Chloe Pakeman-Schiavone talked about the importance of helping learners to understand the reality of learning a language, while noticing the small achievements along the way. Building determination will help them to stick at it. You can do this by:

  • setting smaller, interim goals (as Sarah Mercer mentioned above)
  • making changes in support explicit – learners don’t always notice when you reduce support, for example by them doing something alone which they needed your help with before
  • helping learners spot determination in other people
  • creating a Positivitree – Chloe’s school has one in every classroom where students can add any achievements they want to, no matter how small they may seem to other.

In the training room

Intervening

Amy Blanchard investigated the role of the trainer during teaching practice (TP) on CELTA courses. She advocated interrupting TPs early on in the course if it could be beneficial to trainees, as long as both TP students and trainees know what is happening. The areas she particularly focussed on are the ones where we often find ourselves asking questions like ‘Should I be doing this right now?’ Examples might be:

  • Positioning
  • Instructions
  • Speed of speech
  • Boardwork
  • Concept checking

The benefits are that these interventions are often far more memorable than delayed feedback, which is generally at least a couple of hours and sometimes a couple of days after the lesson (if there’s a weekend in between), that trainees get immediate answers to internal questions, that you are training not just testing, and that information is given at the point of need. Caveats are that trainee and student expectations must be very clear, it requires you to read the situation carefully (it’s not suitable for every trainee), you should only intervene in ‘little’ things not big things that could change the course of the whole lesson, and that support should be withdrawn as the course progresses, so you definitely shouldn’t be intervening in this way in the final TP, and preferably not the last few. It’s also important that all interventions are followed up on in feedback, with action points reflecting the pre-intervention situation, as trainees still need to prove that they can do these things effectively without trainer intervention. Amy got very positive feedback from trainees who she used this technique with, and even months after the course they remembered it in a positive way. This was an interesting idea, and one I’d like to explore with trainees and fellow trainers on the next course I do.

Integrating training

Chris Farrell‘s talk was fast and full on – so many ideas that I couldn’t possibly get them all down, and I will be coming back to them again and again. He was talking about the work they have done at Embassy schools to support bottom-up teacher development. Some of the areas he covered were:

  • making sure that teacher development is an ethos throughout the organisation, not a separate activity (these talks from IATEFL 2018 are related to the kind of culture change that may be required) and that everyone is clear about what this ethos means and how it is communicated
  • evaluating teacher development (see below)
  • using nudges to drive cultural change, and knowing when a nudge is not enough
  • mentoring, particularly for teachers when they join the organisation, and the training needed for mentors to be effective. Senior teachers should not be forgotten here! (Please ask Chris if you want to find out more)
  • lesson aims, success criteria and assessment: making sure we know what the teachers are teaching and they do too, and that they know how to measure whether a learner and/or a lesson has been successful or not, as well as making it as easy and convenient as possible to see the links between these things (an area that bears a LOT more exploration!)

If you don’t know what the students are supposed to be doing, how can you know what you should be doing as a teacher or an organisation?

  • reflective enquiry, with different levels depending on how serious teachers are – these vary from notes and peer observation up to full-blown action research projects, and include professional development groups

Chris also mentioned that students can self-assess their ability to use particular language using a three-point scale:

  • I know.
  • I can use.
  • I do use.

Simple, but effective!

I suspect this is the talk I will come back to most from the whole conference!

Evaluating training

Silvana Richardson talked about an idea so simple that it’s never even occurred to me before: the importance of evaluating the impact of the continuous professional development you offer, both on the teacher and on student learner. I’ve never even asked for trainees to complete a ‘happy sheet’ as Silvana called them – an immediate post-session evaluation. That’ll be changing!

She talked about five levels of evaluation based on Guskey (2000):

1. Participants' reactions, 2. Participants' learning, 3. Institution's capacity to support change, 4. Participants' use of the new knowledge, 5. Students' learning outcomes

She covered a huge range of data collection techniques. Here are just a few.

Level 1 tends to just reveal the entertainment value, but is the easiest one to collect data on, including through using ‘happy sheets’. One way to make it richer is to ask ‘How are you going to apply what you’ve learnt today?’ or ‘What are you going to do with what you’ve learnt today?’

Level 2 could be done through exit tickets for example:

  • What I didn’t know before this session.
  • What I might need support with.
  • How I feel I have progressed as a result of this session.

Level 3 needs to be done at the level of the organisation, and may require institutional change. Silvana gave the example of an altered mobile phone usage policy following a session on mobile learning when they realised that phones were banned in the classroom.

Level 4 requires time to elapse: you can’t measure impact on practice instantly, and you may need to do it at several time intervals, though sometimes we forget! Silvana’s suggestion for this was learning walks, adapted from a system used in state schools. At Bell, they choose one area to focus on (student tutorials in the example Silvana gave), do some CPD based on that area, then drop in to lots of lessons to see how that CPD is being put into action. With the student tutorials, every teacher audio recorded tutorials with student permission, chose one to focus on, completed a feedback form they’d created as a team in a CPD session, had an ‘observer’ listen to the same recording and add comments, then all of the written feedback was anonymised and compiled into a single report. The organisation (it was done across multiple schools) learnt about what was and wasn’t working from their CPD sessions, and uncovered examples of best practice that had previously gone under the radar.

Level 5 is the hardest to assess, as so many factors could contribute to students’ learning outcomes. You can look at assessment scores, retention, changes in study habits, etc, or interview students, parents, teachers or managers to see this. However, it can be hard to assess cause and effect.

Evaluating your CPD programme in a range of different ways covering as many of these levels as possible is the only real way to ensure that it’s actually doing what you want it to do.

In the manager’s office

Curiosity

Monica Green encouraged us to nurture curiosity in ourselves as managers and in our teachers, inspired by this fascinating article from the Harvard Business Review. I really like this quote she finished on:

Albert Einstein on a bike: 'I have no special talents. I am only passionately curious."

Developing everybody

Olga Connolly reminded us of the importance of making sure that senior staff get professional development relevant to their role, not just teachers. For new senior staff at BKC IH Moscow, they have a shadowing programme and five training sessions based on core responsibilities like observations and how to give training sessions. For more experienced senior staff, they meet regularly to have discussions based around a table, the headings of which are:

  • skill/are to develop
  • why is it important
  • how (action points)
  • support needed
  • feedback collection
  • time frame

Senior staff complete what they can by themselves, then Olga helps them with the parts they can’t complete, and works out with them what support and guidance she/the school needs to give them. Examples of areas to work on which her senior staff have looked at include:

  • setting priorities to give more focussed feedback
  • improving body language in promotional videos made by the school
  • improving computer skills to be able to watch webinars
  • noticing strengths and weaknesses when observing lessons in languages you don’t speak
  • increasing the number and variety of warmers in teacher training courses.

This system came about because previously Olga noticed that there was no clear system, no goal and no focus for the development of her senior staff. That’s definitely something I’ve been guilty of, both in my own development and that of the senior team I work with – we’ve just kind of muddled along, though some things have become a bit more systematic as I have built up my own experience. Clearer goals would definitely be useful, though for myself endless curiosity (see above) tends to deal with a lot of things!

Change

Ania Kolbuszewska talked to us about why change does and doesn’t work. The know-feel-do model was new to me:

  • What is the one thing you want me to know?
  • Why do you want me to do this?
  • How do you want me to act as a result?

I like how this boils down change communication to the absolute essentials. She also reminded us that communication is NOT the message sent, but the message received, and that perceptions are an image or idea based on insufficient information – the more information we give to people about a change, the fuller their picture will be. This can help to reduce the amount of fear associated with changes, including fear of:

  • loss of money
  • loss of social or network traditions
  • loss of power
  • loss of control
  • loss of status
  • loss of jobs
  • not having the competences to unlearn old habits or learn new things
  • (not) being involved in the change.

My favourite quote from her talk was by C.S. Lewis:

It may be hard for an egg to turn into a bird: it would be a jolly sight harder for a bird to learn to fly while remaining an egg. We are like eggs at present. And you cannot go on indefinitely being just an ordinary, decent egg. We must be hatched or go bad.

By the way, if you’re interested in change management, I’d highly recommend reading Who Moved My Cheese? by Dr. Spencer Johnson [Amazon affiliate link]. I read it when I was in my teens, and it’s always shaped how I think about change and how to respond to it. It’ll take you all of about an hour to read and will give you a whole new vocabulary 🙂

Evaluation

Giovanni Licata and Lucie Cotterill reminded us that when evaluating courses, we shouldn’t rely on immediate post-course evaluation by students, as this often focuses on the entertainment/ performance value of the course, but try to investigate the longer-term effects on learning. We should also move away from star ratings – as they said, some of the ‘best’ restaurants in the world, and McDonalds, have very similar star ratings, and yet they’re doing very different things! One model you could use is KISS:

  • Keep (what are you doing to keep doing?)
  • Improve (what do you need to improve?)
  • Start (what are you going to start doing?)
  • Stop (what are you going to stop doing?)

In general

Communicating more effectively

Loraine Kennedy did a three-hour workshop entitled ‘The Craft of Conversations’ to kick the conference off. Among other things, she talked about developing emotional intelligence, coaching v. mentoring, and giving and receiving feedback, both positive and negative.

Here are five questions she asked us at one point which you might like to answer:

  • Why is emotional intelligence important in dealing with difficult people and situations?
  • Think about someone you think has high emotional intelligence. Why do you think this is?
  • “Know thyself.” Why is this important before judging others?
  • What can you do deepen you own self awareness?
  • What can you and your team at work do together to increase emotional intelligence?

She reminded us of our own role in any communication:

Your behaviour will influence the way the situation develops.

If you have a problem, you are both part of the problem and part of the solution.

The latter can be particularly hard to remember!

1. Description (what happened?), 2. Feelings (What were you thinking and feeling?), 3. Evaluation (What was good and bad about the experience?), 4. Analysis (What sense can you make of the situation?), 5. Conclusion (What else could you have done?), 6. Action plan (If it arose again, what would you do?)

Shared by http://www.researchgate.net under a CC 4.0 license

We practised using the Gibbs reflective cycle (shown above), as well as focusing on listening and asking questions, and not giving advice. I found this process particularly useful, as it made me realise that an unsuccessful and very negative interaction I had in my first year as a DoS probably came about because I was making statements and telling the teacher about a problem situation, rather than asking questions and helping them to describe the situation themselves.

At every AMT conference, there’s at least one idea which I’ve been struggling with in my own head for a while, and then somebody gives you the answer. In this case, it was Loraine’s guidelines for a complaint conversation:

  1. Prepare, prepare, prepare! Get as much information as possible, including more feedback from the complainant. Write a list of relevant questions.
  2. Explain the reason for the meeting, e.g. student feedback.
  3. Meet in the right place, and make it as comfortable as possible. Do not rush the meeting.
  4. State your position ‘on side with the teacher’, and remind them about confidentiality (yours and theirs). Remind them of the need to agree a way forward together.
  5. Ask the teacher to talk about the class and the students. Any issues?
  6. Outline the feedback received.
  7. Invite comment and discussion. Expect anger, embarrassment, denial.
  8. Listen and use exploratory questions.
  9. Support the teacher. Empathise.
  10. Reaffirm that a way forward needs to be found. Stay focused on this.
  11. It is better if the teacher finds the way forward, but be prepared to offer suggestions. (‘Way forward’ suggests that it is negotiable, it may have various steps, and the person the complaint is about is involved in working it out. ‘Solution’ suggests that there is one answer, and you may go into the conversation thinking that you know what it is.)
  12. Agree on action, and a time to follow-up.

The most important thing to remember is that a complaint must always be responded to, including if the response is that you do not believe that the complaint requires anything to be changed. Loraine also reminded us that if we have more teacher to student feedback, we may avoid complaints in the first place! If you want Loraine to help you out with management training, coaching, and teacher development, you can find out more information on her consultancy work on her website.

In a related talk, Lisa Phillips also talked about the importance of emotional and social intelligence, and making sure we:

  • Pay attention
  • Anticipate situations (both positive and negative)
  • Explain, don’t blame
  • Accept criticism
  • Remember about how contagious emotions are
  • Are human!

Questions I want to keep asking myself

What does success look like in this situation? How will I know when I’ve achieved it? How will my learners/teachers know when they’ve achieved it?

Are we doing enough teacher-student feedback? Are we doing it in the right way?

How can we promote curiosity, not just in learners, but in teachers, trainers and managers too?

How much am I taking what I know about what works as a language learner into the classroom? Do I really give them what I know works for me and a lot of other people?

How can we make our mentoring scheme as effective as possible?

What questions am I asking? Am I asking enough of them or jumping in with advice instead? Are they clear enough?

Am I really listening?

What am I doing to make sure I reduce how much of the problem I am in any given situation?

How can we evaluate what we’re doing more effectively?

I am *super* impressed! (guest post)

This post appeared in my facebook news feed yesterday, and I immediately asked if Tereza would let me share it on my blog. In it, she questions the value of positive feedback.

Today I received my evaluation of the final project in my sports class and it motivated me for a little contemplation on one of the differences between American and Czech (or even European in this respect) culture. The task was to create my own workout and lead my classmates for the fraction of the lecture. Eventually, due to time reasons, it was ONLY 5 MINUTES. So basically, all I did was I came up with 6 exercises, explained and demonstrated them to my classmates and then we performed them for 40s each with 5s break in between. The whole time I commented into the microphone like ’15 seconds, almost there, you can do it!’ because that was one of the requirements. You can see my evaluation below. My teacher was SUPER impressed, I looked like a professional, I should be an instructor.

Tereza's feedback

Tereza’s feedback

And here comes my point – really? I did not do anything impressive, I have never led a sport lecture before so I definitely have no motivation or other techniques developed and yet, based on 5 mins of doing stuff we have been doing in almost every class this semester, I should be an instructor! Americans are just always mega super trooper supportive to students, to kids, to each other, to everyone. Whatever you do, no matter how good or bad, it’s amazing. If you ask a question, however dumb one, teachers always start their answer with ‘That was an excellent question! I’m so glad you’re asking that.’ Whatever you do, it’s awesome, whatever you say, it’s so smart, whatever you wear, it looks cute and wonderful on you. One might think that there is everything perfect in America and everybody is talented and smart here. And that’s exactly mine (and not just mine) observation – people here really do think that. People are convinced that they are all brilliant at everything they do and look great in everything they wear. This might be a too big generalization, I admit. However, I can see evidence that it is mostly true every day.

My boyfriend teaches a calculus class at university in Missouri and his students, all future engineers by the way, are used to being praised their whole lives, getting excellent grades for everything and being told they can do everything and they are the best and the like as you could see on my evaluation. So those students are all shocked when they don’t get partial credit for accidentally guessing the right result, they are all surprised that there is someone who wants them to work hard for excellent grades and does not tell them ‘great job’ if the job is actually not that great. Instead of feeling ashamed they did not learn something or did not do the homework and therefore could not solve some exam problems, they go to him to complain, to accuse him that it is actually his fault they could not solve it and beg for extra points because they are used to always do great. Some time ago I posted here a ‘proof’ which one of my classmates did in a graduate-level math class. It almost made me cry, in short, she factored ‘x’ out of the integral which depended on ‘x’, they would not have let me finish high school if I had done that in the Czech Republic. So this girl still happily attends the class and I got the honor to read one of her papers we had to turn it. It was a complete disaster, she copied every single thing from the paper which it was based on, she not just copied it but also made a lot of mistakes in copying it, her sentences did not make sense, you could not call her proofs ‘proofs’ even if you were drunk and for all that she got 15 points out of 20. I wouldn’t give her even 10. However, that might probably touch her self-esteem and that’s not desired here.

I am not saying that being supportive and appreciating someone is bad. Especially with kids you should do that a lot. However, here it is led to extreme and moreover, college students are not kids anymore. Or at least they should not be. I have already lost the sense of what is meant honestly and what is just ‘American-like’. I basically have no measure whether I did well or bad because I always get a perfect evaluation. You have no idea whether people like you or how high they think of you because they always say you did a fantastic job. At the beginning, it makes you feel good, like you are really special, you do really so well. But with time, you get tired of that because you already see through it. Again, don’t get me wrong, I do not think teachers should be harsh on students, it is good to give someone encouragement and ‘push’ but not the fake one. In the Czech Republic or Germany where I got a chance to study, or even in my family, we do not flatter each other all the time. I know my parents love me and are proud of me but they do not tell me how amazing and talented and extraordinary I am every time I do something. Therefore, when they do tell me that, when they appreciate something I achieved or succeeded in, I can be sure they mean it and I value it very much then.

Tereza Eliášová is from the Czech Republic, and is currently studying for a semester in the United States. She was one of my students in Brno

Tereza

Tag Cloud