This is my second NILE MA module, Materials Development for Language Education, abbreviated to MAT. I have previously complete the Trainer Development module. You can see my related blog posts here.
Here are various bits and pieces from week one of the course, things which I wanted to remember, notes I’ve made while reading, and on-going tasks we’ve been asked to provude. The notes are there for me, but you may find something useful in there, or something you’d like to investigate further. Please note: this is not intended as a subsistute for doing this reading yourself – it’s very subjective and based on my interests! I’ll post one of these in each of the three weeks of the online course.
Unit 1: Introductions
My metaphor for coursebooks is that they can be a guidebook:
- It shows you where you can go, but you can pick and choose.
- There are lots to choose from – different styles suit different people.
- Some people don’t bother with them and prefer to explore by themselves.
- People use it in different ways: some read cover to cover, some dip in at random, some know exactly what they’re looking for.
- You can pick up all kinds of interesting or unusual ideas from it.
- They can inspire you to want to try new things, or tell you more about places (methods) you were already familiar with.
- It can date quite quickly!
Initial beliefs about Teaching, Learning and Materials
These are some of my own beliefs about teaching and learning materials, compiled at the start of the course. The ‘But…?’ part is possible arguments others might have against these beliefs, not necessarily reflecting what I think. I haven’t included the counter-arguments as that would make it very long!
- Teaching and learning materials should be engaging for both learners and teachers.
Why? If teachers or learners aren’t engaged by the materials, they won’t want to interact with them, and they are less likely to be open to learning/teaching with them.
What does it entail? This involves having a clear and transparent aim for the use of the materials, which both learners and teachers can see will develop the English level of the learners. It also involves choosing engaging topics, with clear reasons for learners to care about the topics and the aims of the materials. Those reasons are most likely to come from helping learners to personalise the topic in some way and/or connect it to their own experience. Good design is also an important component of engagement – we have to want to pick up the materials / open the website.
But…? Who decides what is engaging? What role does the teacher play in bringing materials to life? What about self-study materials which need to be self-mediated? What about learners/teachers who feel uncomfortable sharing personal information?
- Materials should enhance and support the learning experience for all learners.
Why? If they don’t do this, then they’re making our jobs harder in some way! Materials which don’t support the learning experience add unnecessary barriers for learners and teachers, and can demotivate them.
What does it entail? A smooth User/Learner Experience (UX/LX) is important – finding your way around the materials easily and with the minimum of stress. This should be true for every learner, not just those who are neurotypical. We need to make sure as many learners as possible are catered for with our materials. This can be done through design aspects, such as our choice of fonts or spacing, as well as through the types of tasks and the options we provide within materials.
But…? How do we know that materials which work for one learner will necessarily work for another? Is there enough space in the materials to provide the necessary support? Or enough time to create materials with this level of scaffolding? Is it the materials job to do this, or should it be the teacher’s?
- Materials should provide opportunities for interaction.
Why? We learn better when we are actively involved, rather than passively receiving information. We retain new knowledge for longer.
What does it entail? This interaction could be with other people, for example sharing or explaining ideas. It could be interacting with the materials themselves, through creating our own notes (as I’m doing now!), diagrams, or summaries of the information. Each of these methods force us to process the content of the materials in some way.
But…? What if learners don’t want to interact with others or with the materials? What if they prefer to just be ‘fed’ information? What happens if you’re working with large groups? How can you manage noise levels during social interaction, or monitor effectively online, or check that they have processed information effectively when they interact with the materials by themselves?
- Materials should not just be about language; they should also include learner training, and, where necessary, teacher training.
Why? We often make assumptions that learners know the best way to learn, but this is rarely true unless they are very experienced language learners, and even then they might pick up something new. Teachers also benefit from support within materials – this is a very valuable avenue of professional development.
What does this entail? Materials should be accompanied by teacher’s notes, explaining the rationale behind methods used, and feeding in variations and extra ideas to support teachers, as well as cultural or other supporting information as appropriate. Learner training can be highlighted by feeding in ideas directly in learner materials, or via teacher’s notes, showing tips and tricks to help them become more effective language learners, and encouraging them to reflect on the learning process and what does and doesn’t work for them. This is particularly true of areas like revision and memorisation, where our instincts might run counter to what science shows are effective learning strategies.
But…? Is it the job of materials to teach teachers? How do you decide what assumptions you should have of learners’ language learning skills or teachers’ methodology knowledge in terms of what you decide to highlight/omit?
Note: I believe this is to some extent what Allwright (1981: 9) calls ‘guidance’ [see quotes below for full reference].
What do we want teaching materials for?
I found this quote from Allwright thought-provoking, partly because of my interest in classroom dynamics, but also because of how many people I know who think they ‘can’t’ learn languages because, I suspect, of attitudes that were ‘available to be learned’ in the classrooms they studied in:
It is well accepted that one of the goals of school language instruction is to improve the attitudes of speakers of different languages to one another. However seldom this may be achieved, the development of positive intercultural attitudes remains important, but it is not often discussed as part of the content of instruction. Even where attitudes are not being explicitly ‘taught’, however, they are almost certainly ‘available to be learned’ in any language classroom, from the teacher and from everyone present. They include attitudes to learning, of course, and not just language or intercultural attitudes. To summarize, anyone involved in the management of language learning has necessarily to deal with attitudes as part of what learners may learn.Allwright, R. L. (October 1981) ‘What do we want teaching materials for?’ ELT Journal Volume 36/1, p8
Another quote from the same article:
‘What activities, or what learning tasks, will best activate the chosen processes, for what elements of content?’ A less deterministic version of this question might be ‘What activities of learning tasks will offer a wide choice of learning processes to the learner, in relation to a wide variety of content options?’ This amendment suggests, I think correctly, that we can neither predict nor determine learning processes, and therefore perhaps should not try as hard to do so as we usually do in our teaching materials.Allwright, R. L. (October 1981) ‘What do we want teaching materials for?’ ELT Journal Volume 36/1, p9
It’s interesting that this quote is nearly 40 years old, and yet the concept of learner choice with regards to processes or content is still not really all that common within materials.
Allwright also mentions the implications for teacher training of his views of materials. Here are a couple of excerpts:
Teachers, it appears, seem to do ‘all the work’ and exhaust themselves in the process. [Allwright goes on to describe the results of this, such as failing to present the language to be learned as clearly as intended]
If, however, we entertain the possibility that teachers are not just doing ‘too much’ work, but doing work that the learners could more profitably be doing for themselves, the immediate implication for teacher-training must be that teachers need to be trained not to do so much work, and trained instead to get the learners to do more. Hence the concept of ‘learner-training’, since it is unlikely that learners will be able to share the burden without some preparation.
Teacher ‘overload’ often entails learner ‘underinvolvement’ since teachers are doing work learners could more profitably do for themselves.
‘Involvement’ means something akin to Curran’s ‘investment’ (Curran, 1972 and 1976), which suggests a deep sort of involvement, relating to the whole-person. [including decision-making and management of language learning]Allwright, R. L. (October 1981) ‘What do we want teaching materials for?’ ELT Journal Volume 36/1, p10
I think this is a balance many teachers, particularly those new to the profession, struggle with – they feel like they need to be seen to be teaching demonstrably to meet learners’ expectations. It’s a real challenge for them to let go. This reinforces my belief above about the importance of teacher’s notes and guidance in terms of how to use materials and how to learn effectively.
He goes on to suggest how teachers can share their expertise with learners, without imposing it on them, in order to make learners more independent:
I suggest that teachers, in addition to their role as ‘activities managers’ in the classroom, need to accept the roles of:
1. ‘ideas’ people, ready with practical advice about language learning strategies and techniques, both for classroom and for outside use;
and 2. ‘rationale’ people, ready to discuss language learning and justify their opinions and advice.Allwright, R. L. (October 1981) ‘What do we want teaching materials for?’ ELT Journal Volume 36/1, p14
For me, this demonstrates the importance of teachers being (language) learners themselves, as they can then share ideas and rationale that have worked for them. While it’s obviously possible to be an excellent language teacher without ever having learnt another language, I do think it can make a huge and very valuable difference (said as an avid language learner myself!)
This is the final sentence from the article:
The most important point for me is that materials should be related to the conception of the whole of language teaching and learning as the cooperative management of language learning.Allwright, R. L. (October 1981) ‘What do we want teaching materials for?’ ELT Journal Volume 36/1, p16
I feel like this is far from true in most materials and most contexts – the teacher uses the book they have chosen/had chosen for them, and they manage the language learning, with learners the somewhat passive recipients of this learning management, regardless of how active they may be in a given lesson. This teacher-/materials-mediated learning may fit into a broader plan of what learners are doing to improve their language, for example through self-study, but there is rarely a connection that could be described as ‘the cooperative management of language learning’.
Why use textbooks?
Robert O’Neill wrote a (kind of) response to Allwright’s article. This is my favourite paragraph from it, particularly the third sentence and the final one.
Even though technology has moved on a lot, and textbooks are more often than not ‘glossy, glittering products in full colour’, I think they are still good value for money and easy to use.
Further down the same page, we find:
In my opinion it is important that textbooks should be so designed and organized that a great deal of improvisation and adaptation by both teacher and class is possible.O’Neill, Robert (January 1982) ‘Why use textbooks?’ ELT Journal Volume 36/2, p107
I’m not convinced how possible this is in the market-driven production of coursebooks which we have today, in terms of how materials writers might put products together: everything needs to have its own USP, and be seen as a complete package. Having said that, this view has implications for teacher training and learner training: both need to know how to improvise and adapt materials as appropriate to meet language learning goals. O’Neill goes on to share his own implications for teacher training:
There can be no model of an ideal teacher, or lesson, or learner (or textbook). […]
A teacher-training programme must seek not to mould all teachers according to a pre-conceived notion of what teachers should be, but must try to build on the individual and differing strengths of each teacher so as to make the maximum effective use of that teacher’s qualities.O’Neill, Robert (January 1982) ‘Why use textbooks?’ ELT Journal Volume 36/2, p108
I think all I can say to that is: Amen!
O’Neill gives an example of a textbook unit with three different objectives designed to cater for learner choice. This is an idea I’d like to explore further, based on his statement that:
There are many ways of designing textbooks so that they can be used by a variety of learners with a variety of ultimate goals, and so they can be taught by a variety of teachers with a variety of teaching styles.O’Neill, Robert (January 1982) ‘Why use textbooks?’ ELT Journal Volume 36/2, p108
I found myself nodding along to the final paragraph of the article. It’s over a page long, but I feel like these excerpts summarise O’Neill’s ideas:
Textbooks can at best provide only a base or a core of materials. They are a jumping-off point for teacher and class. They should not aim to be more than that. A great deal of the most important work in a class may start with the textbook but end outside it, in improvisation and adaptation, in spontaneous interaction in the lcass, and development from that interaction. Textbooks, if they are to provide anything at all, can only provide the prop or framework within which much of this activity occurs. Textbooks, like any other medium, have inherent limitations. The authors of textbooks must make it clear what those limitations are.O’Neill, Robert (January 1982) ‘Why use textbooks?’ ELT Journal Volume 36/2, p110-111
The roles of English teachers and materials
This section is a copy of a(n over)long post I put in the forum – I doubt it makes sense without the article itself! The post was based on McGrath, I. (2013) Teaching Materials and the Role of EFL/ESL Teachers, Bloomsbury pp. 2-24
- Which of the models (Figs. 1.1 – 1.4) best represents your relationship (and that of your learners) with materials?
I think 1.3 (pdf p17) or 1.4 (pdf p18) most closely represent my relationship with materials, depending on who I’m using them with. With adult groups, I think I’d lean more towards 1.3, with learners taking more responsibility for creating content and with less of hierarchical nature in the relationship. With teens and young learners, I suspect it’s more 1.4, with the materials taking precedence in many ways, but trying to feed in bits and pieces of the children’s lives, largely because I feel less confident teaching them – the materials may serve a little as a crutch here too. My role is to try to reduce the distance between the learners and the materials, especially with teen books (the bane of my life!)
- How important for you are the advantages listed in Section 2.3?
As a new teacher, coursebooks were particularly useful for me, and I still find the ‘visible coherent programme of work’ (point 2) to be helpful, though I’m better able to make one of these myself now.
The time-saving element (point 1) is also very important, as it often takes much longer to create the programme of work and find the materials for it yourself than it does to riff off a coursebook.
I learnt a lot from Teacher’s Books, especially English File and Straightforward, when I was a new teacher, due to the clear methodological input in them (point 3).
I also agree with 4, 5 and 7, though I’m not so sure about point 6 in the age of the internet and (for many people, though not all) instant access to up-to-date cultural information.
I’ve found integrated resources to be useful as well, particularly photocopiable extras and suggestions for varying activities – seeing these has provided a lot of the input I’ve had in terms of my own ideas for materials design and different ideas for engaging activities for learners. As a DoS, having editable tests available has also been useful, although they often require a fair amount of work before I’m happy putting them in front of our students! Coursebook software is also very popular at our school, with the Oxford Discover Futures being the best example I’ve used so far.
- Or are you one of McGrath’s ‘doubting voices’?
Not catering for the whole person, etc: I think this has improved over recent years, though there’s still a real need for a well-trained teacher to mediate the materials for learners and bring them to life. The arguments about catering for different needs regarding who learners are likely to interact with are also being addressed in some modern materials, though there is still a way to go. A wider range of voices can be heard, not only British, American and the occasional Australian or Irish speakers, though they are very much still in the majority: “native speaker norms continue to dominate”. One example of a new course trying to change this norm to some extent is National Geographic’s Voices, which aims to take a more global perspective. The idea of a hidden curriculum or what Jill Hadfield calls a ‘covert syllabus’ is a very interesting avenue to explore too.
Not reflective of research, etc: While there is still work to do in this area, I believe most global coursebooks are now based on corpus research, though many are still heavily influenced by the grammar syllabus. Outcomes by Walkley and Dellar is an attempt to create a more lexically-led syllabus, while still having the overt grammar syllabus many stakeholders might look for in a coursebook. The most recent studies quoted were in 2010 – I’d be interested to see how this has changed in the intervening 11 years.
In coursebooks I’m aware of, task design has also improved, though this is again not universal. Some books still isolate grammar and present it out of context, but the vast majority of global coursebooks I’ve come across now use a reading or listening text to introduce new language points in context – they don’t always capitalise on this later in the sequence though, with the context being abandoned once rules or practice activities come into play.
The issue of misrepresentation and underrepresentation in coursebooks has also received ever more attention, though changes in global coursebooks are still somewhat glacial in pace! James Taylor and Ila Coimbra have worked on an independently produced series called Raise Up! which aims to be more representative of the real world than a typical global coursebook. I’ve also recently seen examples of other minor changes, such as a family featuring two female parents in a global teen coursebook (a Cambridge one, I think?) Gone, too, (I hope!) are the mother doing all of the housework or the female secretary supporting the male boss in images, though more diversity would still be good to see here.
Marginalise teachers, etc.:
(pdf, p12) “If teachers hand over responsibility for decision-making to textbooks, the argument goes, this reduces their role to that of mere technicians.” – if they are passive in this, then yes, but it is up to schools, trainers, and managers to make sure that this is not the case, and that teachers are supported in finding their way around the materials, and trained in how to exploit them effectively to meet learner needs. Teachers also need to tell learners why they are making changes: as Bolitho says (pdf, p20), “learners are entitled to know why they are asked to behave in certain ways…and how they can learn most effectively.”
(pdf, p12) “There is now a real danger that it is the coursebook which determines course aims, language content and what will be assessed.” – this was certainly true at our school to a large extent, but I disagree with the wording ‘a real danger’ (note: the section on Control, pdf p22, counters this statement in a way I agree with). For our brand new teachers, this was a boon – 80% of our teachers are in the first 3 years of their teaching career, and this enables us to provide some level of standardisation across the school and maintain a high level of quality in our general English and exam classes (potentially dealing with the deficiences/limitations of new teachers). We train teachers in how to exploit the coursebook and learn more about their group students to adapt it to their needs, as well as learning to critique materials and decide what is good and bad about them (moving towards a difference perspective, reflecting the Harmer quote on p14 of the pdf of reducing “unthinking coursebook use”, as well as the final paragraph of the whole excerpt about implications for teacher education). With 121 and ESP groups we may or may not use a coursebook. Our books are chosen in a (somewhat!) principled way by an experienced senior team who know the school well, and our typical students, somewhat because we are still working on developing these principles (the section on Choice from p20-22 of the pdf is interesting regarding this)
Unit 2: Learners and Context
The implications of context on materials
Here are three different ways in which context might vary, and my ideas about what implications this might have for a materials writer.
- Being ‘seen’ in the materials – not only portraying affluent people, but having a range of images shown or experiences described.
- Realistic target uses of language, for example writing focussing on a range of different genres, not only essays (these may only be relevant for those going on to further study), or doing what Bruno Leys described in The Grammarless Syllabus and focussing on functional language exponents rather than grammar study for learners most likely to use English in vocational contexts, such as working as a mechanic.
- Acknowledgement of challenges and affordances of people from different socio-economic backgrounds, e.g. time available for learning, money available to invest in learning/opportunities/extra materials/resources, space available for study – for example, materials which require learners to pay separately for access to audio which they then need a quite place and a strong internet connection to access may not be achievable for some learners. On the other hand, learners with a lot of time and money available may require materials which provide lots of in-built opportunities for extending their learning.
- Having quiet/loud variants of the same activity.
- Balancing the amount of individual and pair/group work.
- Providing information in teacher’s notes about which activities are likely to be noisier so that teachers can warn colleagues in advance.
Teacher’s training and experience
- The amount of guidance needed in teacher’s notes: balancing spoon-feeding with support.
- Providing opportunities for extending/adapting/reducing materials so teachers can use them flexibly.
- Being aware that materials are not always going to be used ‘as is’ – this may mean including information in teacher’s notes about which activities are reliant on other activities, and which can be used in a more stand-alone way or in a different order.
Considerations I need to remember when writing possible materials for students at IH Bydgoszcz
This is a selection of possible areas based on what we’ve looked at in this unit. I’d be interested to hear what you would add.
Will the materials be for very young learners? Young learners? Teens? Adult groups? Properly adult (i.e. 22/23+) or including older teens/university-age students too?
We teach everything from beginner to proficiency! Also, have students worked through our school to get to this level or have they joined the school at this level? That has implications for the ‘coverage’ of the level and how spiky their profile might be.
Assuming we’re teaching face-to-face, we have projectors and access to the internet. Teachers can also write on the whiteboard to highlight things on projected materials. Learners have coursebooks, so am I writing a coursebook unit? Or supplementary materials? Or stand-alone units?
Courses are generally 90-minutes x 62 lessons per year, running twice a week. Materials need to comfortably fit that time, with some flexibility for teachers to choose what to use. Time for assessment and building good group dynamics also need to be built in.
- Socio-economic profile
As learners can afford private language school classes, they are probably in at least a middle-income bracket. Many of our learners come from families with occupations such as medicine, teaching, law or engineering featuring strongly, or families own their own businesses. Manufacturing and agriculture are also strongly represented. As far as I know, students can all afford holidays, many of them abroad and often in quite far-flung places, despite the Polish zloty being relatively weak compared to the Euro/Pound/Dollar.
Catholicism is an important cultural influence, and caution should be exercised when dealing with potential ‘hot-button’ issues. Particularly controversial areas in Polish politics in the past few years have been abortion and LGBT rights.
- Number in class
Although some students have 121 classes, most students study in groups of 6-12. Materials should include opportunities to exploit the small group nature of the courses.
- Classroom layout
Student chairs have small desks attached which can be folded down out of the way. These can be arranged in many different ways. There is a teacher’s desk with connections for a projector, speakers and the internet – this can be move a little, but not much. There are two display boards in every classroom for student work and other important information. Materials can make use of the opportunity to reorganise the furniture, and to display information in different places in the classroom.
- Noise tolerance
Teachers generally expect other classes to be noisy at points and quiet at others, though occasionally parents complain if they think there is too much noise when they are listening from outside. Most activities that would be classed as noisy are possible within the school, provided they are balanced with quiet activities too.
- Collectivism vs individualism
Learners expect to have individual attention from the teacher, but are also happy to work in groups. Family is very important, and from my observation I believe it is the defining social unit in society. Learners who come from a family background which is considered non-traditional within Polish society may be reluctant to share this information as it can be potentially stigmatising, so this is an area to be treated with potential caution when writing materials. There is generally respect for people in positions of power, including teachers, though there may also be cynicism depending on the people involved. [Please note, these are my personal impressions and should be taken as such. These insights are very interesting and (possibly) more scientific, and seem to reflect at least some of my impressions.]
- Learner expectations
For YLs and teens who have come through our school, they expect engaging lessons with lots of speaking, a bit of writing, and enough of a language focus for a clear sense of progress. For adults, or teens joining our school after learning elsewhere, they tend to expect a strong grammar focus with plenty of speaking. Learners expect their teachers not to speak/know Polish, and for lessons to be completely in English, with materials fully in English to reflect that. Adult learners may expect ‘serious’ lessons, especially older learners who have been out of education for a long time. They may be reluctant to do activities which they feel are too childish or game-like.
Most learners are quite motivated, and if they aren’t, adults tend to quit the course. Teens may be forced to continue by their parents, though thankfully they are very much in the minority. Many students come to us for 6 or more years, working towards Cambridge First or Advanced exams over a period of time. They expect to be trained to succeed in these exams, so materials need to help them achieve this goal, while also catering for the smaller number of students who don’t want to take exams.
Learners (and parents) also expect high quality classes and to have a clear sense of progress over their time at the school. Materials need to factor in opportunities for assessment to help learners to notice this.
- Teacher’s training and experience
The majority of teachers at the school are within the first three years of their career, with an initial CELTA or CertTESOL certificate. Some come to the school with a little prior experience, but most may have only done a few weeks teaching, if any, before they join the school. Materials therefore need to provide guidance and support, be clear and flexible, and be accessible to early career teachers, without assuming too much prior knowledge about how they can be exploited. There is support at the school to help with this, but we also aim to make teachers as independent as possible, so materials which help with this would be a boon.
‘PARSNIP’ topics are often considered taboo. We were asked to consider whether these topics are appropriate or taboo in the culture we work in. These are my answers for Poland.
You’d really have to know your group, as politics can be very divisive and controversial in Poland, especially since 2015 or so. As mentioned above, issues such as the politics of abortion and LGBT rights are particularly divisive.
This should be fine, though portrayals of drunk characters may not be.
Poland is very strongly Catholic, and many issues are tied into religion. Questioning faith or the church would be very controversial. I would generally avoid this topic, unless it was a group I knew well and they specifically asked to be able to talk about it.
Because of religion as well as the politics of abortion, I think this would be a topic to avoid.
I don’t think I’ve ever come across any particular issues with this, but I’d avoid it as it may trigger religious or political topics.
- Isms (such as communism or atheism)
Both communism and atheism are probably topics to avoid, not least because of Poland’s difficult history. However, with a group you knew well who had asked to talk about them, they could be discussed civilly and safely.
This is Poland’s national meat 😉 so it wouldn’t cause any issues.
Beliefs regarding vocabulary in materials
These are some of my own beliefs about vocabulary in materials at this point in the course. The ‘But…?’ part is possible arguments others might have against these beliefs, not necessarily reflecting what I think. I haven’t included the counter-arguments as that would make it very long!
- Learners need to be exposed to the same vocabulary in context and in a range of different ways.
Why? Context aids both understanding and retention. A range of contexts helps learners to see the spread of where the vocabulary can and can’t be used. It also provides exposure to a range of typical co-text – other vocabulary and grammar which typically co-occur with the target items. Fluent language use requires collocational awareness, which cannot be developed if words only appear in isolation.
What does it entail? Providing repeated encounters with the vocabulary within the materials, for example in a reading/listening text, in vocabulary focussed activities (such as matching definitions), and in models for speaking/writing activities. Highlighting co-text and context and drawing attention to collocations.
But…? There’s a limited amount of space in materials and a large vocabulary load: how do you decide what takes precedence? How do you ensure that all vocabulary is encountered sufficiently without contrivance?
- Vocabulary work should provide opportunities for learners to use the vocabulary actively.
Why? Learners need to experiment with the language and get feedback on how successfully they’ve used it, for example whether they have chosen the correct vocabulary item for a given situation. Saying vocabulary enables them to practise the pronunciation, and writing it, to practise the form.
What does it entail? Including activities such as opportunities for personalisation, categorisation, and speaking and/or writing using the new vocabulary.
But…? We need to be clear whether vocabulary is introduced for receptive or productive use. There’s a large vocabulary load, and it’s difficult to provide opportunities to use all vocabulary actively. Learners may be reluctant to experiment with new vocabulary, particularly if they don’t feel confident about it, and may stick with vocabulary they already feel comfortable using.
- A key component of learning vocabulary is memorisation.
Why? If we don’t remember the word/chunk, we can’t use it! A larger ‘in-built’ vocabulary store allows more fluent use of English across all areas: reading, listening, writing and speaking.
What does it entail? Including memorisation stages in the activity sequence, and showing learners why these are useful and how they can work with the same techniques themselves. Including spaced repetition, and requiring learners to attempt to retrieve vocabulary from their memory rather than the teacher/materials always supplying the vocabulary for activities.
But…? We have translation software and dictionaries, so we don’t necessarily need to memorise vocabulary – we can look it up when we need it. Some people find it difficult to memorise language particularly if they have problems associated with their working memory, and others find it boring or demotivating.
How are materials evaluated at IH Bydgoszcz
We mainly evaluate materials when we look at the spread of coursebooks we use each year to help us decide what was (un)successful, what we want to keep and what we want to replace for the following year – this is ‘pre-use’. We also evaluate potential new materials to use at the school – ‘post-use’. We only do informal evaluation ‘in-use’, listening to teacher and learner comments about what they (don’t) like about books and considering how our use of them may need to change based on teacher/student needs as we go through the year – this is particularly possible if the senior team are teaching from the books themselves. These are some of the ways we evaluate materials:
- Flick test
First impressions of the book, including whether teachers/students are likely to want to pick it up, density of information on the page/throughout the book, general impressions of the design (for example, does it look old-fashioned?)
+ Provides a quick way to remember which book is which!
– Very superficial.
– Publishers expect this and might put the ‘shiny things’ in the top right corner to appeal to those flicking through.
- Teacher questionnaire
For books we’ve used previously, we have a short questionnaire for teachers based on various aspects of the book, including usability, general suitability for their groups, topics, engagement, level of challenge, grammar and vocabulary covered, skills work, whether they would want to use it again.
+ Gives teachers a say in the materials evaluation process.
+ They have first-hand experience of using the materials with students, so their opinions are valuable.
+ It tells us what teachers are looking for in coursebooks in general, informing our decisions about which ones to adopt.
+ Getting a range of opinions about the same books can tell us how they suit different teaching styles / groups.
– It’s not obligatory, so we only get a few responses.
– It can take teachers a while to complete.
– It’s very subjective.
– Teachers haven’t been trained to complete such questionnaires, and may only have limited awareness of what makes good or bad materials, especially if they haven’t been teaching for long and have little to compare their current coursebooks to.
– The questions were created by me based on previous experience, without necessarily having a grounding in theory.
- Trialling materials in class.
Some teachers might volunteer to test out a lesson or two from a coursebook we’re considering using.
+ We can see how it might work in practice, including possible student responses.
+ It’s practical, using the materials rather than just discussing them.
– It’s only a snapshot – sometimes one lesson has been fine, but the book as a whole has not worked for our school/ teachers/ students.
- Student feedback
Either based only on the book students have been using, or showing them a range of possible books for their level.
+ They’re the end users of the book, so they should have a say in what materials are chosen.
+ Students who have learnt English for a while have quite a good idea of what might be a good/bad English coursebook would be for them.
+ When they can compare books, students can be very responsible and offer considered and useful insights into the materials which teachers/ managers may not have seen.
– Some students don’t take it seriously.
– Students don’t necessarily have anything to compare the materials to, and they don’t have training in recognising good/bad materials.
– It can be very subjective.
– It can be quite superficial: for example, the design or the topics can influence them, without regard for the quality of the language work.
- Comparative evaluation
This is largely connected to the language and skills syllabus, looking at how the coursebook fits into our overall selection of coursebooks, what the progression is from one level or age group to the next is, and whether there is the coverage of language we’d like.
+ This helps us to provide some level of standardisation across the school, and maintains our sense of progression.
+ As it’s partly based on a list of grammar items compiled a few years ago, there is consistency from year to year.
+ We have practice at doing this now, so compare a wide range of different factors, for example: language clarification, topics, skills coverage, flow of units, length of units, and many others.
– There’s a risk of trying to find a book which is the same as ones we’ve previously used – we may be less likely to take a risk.
– We may end up focussing too much on the grammar syllabus, without considering other areas as much.
Materials and culture
I’ve put this paragraph here because I need to think about it – definitely requires some more processing before I can fully take it in I think!
Another interesting quote from the same chapter:
As readers, we should always be ‘suspicious’ of texts and prepared to challenge or interrogate them. However, in the foreign language classroom, texts are customarily treated as unproblematic, as if their authority need never be questioned. Learners, who may be quite critical readers in their mother tongues, are textually infantilized by the vast majority of course materials and classroom approaches.Pulverness, A. and Tomlinson, B. (2013) ‘Materials for Cultural Awareness’, page 451, in Tomlinson, B. (ed) (2013) Developing Materials for Language Teaching, pp. 443-459 (my emphasis)
This sentence is part of a section on ‘Critical Language Awareness (CLA)’, the idea that “language is always value-laden and that texts are never neutral” (ibid.) This is not something I’ve ever considered before, though I’m not really sure how you would go about remedying this in mainstream materials production, or even in the small amount of materials I’ll be creating for my MAT assignment. I wonder whether the increased inclusion of critical thinking tasks is enough, though ones I remember seeing don’t necessarily ‘challenge or interrogate’ texts in the materials. This is what they go on to suggest as a possible solution:
I think some of the questions they mention are reflective of some of the critical thinking tasks now included – I wonder how they would rewrite the chapter if they published it today?
Evaluation of materials
These notes are based on chapter 3 of McGrath (2002) Materials Evaluation and Design for Language Teaching, looking at close evaluation when choosing a coursebook. They are my summary of the main points of the chapter to refer to when writing my MAT assignments, so there’s not much commentary.
The first section concerns using a checklist. The examples of published checklists include the following variants in design:
- Rating systems
Value x Merit = Product (from Tucker (1975: 360-1)): Value rated 0-5, Merit 0-4
Weight / Rating: Ratings 4-0
Rating and comments: Ratings = Poor, Fair, Good, Excellent (1-4)
Yes / No / Comment
Tick boxes (next to some quite long questions, not always with yes/no answers!)
Yes / Partly / No: scored as 2, 1, 0 respectively
Pronunciation Criteria (3) / Grammar Criteria (4) / Content Criteria (3) / General Criteria (8)
General (4) / Speech (4) / Grammar (3)
Factual Details (16) / Factors (17)
No categories – only 3 long questions
Language content (5) / Skills (6) / Topic (7) / Methodology (7) – though somewhat misleading as the questions are long and often cover multiple areas
Does the book suit your students? (10) / Does the book suit the teacher? (10) / Dose the book suit the syllabus and the examination? (10)
- Criteria expressed as:
Noun phrases of 3-7 words
Statements of around 10 words
Nouns, occasionally with adjectives (max. 4 words)
Yes/No questions, all at least 10 words long
Yes/No questions, sometimes followed by information questions, all at least 5 words, but averaging at least 10
Yes/No questions, varying in length from 4 to about 20 words
Potential problems with textbook evaluations based on checklists (based on McGrath, 2002):
- If you decide to have a specific number of items in each category (like the final one which has 10 questions in each), you may exclude important information or include trival questions to make up the number. (p42)
- Weighting is complicated – it’s important to ensure that different items are weighted appropriately. (p42) This is especially important as weighting can help you to differentiate between materials which may seem to have a similar number of strengths and weaknesses. (p52)
- Having the same kind of response to every question might not be appropriate – some may lend themselves to a score, others to a comment for example. (p42)
- It’s important to only have one focus per question. (p42)
- You need to consider the difference between answers of ‘No’ and ‘Not applicable’, especially if connected to weighting. Do you ignore statements which are ‘Not applicable’? What does this do to your total scores if you have them? (p42-43)
- Transparency of criteria (p44) – “certain concepts […] may be unfamiliar to or only partially understood by potential teacher-users. (= you very much need to be aware of the target user your checklist)
- Criteria date – they need to “reflect new insights into language description, theories of learning and teaching and changes in society.” (p47)
- Evaluation is values laden. (p48)
- The conflicts “between breadth and depth, between informativity and economy, between the needs of the evaluator and the needs of the checklist designer – if these are different people, and between the forces of conservatism and innovation.” (p48)
- Making a final decision can still be difficult, as you might struggle to “reconcile strengths and weaknesses in the same textbook” (p53)
- You have to ensure validity and reliability, perhaps through arriving at a consensus for criteria (inc. involving end users) for validity, and carefully briefing evaluators for reliability. (p53)
- They can “encourage rather superficial judgements.” (p54)
McGrath (2002: 43) comments that while published checklists “vary considerably in their scope, form, detailed criteria and the terms used to describe criteria”, most make reference to:
Specific areas which criteria might I might want to include when compiling my own list, in no particular order and taken from throughout the chapter:
- Representation: gender, disability, ethnicity etc.
- Learner training
- Purposeful communication (key word!)
- Rehearsing for real-world target language use
- “Opportunities to express their own meanings in their own words” (p46)
- Balance between meaning/use and form
- Inclusion of pronunciation work
- Varieties of English represented
- Authenticity of language
- Opportunities for assessment
This sums up some of what I’ve written about elsewhere in this post:
The reality is that evaluation is value laden, and this will be less of a problem if evaluators (1) look critically at the criteria formulated by others; (2) are aware of their own values; and (3) in specifying criteria for use by others, investigate and take the values of the ultimate users into account.McGrath (2002) Materials Evaluation and Design for Language Teaching, p48
McGrath describes some of the potential conflicts inherent in creating evaluation checklists:
The challenge is to “minimise the chance of decisions being taken on the basis of individual subjective judgement.” (p48)
When deciding how to format a checklist, McGrath mentions the following (p48-):
- Including a summary of basics about the book at the top. (e.g. title, publisher etc.)
- Decide between (a combination of?) open-ended questions and questions/statements/prompts requiring a tick/score: the latter allows easier comparison and can be completed faster, the former adds information
- Consider the order of categories / criteria, including whether any overlap
- Rating, weighting and scoring:
Rating is often 3-5 points – picking 4 means the evaluator has to make a decision.
Weighting could be scored, or a system like A / B / N – absolutely essential, beneficial / preferred, not applicable (Skierso, 1991), rated as 4 / 2 / 0 if a numerical score is needed
Score = R(ating) x W(eighting) (p50)
Improving your evaluation:
- McGrath advises piloting a checklist if at all possible (p51), preferably against both a familiar and an unfamiliar book.
- Daoud and Celce-Murvia (1979) suggest group evaluation, by three experienced teachers. (p52 of McGrath), thus creating discussion, a more thorough examination, and shared responsibility.
- Teachers may need time to understand the checklist, especially important if different teachers have the responsibility for evaluating different materials. Some kind of practice (standardisation?) would be useful by working through a familar book and “checking that all would make similar judgements about its key features”. (p52)
- In addition to using a checklist, do an in-depth analysis of one or two units, along with analysing some specific features, for example the treament of a particular grammatical feature (Cunningsworth 1995 in McGrath 2002: 54). This “affords an insight into the view of language learning on which the materials are based” (McGrath 2002: 54). However, this can create a lot of demands on the evaluator, requiring a lot of effort and analytical expertise. (p55)
That’s it for week one. Next week: Units 3, 4 and 5.