Three Test Design Tips for EFL & ESL Teachers

--- Advertisement ---

There’s one task that teachers seem to dread: designing tests. I completely understand. It seems quite hard to test everything both fairly and in a way that’s easy to administer and mark. Below are three tips, however, to set you on the right track.

I can’t, of course, cover the whole scope of test design in this short article, but I can offer a few points to consider when making your own English tests.

Some teachers probably make their entire exam from scratch, but I’d say most of us take existing “end of unit” tests and cobble together bits and pieces to make our own class tests. I see nothing wrong with this approach in and of itself. However, there are some pitfalls that you need to be aware of when doing this.

Probably the most prevalent is not being critical enough of the published materials we have access to for making our tests. Simply because it’s published material doesn’t mean that it’s good. Or that it’s a valid testing instrument. I know, I know… you’d hope this would be the case, but sadly I’ve found a lot of instances over the years where it is not. Don’t get tricked into thinking “material from a big publishing house automatically equals correct and good”.

For this article, let’s take a listening task as an example. We could have looked at any kind of test (and depending on what kind of comments people leave, I may write a whole series of articles on different types of tests: speaking, reading, writing, grammar, vocab), but for the moment, let’s just take a listening task as our example. The points below can be applied to the other task-types too.

#1. Make sure the task actually tests what you want it to.

This one may seem obvious, but believe me, there’s more than meets the eye.

You have to look carefully at the task and ask these two critical questions:

(a) What do the students have to do?
(b) What do the students have to be able to do?

[Sidenote: This is also critical for giving clear and simple instructions, but we'll discuss that another day]

Write down the answers to these questions in note form. For example:

(a) They have to listen and choose the correct picture from the set

This first part is usually pretty easy. The second one is where you can get tripped up if you’re not careful. To stay with the example of a listening test task for a moment, look at the tapescript. Which language in the tapescript guides students toward the correct answer? What do they need to be able to understand here in order to arrive at the correct answer? Is there a distractor? If so, how does it work? Which area or point of possibly confusion does it rely on?

You might come up with something like this:

(b) They need to know the names of farm animals and prepositions of

location. The key thing seems to be knowing the difference between

“in front of” and “behind” and not being tricked when it says “beside”.

When approached like this, you’re making sure that the requirements are valid for your learners. If they haven’t covered BOTH of these points (i.e. that particular lexical set and the grammar point) then you can’t use the question; it’s unfair.

This is only the first part of #1: “Is it testing what it claims to be testing?”

The second part is whether or not it tests listening skills (in our example). If they can answer the question based on their knowledge of the world, for example, then there is no need to listen—in which case the question doesn’t test listening skills.

It’s hard to think of an example using a simple listening task like the position of animals on a farm for this :lol: so let’s take a more advanced level question…

Imagine students have to listen to an interview with someone on a general topic and either choose answers from options (multiple choice) or write short answers. If the students can answer ANY of the questions from their own general knowledge, then those questions are not testing listening skills.

The answer, of course, will be in the interview, but if it’s not necessary to actually hear the answer in order to answer the question, this question is not a test of listening skills.

You need to remove these questions altogether. Then, either write a new question that’s similar but can’t be answered from general knowledge (i.e. you have to listen and understand in order to be able to answer it). Or just cut it and adjust the score for that part of the test.

Another one, similar to this is where there are four pictures, the students listen to four different statements on the CD about the pictures and then match 1-4 with A-D. I’ve never understood this one. If they’re confident with their first three answers, then there’s really no reason to listen to the last one. Of course, they WILL listen, but there’s no actual test there.

This one is easy to fix. Just add another picture from the same artist. Just take it from somewhere else in your teacher’s book. Cut out the existing pictures from any kind of frame that they’re in (which will obviously show the extra one as not belonging!) and add the new pic in in. Then shuffle them around and re-stick them to your test sheet Master. Write in the “A”, “B”, “C”, “D”, “E” by hand so that the extra one doesn’t stick out.

Make sure, of course, that the picture can’t actually be interpreted as the answer! This is highly unlikely, but you should probably double-check.

--- Advertisement ---

These ads are served by Google and are, hopefully, tailored to the content of this page.
If you see anything you think is inappropriate, please contact me and, if possible, I'll block it.

#2. Don’t ask for more than is absolutely necessary to test what you want to test

Similarly, if you want to test listening skills, keep the response or required action to the simplest possible level you can. Don’t, for example, ask students to write their answer in words if circling a picture will suffice.

And don’t mark an answer wrong simply because the spelling is incorrect. Is this part of the test a listening test or a spelling test? (I’ve seen this a lot.)

The only exception I can think of to this is the IELTS-style question in which someone spells their name or the name of a place or brand or whatever. They say the word and then spell it out letter by letter (e.g. “My name is Leslie, ‘L-E-S-L-I-E’. No, not ‘E-Y’, ‘I-E’.”) In this type of question, listening to the exact spelling is what’s being tested.

All other instances of spelling mistakes should be marked correct as long as you can see what the student meant. Obviously, in cases where you have no idea what they’re trying to communicate, you can’t mark it correct. These judgement calls are tough because it may be that the student does, in fact, have the correct answer but absolutely no idea how to spell it.

Which is, of course, more argument for keeping the required response to an absolute minimum. “Ah, but… what about guessing?” Yeah, agreed. This is a problem with multiple-choice questions. That’s why I actually prefer the one-word or two-word answers for students who have reasonably good writing skills (otherwise it’s placing too much load; Is this a listening test or a writing test?).

The key here is to be VERY confident, as the teacher, that even the weakest student in the class can fairly accurately communicate the correct answer even if their spelling is awful. Remember, it’s a listening test, not a spelling test.

And if a simple action can demonstrate that they’ve understood, then go for that. It may mean drawing some pictures. A lot of teachers worry about not being an artist. Really, stop worrying. Stick figures are fine. Your job is to make a good test, not a work of art. If some stick figure pictures is the best route, then start sharpening those pencils. :wink:

#3. Is the answer too obvious?

In contrast to the previous point, I’ve seen a lot of published listening tests where there is no “distractor”. (This is, in a way, very similar to Point #1, to be honest).

Let’s take a very, very simple example task: Students listen to a statement and then choose the correct picture.

In these tasks, it’s usually some grammatical point that’s being tested. Or so the test-writer thinks. Actually what happens is that it’s possible for the students to entirely tune out the grammar and answer the question from the key vocabulary alone.

An example: Three pics of three people doing three different things (e.g. playing piano, swimming, walking the dog) and three statements. They listen and match. The first statement says something like “Sarah is taking her dog for a walk” and so on.

Unless this task is for SUPER elementary learners, it is far, far too obvious. And it’s not a test of grammar, it’s just a test of vocab. Which is fine if it’s what you want it to test. But if you want it to test an understanding of grammar, then there’s a problem.

One solution might be to add some pictures and alter the statements slightly (and then just read them yourself instead of relying too heavily on the CD).

For example, 9 pictures; 3 statements. Picture one: Sarah taking her dog for a walk in the morning. Picture two: Sarah taking her dog for a walk in the evening. Picture three: Smaller pictures all inside the same frame of Sarah taking her dog for a walk in the morning with “Monday”, “Tuesday”, Wednesday” etc. written in the corner.

The statement that goes with this set of pictures is “Sarah takes her dog for a walk in the morning”.
The answer is, of course, Picture 3. Picture 2 is easy to eliminate (this is a test of vocab, i.e. the difference between “morning” and “evening”). But the only way to know the difference between Picture #1 and Picture #3 is to understand the grammar. (Or guess, of course).

It’s still not a super difficult question, but it’s actually testing what you want to test. Note: I avoided using the word “always” in the statement (as in “Sarah always takes her dog…”).


Okay, so there are a few tips to get you started in designing you own tests. :smile:

Please feel free to share your own experiences with test design by leaving a comment below. What do you find most challenging? Any tips?

Best regards,
Leslie

Vote this up or down. Select: Vote This DownVote This Up
Overall rating: 2 | Total votes: 4
Loading ... Loading ...


--- Advertisement ---

These ads are served by third-party sites and are hopefully, tailored to the content of this page.
If you see anything you think is inappropriate, please contact me and, if possible, I'll block it.


Feel Free to Suggest a Topic

Would you like me to cover something in particular?
Just get in touch via my Contact Form.

7 Responses to “Three Test Design Tips for EFL & ESL Teachers”

  1. ESL Daily ESL Daily says:

    Great post! In my experience as an EFL teacher in both Korea and Thailand, most schools do not require students to take tests (other than the standard tests given by the school). However, with this approach it is difficult to assess your student improvement. I personally would test my students, however the results would be private. When student evaluation would come around I would base some of my comments on the students test work. However, grading is a different story in Korea (especially in English class). You can not give an F; D is essentially an F so you can’t give it unless the student has never attended; C you should only give one or two in a class of 40; B is satisfactory; A is average.

    This was at a public school (almost the same at the university I taught at too, but a few more C’s and B was the average). Not all schools are the same, however this has been my experience while teaching for 7 years in the country. Test giving would be for my own advantage to see what the students have learned and what needs to be improved.

    I hope to see more posts in the future!

    Thumb up 1 Thumb down 0

  2.  Leslie says:

    Thanks for the kind words, Jim. :smile:

    Good tip re: testing whether you’re required to or not and then using that as part of your empirical end-of-term grading.

    I wrote this article because lots of—in fact, I’d say the vast majority of—teachers are, of course, required to test their students both mid-term and end-of-term, but do not get tests that are centrally-designed at prefectural (or even local, or even college) level.

    What this means is that they have to test, but are left to their own devices as to how they design those tests. And of course, most teachers don’t have any kind of real training in Test Design. I do, and I was talking with an ex-colleague the other day who suggested I write a series of articles about testing. So I knocked this one up the other night. Glad you liked it.

    Your point about the issuing of grades is a crying shame, innit? What an absolute joke. Naturally, this is not the first such story about the pathetic assessment restrictions placed on teachers. There was an incident a couple of years ago at an Australian university where a lecturer’s pay was withheld (!) at the end of the academic year because she had failed some students. Yeah, really.

    Which implies (rather than openly states) the message that unless you pass those cashcow foreign students that prop up the institution by paying exorbitant fees—and thereby, effectively, paying your salary and keeping you in a job, Missy!—there will be unpleasant consequences.

    Anyway, thanks again for the big-up and your contribution. Great site you‘ve got there, too, by the way. I must make a point of visiting more often. And for readers of this site:

    I recommend you check out Jim’s blog (blog.esldaily.org)—and his main site. Great info there!

    Cheers,
    Leslie

    Thumb up 0 Thumb down 0

  3. ESL Daily ESL Daily says:

    Thanks Leslie and glad you like my blog

    A story to go along with the one teacher you talked about. I was teaching at a lower level university and I had failed one student. I really had no choice but to fail him seeing that I had NEVER seen him. He did not show up for a single class, no homework (which I did give), and he did not show up for any tests (which I did give). So when it came time to grade, he got a big F.

    A day later, the student, his mom and the dean showed up at my office. The student was balling his eyes out, his mom was ready to rip my head off and the dean looked dizzy. The dean goes up to me and says… “Jim, what do you think about giving him a B+?” Oh my… I think that is when I fell off my chair laughing. “How about a B-” he asked then. I said not a fat chance. However, if the student is willing right now to sit down and write the test (which was relatively easy), I will put the grade he gets on the test on his report card. The mom went red with shame, the 22 year old kid cried even harder, and the dean did not say a word. Needless to say they all left my office and it was quite and peaceful again. My conscience was clean… I left a few weeks later for a new job in Thailand.

    Killer story, dude! LOL!
    Leslie

    Thumb up 1 Thumb down 0

  4. Youcef ELMECHERFI Youcef ELMECHERFI says:

    Dear Leslie,

    First of all, I’m sorry to not reply your valuable article on time and this is for two reasons: the first one is that I was so busy traveling, visiting teachers and writing reports (I hate writing reports). The second one is that there was an internet breakdown all over the country because of the bad weather.

    I read your three tips for designing a listening test earnestly and I should limit my self to two of them.

    *Make sure the task tests actually what you want it to.

    This is a very important criterion in designing tests since testing should be objective and valid .Teachers are supposed to test what their pupils have learnt.So,if you test your pupils about something they haven’t seen, the test will be neither objective nor valid.

    The second point you raised is the objective of my test; am I going to test grammar or vocabulary. If the objective is listening comprehension I should disregard grammar and vocabulary mistake-though grammar and vocabulary are important for understanding a script-.since my objective is to test their capacity of understanding a script; they can be asked to circle the words they hear or just to tick true or false, or match pictures to statements.

    *Is the answer too obvious?

    A test should not be too easy or too difficult in order to be fair.A test should be fair enough because fairness helps teachers to get a clear idea about the real level of pupils and allow learners/teachers to see if there something wrong with their techings/learning in order to bring solutions through remedial works/teaching.

    Best wishes,
    Youcef

    Thumb up 1 Thumb down 0

    •  Leslie says:

      Thank you for your kind words — as always, Youcef! (Youcef has emailed me several times. He’s a Superintendent of Teacher-Training and is responsible for a lot of teachers in his area of the world. If you’re on my mailing list, don’t be shy. Just hit “Reply” and I will get back to you when I get the chance! :grin: )

      And feel free to comment whenever you like, of course! In your own sweet time, in your own sweet time. But do comment! I love to hear what readers have to say. These posts are not supposed to be gospel; they’re supposed to open up discussion and generate further ideas for investigation.

      Youcef, I’m glad you liked the post and you make a good point at the end in that a test is another instrument in a loop-input syllabus. That is, you should be considering the points and/or areas that students don’t perform well on in your tests and use those as areas to incorporate into subsequent lessons by way of review and extension (and then test it again to see if they’ve made any improvement). Good point! Thank you!

      Stay well,
      Leslie

      P.S. How’s that torn muscle? All healed by now, I hope. Back on the football pitch now that spring is here in your hemisphere?

      Thumb up 0 Thumb down 0

      • Youcef ELMECHERFI Youcef ELMECHERFI says:

        Dear Leslie.

        Happy to read your nice and valuable comment.Last week,I was very busy.I visited some teachers and I gave a seminar about How to teach reading to middle school in teachers.I was satisfied because my teachers got a new things/ides from that seminar.Most them had a false conception of this skill.Reading comprehension was assimilated to reading aloud.

        Best
        YOUCEF

        P.S Thank you for your care.I’m well and I return to stadiums next week.The spring holidays(15 days) starts today March19th.

        Thumb up 0 Thumb down 0

        •  Leslie says:

          Hey there, Youcef. Glad to hear you’re on the mend! (Which means “getting better” in case you’re not familiar with that expression).

          And nice one regarding the seminar! Remarkable isn’t it that so many teachers think ‘reading aloud’ = ‘reading skills’? I’m sure they were very happy to receive your input. I’m sure their students will be even happier! :wink:

          Enjoy your holidays.

          Thumb up 0 Thumb down 0

Leave a Reply