This article was originally published in ENGLISH TEACHING professional, Issue 20, July 2001. © Copyright Anthony Bynom.

TESTING: BASIC CONCEPTS: BASIC TERMINOLOGY

by Anthony Bynom, Ph.D., December 2001

Most teachers are involved in testing in some form, either invigilating, marking or actually writing tests. This article is aimed at teachers who may be either assessing test material or writing their own tests. The basic question to begin with is why do we test? 

I hope there are not many of you who will follow the example of a former colleague. This person would use tests as a punishment. If his class became too boisterous he would announce a test. Then he would retire to his desk and set the most difficult grammar test he could find in order, as he put it, ‘to teach the blighters a lesson.’ In this instance testing was used as a means of classroom management. The more conventional answer to the question of why we test is to get information. The type of information required will dictate the type of test needed. Types of information needed could come under the following headings.

·      SELECTION

     This is where testees are being selected for some future course or type of 

     employment. You are trying to find out if the people concerned have the right

     attributes to benefit from the course.

·        PLACEMENT

Is when you want to place testees at a correct level for their abilities.

·        RANK ORDERING OF TESTEES

This familiar process places testees in order, who is first second etc.

·        APTITUDE

This attempts to Predict likely future performance.

·       DIAGNOSIS OF PROBLEM AREAS

An attempt to find out why things are happening. 

·        VALIDATION OF CURRICULA, PEDAGOGICAL PRACTICES AND MATERIALS

Is the curriculum working? Do your classroom practices succeed? Are the materials you are using appropriate?

·       FEEDBACK

     This is used to amend procedures, if necessary. 

·        EXPERIMENTATION

When you want to try something new or different.

TYPES OF TESTING

Testing may initially be divided into two types.

1.      Norm Referenced Tests

Norm Referenced tests answer such questions as, how student ‘A’ compares with student ‘B’. Attainment or Achievement tests should be specifically designed to answer this question.

2.      Criterion Referenced Tests

Criterion Referenced tests answer such questions as, How much has student ‘Y’ learnt? Or, How much does student ‘X’ know? Proficiency tests should be designed to answer such questions.

In all the other test areas you should always bear in mind the purpose of your test. For example;

Aptitude tests should be designed to provide information to assist prediction of future learning success.

Diagnostic tests should obviously provide information on areas of difficulty.

Performance tests should be designed to provide information for the evaluation of a specific skill or task.

 RELIABILITY AND VALIDITY

We now move on to the two key issues for any test, reliability and validity.

These two concepts and their relationship to testing form the most fundamental issue in current thinking on testing. Although they are absolutely basic they very often appear to be incompatible, in that some tests that are known as reliable are seen as falling short in validity. While criticism of test validity is based on the question of how reliable they are In fact it might be truly said that the whole art and science of testing lies in attempting to find ways of harmonising these two important qualities. To help with this let’s look at their meaning.

Reliability.  This means the consistency of the test’s judgement and results. It is about producing precise and repeatable measurements on a clear scale of measurement units. Such tests give consistent results across a wide range of situations. This is achieved by carefully piloted trials of the test. Sometimes, several versions of the test may be used on a controlled population of testees. The outcomes of these trials are carefully analysed in order to establish consistency. When a consistent set of figures is achieved the test may be deemed reliable.

Validity. This means the truth of the test in relation to what it is supposed to evaluate.  It concerns the relevance and usefulness of what you are is measuring. The difficulty in setting such tests lies in the problem how sure you can be about what is actually being measured. Is it consistent with the worthwhile quality you think you’re measuring? 

To help with this you should consider the following: 

·        Content validity.  Have satisfactory samples of language and language skills been selected for testing?

·        Construct validity. Is the test based on the best available theory of language and language use?

·        Predictive validity. Does the test successfully predict future outcomes?

·        Concurrent validity. Does the test correlate with other existing measures? Usually a similar test.

There are other ways one can look at the subject of validity but the above are the main ones and give you the basic idea.

DISCREET POINT AND INTEGRATIVE TESTING

You may see or hear these words when being asked to assess or even write a test so let’s see what they mean.

Discrete Point tests are based on an analytical view of language. This is where language is divided up so that components of it may be tested. Discrete point tests aim to achieve a high reliability factor by testing a large number of discrete items. From these separated parts, you can form an opinion is which is then applied to language as an entity.  You may recognise some of the following Discrete Point tests:

1.      Phoneme recognition.

2.      Yes/No, True/ False answers.

3.      Spelling.

4.      Word completion.

5.      Grammar items.

6.      Most multiple choice tests.

Such tests have a down side in that they take language out of context and usually bear no relationship to the concept or use of whole language.

Integrative tests

 In order to overcome the above defect, you should consider Integrative tests. Such tests usually require the testees to demonstrate simultaneous control over several aspects of language, just as they would in real language use situations. Examples of Integrative tests that you may be familiar with include:

1.      Cloze tests

2.      Dictation

3.      Translation

4.      Essays and other coherent writing tasks

5.      Oral interviews and conversation

6.      Reading, or other extended samples of real text

OTHER ISSUES

Should you aim for direct or indirect testing? To help with this decision you may find the following helpful:

Indirect testing makes no attempt to measure the way language is used in real life, but proceeds by means of analogy. Some examples that you may have used are:

·        Most, if not all, of the discrete point tests mentioned above.

·        Cloze tests

·        Dictation (unless on a specific office skills course)

Indirect tests have the big advantage of being very ‘test-like’.  They are popular with some teachers and most administrators because can be easily administered and scored, they also produce measurable results and have a high degree of reliability.

Direct tests, on the other hand, try to introduce authentic tasks, which model the student’s real life future use of language. Such tests include:

·        Role-playing.

·        Information gap tasks.

·        Reading authentic texts, listening to authentic texts.

·        Writing letters, reports, form filling and note taking.

·        Summarising.

Direct tests are task oriented rather than test oriented, they require the ability to use language in real situations, and they therefore should have a good formative effect on your future teaching methods and help you with curricula writing. However, they do call for skill and judgment on the part of the teacher.

COMMUNICATIVE LANGUAGE TESTING

Since the late 1970s and early 1980s the Communicative approach to language teaching has gained dominance. What is actually meant by ‘Communicative ability’ is still a matter of academic interest and research. Broadly speaking communicative ability should encompass the following skills:

·        Grammatical competence. How grammar rules are actually applied in written and oral real life language situations.

·        Sociolinguistic competence. Knowing the rules of language use, ‘Turn taking’ during conversation discourse, etc. or using appropriate language for a given situation.

·        Strategic competence. Being able to use appropriate verbal and non-verbal communication strategies.

Communicative tests are concerned not only with these different aspects of knowledge but on the testees’ ability to demonstrate them in actual situations. So, how should you go about setting a Communicative test?

Firstly, you should attempt to replicate real life situations. Within these situations communicative ability can be tested as representatively as possible. There is a strong emphasis on the purpose of the test.  The importance of context is recognised. There should be both authenticity of task and genuiness of texts.  Tasks ought to be as direct as possible.  When engaged in oral assessment you should attempt to reflect the interactive nature of normal speech and also assess pragmatic skills being used.

Communicative tests are both direct and integrative.  They attempt to focus on the expression and understanding of the functional use of language rather than on the more limited mastery of language form found in discreet point tests. 

The theoretical status of communicative testing is still subject to criticism in some quarters, yet as language teachers see the positive benefits accruing from such testing, they are becoming more and more acceptable. They will not only help you to develop communicative classroom competence but also to bridge the gap between teaching, testing and real life.  They are useful tools in the areas of curriculum development and in the assessment of future needs, as they aim to reflect real life situations. For participating teachers and students this can only be beneficial. 

Please feel free to email Anthony with any comments or queries about this paper, or leave a message in the guestbook at Karen's Linguistics Issues.

©Anthony Bynom 2001. All rights reserved. Webmaster