Feature

School Cert Non-Science

The 1990 School Certificate Science Examination tested something, but not science.

By Dr J E Packer

One might expect the School Certificate Science examination -- the first external science paper that thousands of New Zealand pupils sit -- to be a model of clarity, and educationally and scientifically sound. Instead, the 1990 paper was a national disgrace.

Delays in the paper's production allowed no time for proper proof-reading, resulting in a number of mistakes. These required correction in the examination rooms, unsettling the candidates. However, from educational and scientific viewpoints, the paper had far more serious deficiencies involving format, scientific method and exactness.

The paper had 54 multiple-choice questions with four answers to choose from, worth 54 marks. These required 14 pages of reading. The remainder of the paper consisted of 9 questions over 24 pages. Multiple choice questions are of dubious value in an examination used for assessment. They can be an excellent teaching vehicle if followed up by a discussion of why answers are correct or incorrect. They can act as a probe of student misunderstanding. However, they are confusing for students not strong in English and there is apparently evidence that boys do better than girls with this format. Further, a student having no scientific knowledge could, with average luck, get 13% from guessing alone.

With years of experience in this format, almost all questions should have been pretested and shown to be satisfactory, unambiguous, and with only one reasonable answer. Yet a meeting of a science teachers' association saw heated debate over the correct answers to some questions and about the quality of the questions themselves.

Some questions had no correct answers, some had more than one correct answer. Others were ambiguous or misleading. One used an incorrect symbol for volt, another asked for the result of an experiment designed to find the result -- poor science.

The longer questions contained "stimulus material", such as diagrams, statements, drawings and photographs. There was even more to criticise and debate in this section. Questions on the ripening of tomatoes -- "Why does the percentage of starch decrease?" and "Why does the rate of CO₂ usage decrease? -- should have been worded much more specifically.

Question 2, on blowflies, showed in diagrammatic form the results of some experiments. Students were asked to state the aim of one experiment and conclusions that could be drawn from others. Surely one designs experiments to test or measure something. To ask for the aim from the result, or what you can conclude without knowing what the experiment was designed to study, is science back to front. In one experiment, the flight time of flies was plotted for fed and for hungry flies, and at different temperatures. What was meant by "flight time" was not defined. It would lead to opposing conclusions depending on whether it meant the time to fly a certain distance or the time spent flying regardless of distance. The whole question was appalling science.

In a question on waves, there was a graph with no title -- something I hope no teacher would permit. In a diagram using drops of oil left by a car, there was no indication of in which direction the car was travelling -- again providing the chance for opposing conclusions to be equally right.

In the section on copper, an example listed W as the symbol for sulphur, although the chemical symbol S was used in the box above. Subquestion 7 asked students to write a "symbol equation" for a reaction, without it being clear that it referred to a sentence at the beginning of subquestion 6. And why not use "chemical equation", instead of a phrase a practising chemist would not recognise?

The percentages on the pie chart in a question on atmosphere added up to 100.9%. I also question the wisdom of defining ozone and chlorofluorocarbons as "greenhouse gases" and putting in a diagram of the ozone layer when the question was on the greenhouse effect. The greenhouse effect and ozone thinning are distinctly different phenomena.

Not many chemists would consider the equation CaCO₃ + H₂O + CO₂ --> 2Ca(HCO₃)₂ represents the dissolving of limestone by acid rain. Nor would they dream of collecting ammonia gas by displacement from an open test tube, or removing CO₂ from air by bubbling the air through concentrated sodium hydroxide in a beaker filled to the brim and with an inverted test tube full of the same solution. These are appalling practices and completely irresponsible from a safety point of view. And this criticism has been confined almost exclusively to chemistry!

It is asking a lot of one examiner to set a general science paper that is completely sound. With the aid of appropriate teaching and scientific moderators, it should be no problem. When many groups are striving to improve the image of science and to encourage more students to study it, the NZQA must ensure that it produces exam papers of high quality, both in presentation and content. The 1990 paper was no way to introduce the 1990s!

Two of the objectionable questions

11. Many people claim they smoke to relieve tension. This is because

A. the inhaled nicotine relaxes the muscles.

B. carbon monoxide dulls the brain.

C. carbon dioxide lowers the pulse rate.

D. less oxygen reduces stress.

This appears to have two answers.

21. A sound is heard when

A. sound waves reach the ear drum.

B. bones in the ear begin to vibrate.

C. fluid in the inner ear begins to vibrate.

D. a nerve message reaches the brain.

This question is philosophical rather than scientific.

Dr Packer is a lecturer in chemistry at Auckland University and has set examination papers in Bursary Chemistry.