IQ Test used in Public Education

Annotated Bibliography
Thesis Statement

Testing of intelligence Quotient has become a common practice in today’s schools.  Various tests that have been devised not only address and evaluate a child’s intelligence level but also help in diagnosing children with special educational requirements. It goes without saying that the magnitude of the decisions placed on these tests demand that they be accurate and precise in their construction and evaluation. However, various environmental, structural and other issues might cause variations in the results of these tests even among similar test samples. A major cause of error is administrative in nature implying the direct or indirect involvement of the examiner in the process. This thesis aims to undertake an exhaustive study of the subject and suggest recommendations if any.  The various intelligence test batteries assessed in this context include the Kauffman Assessment Battery for Children, The Wechsler Preschool and Primary Scale of Intelligence, Corge- Thorndite assessment system, PPVT etc.

The most significant fact that emerged during the construction of this bibliography is that numerous studies have already been conducted on the subject. The literature reviewed covers a wide variety of authors, cultural contexts and extend over a substantial time period. Most examiners, learning an intelligent assessment battery represent a crash course in multitasking. Each response must be recorded and evaluated in real time with respect to both administration and scoring procedures while simultaneously maintain rapport with the examinee and attending a multitude of behavioral and affective processes. The resources covered have been classified 1) as being immediately relevant to the topic, 2) providing conceptual insight into the topic and 3) of indirect or secondary implication.

Primary Sources

1. Scott A. Loe, Renee M. Kadlubek & William J. Marks (2007), Administration and Scoring Errors on the WISC-IV Among Graduate Student Examiners, Journal of Psycho educational Assessment, 25; 237

This highly insightful paper highlights the importance of examiner error in intelligence testing batteries by undertaking a detailed experiment where a total of 51 Wechsler Intelligence Scale for Children protocols, administered by graduate students in training, were examined to obtain data describing the frequency of examiner errors and the impact of errors on resultant test scores. It was found that errors were committed on as many as 98% of the protocols and averaged 25.8 errors per protocol.  The most common errors were failure to query verbal responses, assigning too many points to an answer, and failure to record an examinee’s response on the test protocol. In general, the authors were able to identify three categories of errors within the sample. Administration errors consisted of mistakes made during administration of the test. Computation errors were defined as mistakes that resulted in erroneous subtest scaled scores, factor index scores, or FSIQ scores while Recording errors consisted of failures to record responses or completion times on the test protocol when indicated by the test manual. In addition to providing valuable insights into the topic, the paper also defines accurate benchmarks on which future work can be based.

2. Douglas Fuchs and Lynn S. Fuchs (1986), Test Procedure Bias: A Meta-Analysis of Examiner Familiarity Effects, Review of Educational Research; 56; 243

This article presents a meta-analysis of the effects of examiner familiarity on children’s test performance. The data for the meta-analysis came from 22 controlled studies involving 1,489 subjects. In the typical study, the effect of examiner familiarity raised test performance by .28 standard deviations. Differential performance favoring the familiar examiner condition was greater when subjects were of low socioeconomic status (SES), were tested on comparatively difficult tests, and knew the examiner for a relatively long duration. It was found that examinees scored higher when tested by familiar examiners rather than unfamiliar ones. Also, the magnitude of this difference is both statistically and practically significant. In addition to validating the thesis statement, the work suggests numerous avenues for further work which may incorporate social levels of the examiner and the examinee as well as their cultural backgrounds.

3. Joseph J. Ryan and Summer D. Schnakenberg-Ott, (2003), Scoring Reliability on the Wechsler Adult Intelligence Scale-Third Edition, Assessment; 10; 151

Though focused more on adults, this study demonstrates beyond doubt that psychological examiners of all levels of skill and training make mistakes while assessing candidates. Many of the conclusions of this study are equally applicable to students in the age group of our concern. Common mistakes were found to be improper conduct of subtests and failure to properly credit individual response on the information, comprehension and vocabulary subtests. Some examiners were found to discontinue digit prior to failure on the trials of an item while others continues it even though the termination criteria had been met. It is interesting that twenty five doctoral level psychologists with extensive testing experience were used for this test. The methodology employed in this particular study is appreciable and many aspects of it can be translated into the current thesis.

4. Joseph A. Buckhalt, (1987), Examiner Comments during Intelligence Testing: an Attributional Analysis, Journal of Psycho educational Assessment; 5; 31

This study focuses specifically on the consequences that the comments of the examiners can have on test results. Most previous studies that have considered the effects of examiner comments during the administration of intelligence tests were conducted within the framework of reinforcement theory. This study is a descriptive analysis of examiner comments from the perspective of attributional theory of achievement motivation. Transcripts of examiners’ administrations of intelligence tests to children referred because of learning difficulties showed that comments that fell into the attributional categories of Task Difficulty, Effort, Ability, Practice, and Fatigue were distributed unevenly across subtests. Comments can be attributional in nature relating to task difficulty, effort, practice, ability and fatigue; or can be words of encouragement. A great many individual differences in the number and type of attributional comments were observes. The relevance of this study in our context is that examiners tend to interact more with young children which can lead them influencing their choices through various attributional comments. This is certainly a matter that has to be thoroughly understood in order to gain competency as a test examiner.

5. Christopher J. Hopwood and David C. S. Richard (2005), Graduate Student WAIS-III Scoring Accuracy Is a Function of Full Scale IQ and Complexity of Examiner Tasks, Assessment; 12; 445

This Research on the Wechsler Adult Intelligence Scale–Revised and Wechsler Adult Intelligence Scale–Third Edition (WAIS-III) suggests that practicing clinical psychologists and graduate students make item-level scoring errors that affect IQ, index, and subtest scores. In this study, graduate student participants score a high and low IQ record form in one of two stimulus conditions: digitized film clips (N = 13) or partially completed record forms (N = 11). Results demonstrate that examiners are less accurate in the high IQ condition, and that recording examinee response from scoring video clips results in more scoring errors. This study is based on the Wechsler scale for adults but its fundamental attributes remain the same. The use of stimulus conditions in particular provides a very useful mechanism whereby the test environment can be manipulated to implement variations in parameters.

Secondary Source

6. William G. Graziano, Philip E. Varca ; Jodie C. Levy, (1982), Race of Examiner Effects and the Validity of Intelligence Tests, Review of Educational Research; 52; 469

This study is a literature review of over 29 previous studies that explores the question of the influence of the racial background of the examiner on the test results. More than the implication on race, this study provides numerous insights into the nature of the ‘examiner – examinee’ interactions. Of the 29 works reviewed, 12 reported statistically significant data that the race of the examiner does influence the examiner – examinee interaction, though the authors do accept that the quality of research in the 12 papers is not good. More importantly, the study draws a comparison of various intelligence testing batteries such as WAISC Subtests, PPVT, WAIS Subtests, Corg – Thorndite etc. The specifications of this interaction are crucial for the current study as they outline the mechanisms by which errors can occur.

Works Based on Psychometric Theory and Practice

7. Joseph E. Zins and David W Barnett, (1983), The Kaufman Assessment Battery for Children and School Achievement: a Validity Study, Journal of Psycho educational Assessment; 1; 235

This is a validating analytical study of the Kauffman Assessment Battery for Children (K-ABC). The work compares the K-ABC to the wide range achievement test with a non referred sample of 40 children. The results were found to be substantially correlated for reading subtests. A major advantage of this study is that the Intelligence and Achievement scales were standardized on the same population. The study offers some preliminary insights into the structure of the Kauffman Assessment Battery for Children and warrants further study. The K-ABC was designed to measure problem solving skills in a manner not directly related to prior academic achievement. The understanding of the Kauffman set of batteries and specifically its correlation to other sets of intelligence assessment tests are very significant for our study as it helps us in understanding why the K – ABC is widely preferred as an intelligence assessment model.

8. Jason C. Vladescu (2007), Test Review: Kaufman Test of Educational Achievement-Second Edition (KTEA-II), Journal of Psycho educational Assessment; 25; 92

The paper gives details of the Kauffman test with detailed emphasis on the standardized battery and the test kits. The study also addresses the fundamental question of the discrepancies between achievement and intellectual ability. It is observed often that a student who performs very well in the intelligence assessment battery might be a underperformer academically. The reserve is also true. Such facts have pointed a questioning finger at the standard test to measure intelligence. Also, the work shows that within the battery, various tests have low inter correlation while the overall validity of the battery is maintained. Letter ; Word Recognition, Reading Comprehension, Math Concepts ; Applications, Math Computation, Written Expression, Spelling, Listening comprehension, Oral Expression, Phonological Awareness, Word Recognition Fluency, Nonsense Word decoding, Decoding Fluency., Associational Fluency, Naming Facility etc are the various parts of the battery.

9. Gary L. Canivez, (1996), Validity and Diagnostic Efficiency of the Kaufman Brief Intelligence Test in Reevaluating Students with Learning Disability, Journal of Psycho educational Assessment; 14; 4

This study gives theoretical details of various intelligence scales. The Kauffman Brief Intelligence test is described as a brief, individually administered measure of the verbal and nonverbal intelligence of a wide range of children, adolescents and adults spanning a range of 4 to 90. The correlations between various types of tests are found out. The Pearson product moment correlation coefficients for various compositions and subsets are given.

10. Alan S. Kaufman, (1983), Some Questions and Answers about the Kaufman Assessment Battery for Children (K-Abc), Journal of Psycho educational Assessment; 1; 205

This is a magnificent window to the Kauffman assessment model written by Kauffman himself. Kauffman states that problem solving efficiency of children reaches a peak by age 12 and levels off through age 16 and above. Despite the nature of the teaching for the examiners being specified in K – ABC Scoring and Administration manual, Kauffman admits that there exists no uniformity in the way examiners conduct their tests. Over half the examiners will stick to the words in the manual while the other half will modify the words to communicate the nature of the task or instruction to the child. Therefore, there may be more unreliability due to the examiner variability built into the existing intelligence tests.

.

;