Posts Tagged ‘tests


A liddle mistake means a lot

To be a good diagnostician, you have to attend to what a person does wrong.  What one does correctly is largely uninformative. And as  I’ve said elsewhere , scores are even less informative.

Jill scores at the 35th percentile on a standardized spelling test. That’s “average”; that’s about all we can say. We inspect her actual spelling work and see that she spelled train, bridge and receive correctly. So what have we learned? Not much, except it’s nice she can spell bridge.

But we also notice that she spelled little as “liddle“.  Now here’s something to think about, something that gives a clue as to why Jill’s teachers complain that she doesn’t know her phonics.  And how wrong they are when they say this!

Why , then, did Jill spell little as liddle? First off, NOBODY pronounces little with a /t/ sound. You can try to, but you’ll sound like you have a speech impairment. Secondly, NOBODY pronounces TWO /t/ sounds. If you do, you’ll be identified as a stutterer. As a matter of fact, no one ever says any “double” consonants. Even when the same consonant appears at the end of one word and the beginning of the next, you only pronounce the first one. Try saying “great taste” by saying one of the t’s immediately followed by another.  Makes you sound like a non-native speaker, one who hasn’t really mastered English.

Why our spelling system has rules for “doubing consonants” seems quite pointless then, if we never really say them both. But there are reasons, ones we “know” rather unconsciously, ones that have help reveal both the “phonetic” and “nonphonetic” basis to that system.

We can learn aout this from Jill . We can advance the following theories about her and this rather schizoid spelling system.

1.  Jill is quite aware of the sounds of English; she clearly noted the /d/ sound in the word little.

2.  Jill knows that when a consonant sound follows a short vowel,  short “i” for example as in hit, the consonant letters representing that consonant sound are doubled (compare later and latter). So she knows the rule for consonant doubling is based on this “diacritic” function of some spelling patterns, i.e. that double letters indicate the vowel preceding those letters is “short”. She also recognizes the “phonetic” rule that “t” stands for the sound /t/….except sometimes??

3.  Being a keen discriminator of sounds and knowledgeable about spelling rules, Jill spells the word “middle” in one of her home assignments as “mittle”.  Here she is “hypercorrecting”. That is, she is applying a rule that should NOT be applied. She’s saying to herself, “Aha!  I hear a /d/ sound in middle just as I do in little, so I’ll spell middle with two “t” s just as I did with little.  I also will use two letters because the first vowel in middle is short.”

4.  Jill will learn to be careful with words like ” liter” and “feeder” because the / d/ sound follows a long vowel. (i.e. they both are pronounced /leeder/). She might then spell liter as lider, knowing that double consonants never follow a long vowel. But she may be confused for awhile until she memorizes those sorts of words where a long vowel comes before a /d/ or / t/ sound.

What  we can recommend for Jill is instruction that makes her conscious of the fact that a double letter has a diacritic function (marking the preceding vowel as “short) AND a phonetic function (i.e. that it also represents a sound). We will NOT recommend phonics. She already discriminates sounds just fine. And we’ll teach her how to memorize the visual features of words.

Finally, we’ll appreciate how much a people can know about something when we examine what they seem not to know.


I heard what you said; I just forgot

An Attention Disorder is manifested in numerous ways because, quite simply, attention is involved in nearly every activity, even the generation of thoughts.

One common feature of an Attention Deficit is rapid forgetting of what one has just read or heard.  Consider Liz, whom I described in my 1/22/2012 posting.  Essentially, she couldn’t reliably register and maintain incoming information for a period of time sufficient to interpret or manipulate it in some way.  This process of data-maintenance is a function of Working Memory.  But Working Memory must be extremely reliable if one is to respond correctly to whatever the incoming information requires. And its processing powers are formidable.

If someone says the word “grand”, for instance, you will not be able to understand it until you register all the sounds of that word in Working Memory.  “Grand” consists of five sounds, /g/, /r/, /short a/, /n/, and /d/.  So each of these five sounds must be identified precisely.  If any one is inaccurately identified, you will not be able to identify the word.  Or you will misidentify the word (e.g. as “grand” or “grunt” or even “”can’t”).  In fact, you may not be able to tell whether what you heard is a word or not. When you listen to a young child speak, a person with a foreign accent, a drunken slur  or one with an unfamiliar dialect, you find yourself concentrating very hard. You narrow your eyes, watch their lips and facial expressions, focus intently on the sounds they’re producing and often enough ask them to repeat.  All you are trying to do is identify  sounds  so that you can assemble those sounds into a word you recognize.

Meanwhile, you are “holding” the sounds you do hear in Working Memory, sustaining them there until you finish your analysis or give up. I’ve been amazed with young mothers whose toddlers seem to be saying “Ahwanceal” and the mother instantly responds,“No it’s lunch time; cereal is only for breakfast.”  You might notice here that even determining how many words were said can be daunting because you can’t tell where one word ends and the next one begins (locally, the spoken unit “Jeweet?” is easily segmented into “Did you eat?” by a listener familiar with the local dialect patterns.).  Indeed, our ability to recognize slang, “accent” and dialect variations depends on precise identification of aberrant sounds as acceptable variants of standard sounds (the unit “Yall” is nonsense since there is no such word, yet few have difficulty assigning it the meaning of “You all”.)

Recognize that the time it takes to say “grand” amounts to a small fraction of a second. Recognize further that you effortlessly identify all its sounds in that sliver of time. The sentence, “I’ll be reading Dickens this afternoon” contains 25 sounds, yet all of them are easily identified in about 1.5 – 2.0 seconds. That’s over ten sounds a second.  And all this “Auditory Processing” takes place in Working Memory.

So, Working Memory  has to be extremely efficient and reliable.  But suppose it is NOT.  Suppose information (sounds) entering this system are very rapidly corrupted (deleted, mis-sequenced, garbled).  Or suppose that something heard exceeds the limits of Working Memory because it’s too long. Then, the information is lost or misinterpreted.  So, what was happening to Liz was that her Working Memory could not reliably and correctly “hold onto” language data (sounds, mainly).  So, before she could interpret what she heard, the data themselves were being corrupted…or erased.  And then, she suffered the consequences: misunderstanding in some cases, in others, instant and wholly INvoluntary forgetting.  In circumstances where she couldn’t request a repetition, she was defeated.  For her, forgetting what people said to her, forgetting what she’d just read were all too common experiences.  And when her job depended on efficient auditory processing, where the consequences of mis-hearing or mis-reading a direction, dosage amount, drug or patient name could be lethal, she despaired.  Because she couldn’t stop forgetting.


The Reality of Testing Human Beings

As I sit in my chair one day, and across several days, ask 10 different  10 year-olds  this question: “Who discovered America?”. It’s  one of the questions on the INFORMATION subtest of the Wechsler IQ scales for children. It’s a simple “right-wrong” item. Isn’t it? Easy to score?  It should be. But it’s not! Below is what you may well hear as  responses. Decide how you would score the response: right earns one point; wrong earns a zero.

Alison: “God, we learned that in second grade. Mrs. Jones . She was mean. Lessee, discovered America?? Well, the Indians. I mean people say Columbus. But it was really the Indians. (You ask: “So, what’s your answer?”). “Well…Indians. I’ll go with that. “

Greg: ” The Norsemen!…I’m pretty sure. Yeah, that’s it!”

Rene: “Christopher…uh…oh, what’s his name? Christopher…Vespucci? No, No. Oh, god! I KNOW this! I can’t get it! Aw, I don’t know!” 

Benny: “George Washington! Wait. What’d you say? I forgot. (You repeat the question). Oh! Columbus!”

Rosalie: “It begins with a “C”. I never really learned it. Leif Ericson? Jefferson? Something Columbia? I know his first name was Leonardo. I can’t remember. We were never taught it!”

Bill: “America? That’d be, uh , can I  take a guess? Columbus? Naaa. He was another explorer who came over and found, like, the Bahamas. So, I’ll pass…I don’t know!”

Raine: ” I can never remember this! We studied it a loooong time ago. The Pilgrims did, right before Columbus!”

Charlie: “Welp…that’s eeeezee! Vikings. I remember that because Mrs. Dinwiddie showed us pictures of their boats. It gets confusing sometimes, because my kindergarten teacher said the guy’s name was ‘Clumba’. But he was later.

Jim: “Columbus! Christopher Columbus! Is that it?  Just a sec…Yeah. Whatever”

I don’t know how easy it was for you to score these right or wrong. If you score it as wrong, the child gets a zero and is thereby less “intelligent” than the one who gets it right.

Human beings simple don’t do what they’re supposed to do, even when they try their hardest to do so. The sheer variation in these responses is sufficient to demonstrate that no matter how much you STANDARDIZE test items, you can’t standardize  responses to those items. And this “Columbus” question is just one of the many, many items on the 10 or so subtests of the IQ battery. As variations pile upon variations, it doesn’t take long to discover the uniqueness of human beings. But how can this uniqueness be defined and communicated to others? That is the question to be addressed, And I’ll be doing it in many postings to follow. Come along!


Average for whom: That’s the Question

So your son obtains scores in the “average” range on a group-administered, commercial achievement test (think Iowa’s or Stanford’s). You look at his charts accompanying your son’s score report and your eyes blur a bit, what with all the dotted lines, graphs, numbers and terms like percentile or standard score. But you do see that his score is visually in the middle of one of the graphs and that it has been described as “average” by whoever (or whatever) scored the test.

Your next question should be “So what”. Lamentably, the answer to that question is not at all easy to answer. A basic answer would be, “It depends”.

If your son is attending a public school, you can infer that in his class, he is NOT atypical, abnormal.

But suppose that public school is located in a more affluent school district. In this setting, his “average” score nationally speaking may be “below average” relative to his classmates. Students in more affluent  communities are much more likely to be brighter because their parents are more often well-educated, more invested in their children and more involved in their child’s education. So, in this more selective school, the “average” achievement level FOR THE SCHOOL is “above -average” for the national population. His 75th percentile (above-average) score may well be at the 50th (average) FOR HIS SCHOOL (just as a 6’2″ basketball player may be at the 50th percentile for his team BUT the 75th percentile compared to the national population).

It gets worse. If your son is attending an independent school, the “average” achievement level can be the 90th percentile (superior). So his “average” achievement score at the 50th percentile turns out to be “below-average” (<25th percentile) for the independent school. Indeed, his average skill levels put him at a real DISadvantage in an independent school setting. Many, many times I have advised parents to choose less competitive settings so the needless stresses of unrelenting achievement pressures and the mediocre grades that even this child’s BEST efforts produce do not poison  his sense of competence.

Interestingly, since the “average” student in an independent school  population is a “superior” student relative to the national population, the independent school student – after spending his elementary-school years there – comes to think of himself as “just average”.  I’ve heard many a very bright (90th percentile ) independent school student describe himself that way. He hugely underestimates his competence and potentials.

And so we see again, especially in view of what I’ve said in previous postings, the word “average” becoming a “weasel” word, a nonsense term that is best ignored in favor of more meaningful terminolgy. I will address the issue of meaningfulness in testing in future posts.


When the word “average” hurts badly

Jeremy at 9 years of age was altogether immersed in the tasks of assembling small red and white  blocks to duplicate  designs printed in a stimulus manual. He spun the blocks deftly in his fingers, sometimes using both hands to bring blocks together in a near-perfect integration of small motor movements.

Jeremy’s speed and the accuracy of his final products earned him an “exceptional” score (99th percentile) on the IQ subtest called “Block Design”. His speed itself won him “bonus” points awarded for “speed of assembly”. His parents and teachers were told of his score and how its exceptionality disposed him to much achievement and success in domains requiring such ability: mathematics, the sciences, engineering, Later, he was invited to take advanced coursework in science/math. And he prospered, delighting in the exercise of his “gift”.  His parents exposed him to stimuli and learning opportunities that refined and elaborated on his special competence.

Evan, another 9  year-old, was equally enthralled by these block assembly challenges. And he devoted himself to each one. Unfortunately, he kept dropping the blocks just before he placed them on the desk surface where he was working. Frequently, he’d turn a block, by accident, a bit too much, these causing a misalignment of block edges that required correction. At these times, frequent enough to exasperate many a child, Evan seemed undeterred…as if he were quite use to the clumsiness of his fungers. Indeed, on more than one occasion he would apply just a tad too much pressure as he moved blocks together, this “collision” misaligning already-correctly aligned edges which then required repair. But Evan loved the tasks, often exclaiming “cool!” or “that’s neat”  as a new design was presented to him. He got them all right. He produced perfectly accurate designs.

BUT, Evan earned NO bonus points for speed even though he finished them all well before the time limits set for each design assembly. As a result, he earned an “average” score…not because he was less perceptive, less competent than Jeremy. And certainly NOT because he was at all perplexed by the designs. For he knew EXACTLY what to do and how to do it, regardless of difficulty level. He was as expert as Jeremy. But his fingers wouldn’t move fast or deftly; they did not do what he wanted them to do, what he knew they had to do.  Needless to say, his “average” score won him no special consideration, no offers of advanced work in mathematics and science, no recognition of his expertise AT ALL. His teachers and parents were told by the school psychologist that he had “average” non-verbal reasoning…and that was that. Later, because he loved “non-verbal reasoning” tasks so much, he became a highly regarded and exceptionally competent architect. Colleagues marveled at the precision and elegance of his work, especially at how he “took his time” to get everything right in his drawings and models.

Ironic how the “real” world can be so much more competence-revealing and competence-affirming than the educational world that assigned him the label “average” and they dismissed him. Worrisome , however, is the fact that, had Evan been less determined, less inclined to believe in himself, he would never have discovered the delights of his own gift. He would have given up much earlier on.


An “average” score can mask essential facts

There’s a short Intelligence subtest called “Digit Span”.  A string of digits is dictated to examinees and they must repeat the string in the same order as the original string. After an upper level is reached (usually two strings in a row), digit strings are repeated backward by examinees. The resulting score purportedly measures something like “attention span”.  And if this score falls bewtween the 25th and 75th percentiles, it is called “average”.

Joan, for example, got a Digit Span score at the 63rd percentile. She was average…But not so fast. Consider HOW she handled the task.

First, while listening to the “digits forward” series, she stared directly at the examiner, “tracked” the numbers by touching her fingers for each digit said and responded only after she had subvocally repeated the string to herself several times. She accurately recalled enough strings to score in the superior range (the 90th percentile).

BUT, when she had to repeat digits in reverse, her performance dropped off dramatically, so much so that her “digits reverse” score fell to the deficit range (10th percentile). Moreover, she labored much more intensively in the reverse conditions, despite lots of subvocalizing, rarely able to keep the digits in order, even the few she could recall.

Even taking into account that” digits reverse” is more difficult, Joan’s very inconsistent behavior strongly suggests that some underlying “information-processing” system is erratic and unreliable.

BUT, since her final “Digit Span” subtest score reflected the combined forward and reverse scores, she turned out to have an “average” score.  As such, any possibility of an “attention” deficit was effectively eliminated. She was described as “of  average attention”,  the 50th percentile

In fact, Joan’s teachers and parents had reported quite a few attention-related problems. But her “average” Digit Span score eliminated Attention-Disorder as a possible cause.  Indeed, the person testing Joan  concluded that whatever academic problems she was having were from “emotional” conflicts. And she was sent off to therapy.

Yes, “average” can conceal real problems and deficit. The term can also conceal “gifts” as I’ll explain in my next report.