What Monster are we Creating?

More "great" news in our (Tacoma) school district...they're using iPads for little kids to learn letters. 

One teacher says, the iPads "enhance instruction" and the reporter notes that one student sat, "beguiled" by it. 

One effect that our short-term vision of 'enhancing' things like letter learning now is that we stimulate an expectation for "wow" later.  For some thought on this, read all the articles on electronics, brain development and attention (start with the NY Times from two days ago).

I see it routinely in 8th grade, where 76 of my students acknowledged owning a total of 247 personal electronic gadgets, and using 181 of them while doing school work, though not using them FOR school work.

On the other end of the (socio-economic) spectrum, it's not hard to imagine a student getting WOW letter learning on the school iPad then going home to much less 'engaging' reading material so not spending more time in reading at home.  

There is an implicit assumption underlying all this enthusiasm for technology that the reading mechanics learned on the WOW device will seamlessly transfer over to reading in any and all media.  My experience in 8th grade, however, is that the technology transfers, not the reading.  So youngsters are doing more with and on their gadgets, where socializing and entertainment grip them much more than reading (even on the gadgets) or schoolwork.

Yes, these electronics may 'help' now, but they also might create a path dependence that leads to educational difficulty later.  Unfortunately, school districts find themselves unable but to start down this path.  Faith in technology, plus social demand that schools--not parents, families or communities--do everything possible to improve outcomes, times Bill Gates insisting that computers are more effective than people equals a cultural climate in which schools MUST get more computers or they'll be deemed derelict. 

What gets lost in the discussion is that the parents committed to and involved in their children's education is much more potent than any Apple product.

Read more here: http://www.thenewstribune.com/2012/11/03/2354529/ipads-are-their-favorites.html#storylink=cpy

Groucho may have been right

Last week, I sat on a panel of union members interviewing one of the candidates for state legislature in my area.  I am a full member of the union, though with reservations.  I found it hard to ask questions so laced with self-interest, and I ended up unsure whether I could support the candidate BECAUSE he was so stridently union.  "Over my dead body," he assured us regarding any legislation that would undermine public employee unions.

It seems to me that the urgency of protecting the union is treating the symptom, rather than the disease.  The social climate regarding schools is poisonous--distrust fouls the air.  Those who distrust teachers want accountability.  Those same are distrusted by teachers, who respond by seeking mechanisms of protection.

If the reformers spent some more time in class rooms and worked along with teachers, rather than mandating at them, we might have a chance at real trust-building that minimized the need for convoluted and ineffective instruments of accountability and protection.

Short of that, I may need to invoke a stylization of Groucho and get out of this club that wants me as a member.

How does this affect the standardized test score?

A colleague related a story of some people she was with who knowingly parked their cars in a business' parking lot to attend an event not associated with the business, under signs warning that such conduct would result in the vehicle's getting towed.  When they returned, one of the cars was being towed.  They were able to get the car back on the spot, but still had to pay a 3-figure fee for its having begun the towing journey.

They were furious enough that they complained to the business manager--offering some sort of lame explanation about it having been a community event, after all--and were reimbursed the towing fee and given a sizable gift card to the business.

How in the world, you may ask, does this have anything to do with education?

Two things.  First, standardized test scores can go hang if our education system doesn't do something about this kind of corrosive self-absorption.  Second, and more importantly, how can "education" (the system, individual teachers, etc.) do anything about such corrosive self-absorption when our society simply swims in such ridiculous justifications for insanely selfish behavior?

How in the world are a bunch of maligned and beleaguered teachers going to do anything about the teenage children of people who do such a thing as this?  And yes, this does affect standardized test scores.  If a self-absorbed school student doesn't see a need to apply him or herself to school work and test readiness, then s/he won't, and too often there's little available to compel greater attentiveness, especially in the face of such well-honed self-justification machinery.

Ask any teacher you know.  They can tell you any number of stories of such selfishness, responsibility-avoidance, and blame-deflection.

Another Update to the Test?

State Superintendent Randy Dorn is apparently imagining yet another change to Washington's test.  It seems the Legislature "funded a report on the merits of formative testing"--tests done at the beginning of the learning.  So he wants to do more smaller tests through the year.


It took a study to figure out this was worth doing.  If you want to evaluate teachers based on scores, the only study necessary would have been to ask teachers.  They are desperate to have a baseline by which their impact on a student can be more accurately and usefully measured.


My district used to do this.  A one-day computer-based test that correlated pretty well with WASL/MSP strand content and score outcomes.  I could project MSP results based on our internal test results, and I could identify what students needed to work on by their results of early tests.  We abandoned that for lack of money.


Another aspect of this--we keep changing the test, always presuming it's the 'same' (if we're going to do a lot evaluating of teachers under the current testing structure, it will be across test years) throughout all the changes.  Consistency of the test instrument is part of 'standardized' also.  Not sure how safe it is to assume consistency will be maintained, though.

Tests with High Stakes...but for whom?

Most of the discussion of high stakes testing of public school students tends to overlook some fundamental, but crucial, aspects of the whole notion of such tests, especially their connection to teacher evaluations.

The test outcomes have no impact on the students' academic lives, so the stakes are not high for them.  Rather, teachers feel the pressure of the high stakes, and increasingly so when student performance on those tests is going to be a factor in teacher evaluations, as it will undoubtedly be almost everywhere soon.

But any notion that we can rightly and wisely connect scores to evaluations suffers from several serious logical errors or flaws.

First, and most importantly, we are, to be logic of 'sciencey' about it, affirming the consequent--using test score changes as proof of teacher quality.  The whole issue becomes tautological, as the measure of the independent variable (Teacher Impact) is observed as the result on the dependent variable (Test Score). This an invalid form of argument.  To make the Teachers --> Test Score claims valid we must define measures of Teacher Impact prior to and separate from Test Scores, then hypothesize that Higher Teacher Impact scores will cause Higher Test Scores.  Sadly, I don't see that happening.  It's far too easy to simply define Teacher quality by Test Score results.

Second, connecting scores and evaluations implicitly looks upon the students as neutral actors in the whole scenario.  By that I mean we must assume students are merely objects being turned or maneuvered by teachers and that whatever teachers do well or badly transmits fairly directly to students and shows up in their test scores.  If student scores go up, that teacher did a good job.  If not, not.

Or think of it this way.  If something intervenes between the causal agent Teacher Impact and the outcome Student Performance which causes the measurement device Test Score to register something more or less than actual Teacher Impact on Student Performance, then Test Score is a less than fully accurate accounting of Teacher Impact.

We're assuming, in still other words, that Student Performance measured as Test Scores is actually an accurate (if logically invalid) measure of Teacher Impact.

Dubious assumption, as stuff intervenes, no doubt, between Teacher and Test Score.   The question is, how much stuff, and how do we tell what effect it has?

But even if we could figure all that out, we have a third concern, this time about motivation.  The students are the ones taking the tests, and their scores are a measure of accountability for the teacher, not for the student.  The study of economics teaching us nothing if not this:  You have to watch the incentives.  Pay attention to who has incentive to do what things.

As we are constructing the situation, the teacher has a lot of incentive to make sure students do well.  And I assume that's the hope.  Motivate teachers and they'll go motivate and teach students.  But the students don't have much incentive beyond some amorphously constructed internal drive to do well on the test.

To put it in a social sciencey kind of way, the agents (students) whose practical ability we are hoping to see (registered as test score outcomes) don't have appropriate incentives (or not as much as some other actors--teachers--who lack practical ability) to necessarily maximize performance.  Nor is a legal authority available that could compel students to seek maximal outcomes against their own preference to do so or not.

The so-called strategic triangle of compliance (thanks to Ron Mitchell, p. 14 for teaching me that one), which in this case is test score maximization, does not get rightly made in this situation.  The actors with incentives (Teachers) have some, but limited impact on the actors with the practical capacity (Students) to maximize scores, and the actor with 'legal' authority (Parents) to compel the exercise of greater student capacity has been dropped out of the scenario.

Unfortunately, I do not expect these concerns to derail a train with so much steam built up.  Bring on the Test Tricks.

More Data, Please

Imagine my vexation when one moment I read in The News Tribune that, according to a survey by the University of Washington, Washington schools have not done too well with their federal turnaround money then turn around and read that a Center for Education Policy report says that most schools across the country have made gains with their turnaround money.

On the one hand, WA State Superintendent Dorn says he’s going to wait for the “actual data” before judging in Washington.

On the other hand, Secretary of Education Duncan observes that the nation's lowest-performing schools appear to be showing preliminary promise, according to student-achievement data.  But it’s too early to draw hard and fast conclusions from only one year of data.

More data is the only answer.  I’m sure it’ll all be clear next year...when we get the school district data.

Replicating the findings raises concern about the standardized test

After teaching 95 students how to decode the standardized test questions and answers without reading the passage, I gave them an 8-question, no passages reading test.  Where they should have gotten 25 per cent (random guessing with four answer options), the group average was over 60 per cent.  Indeed, the average for the 95 students was 5.1 corrects, 2 ½ times the expected outcome from chance.  The standard deviation of the set was 1.38.  The t-test p value for these results 0.0001.  In other words, the probability that 95 testers would average 5.1 corrects when they should have averaged 2 (according to chance) is exceedingly low.

I repeated the test with 89 of the same 8^th graders, using 8 different question and answer sets.  This time the only instruction was to "think about the patterns we talked about last week."  The group average was 5.3 corrects with a standard deviation of 1.33.  These results also return a p value of 0.0001.  If possible, the student performance in the second test was even more “extremely statistically significant” than in the first test.

The replication of this test of students' ability to identify patterns in questions and answers calls into doubt the accuracy and usefulness of Washington state's standardized reading multiple choice questions.

Bad Tests, Teacher Evaluations and Incentives

I taught 95 students how to decode the standardized test questions and answers without reading the passage.  Then I gave them an 8-question, no passages reading test, and where they should have gotten 25 per cent (random guessing with four answer options), the group average was over 60 per cent.  The identifiability of patterns in questions and answers calls into doubt the accuracy and usefulness of Washington state's standardized reading multiple choice questions.




More than 40 states have joined the so-called Common Core State Standards and the associated testing consortium that allows the members to coordinate their standard-setting and assessment.  This is the closest we've yet gotten to a national education standard.  

One consequence of this development is that we can move even more determinedly forward to connect students' standardized test scores to teacher evaluations.  Along the way, though, many teeth are gnashing over the mechanics of just how to make this linkage.  For instance, a significant impetus last fall for the teacher strike in the district where I live (Tacoma, WA) derived from the contention over the way to join evaluations and scores, and the resolution of that strike created a(nother!) committee to figure it out.  Their recommendations are still forthcoming.

But the anxiety over implementation is only part of the story.  We would do well to take a serious look at the test side of the equation by itself.  Education reformers and the business community, among others, come out big for testing, with business cheering for increased accountability of teachers and schools by way of the politically elusive test-evaluation connection.

The debate about teacher evaluations and test scores proceeds along predictable lines.  Test opponents are tagged as unionists only interested in keeping cushy jobs.  Test supporters are thought woefully out of touch about how class rooms really function.  All the while, the test itself, that thing and process on which so much of the acrimony suspends, sits rather unassumingly by.  We talk little about the test or the test process, and, by implication, bear great faith in the device and its accuracy and reliability in assessing students' knowledge and capacities.

But just how much faith should we put in the test and in procedures that use the scores as evidence to determine anything beyond whether a student did well or poorly on that specific test?  Can we rely on the tests to actually and accurately measure knowledge and capability in a particular subject area?  

It turns out that for the reading test at least, the answers may be disquieting.  The reading MSP (Measurement of Student Progress, Washington's state standardized test) exhibits patterns which make it more an examination of 'test taking' than of reading.  Sampling a few test questions indicates that we can discern a set of predictable patterns in the question-making and the answer construction.  These patterns give savvy test takers an advantage and at the same time make the test a less than useful or accurate measure of a student's reading performance, or of how well a particular teacher is doing his or her work.

The following tutorial, is based on a 3-question OSPI (WA's state education agency) 'released item.' Released items are test material available at the OSPI web site, and consist of a passage and question set that was earlier beta-tested on real students, unbeknownst to them, as a non-scored section of a real MSP test, will prepare you to "take the MSP" by using the patterns identified here.  

Following the tutorial are two sets of four questions from other released items.  See if you can't make a pretty good guess about the answers, or at least narrow your choice down to the two best answers, or identify the easy way to answer the question.  (Correct answers follow at the end.)

Yes, to best evaluate the presence and identifiability of patterns, this test process will proceed without any reading passages.  Just the title, the questions and the answer options.

Most 8^th grade students (the group I teach) have up to 5 years of experience with Washington's standardized tests and when I explained the patterns described below, many realized they had a general, if somewhat unconscious, awareness that they knew or at least recognized them.   All patterns explained here have been identified by 6 years of working with OSPI released items, and listening to students observe--in ways only teenagers can--the similarity between released items and actual MSP items.  Teachers are forbidden—along with everyone besides students and the state bureaucrats—from looking at actual MSP tests.

Clearly, released items are not test items, but with a review process as tortuous as what each item must pass through, released items and actual test items are unlikely to be significantly different.  After all the bias and sensitivity screening prospective test items go through.)  If they are different, then the students' years of experience with real MSP questions shouldn't transfer to success on released items.

Just what are these patterns, then?  The best explanation comes from looking at the examples below.

1.   What is the main idea of "Excerpt from Iditarod Dream"?

1. Sled dog racing is a thrilling and dangerous sport.
2. Sled dog racing requires teamwork and training. 
3. Sled dog racing requires specialized equipment. 
4. Sled dog racing can be a family activity.

First, this is a main idea question, so we need to have a sentence that is 'worthy' of serving as a main idea.  It's hard to explain, but ask a nearby 8th grader, he or she will understand that some of these just don't 'feel' like MSP-type main idea answers.  They're not serious or important or high-quality enough, or at least they're not as serious as some other options.  

'Specialized equipment' isn't as important a point as either 'teamwork and training' or 'thrilling and dangerous.'  'Family activity' is almost non-sensical in that it violates expectation of what we might think or hear about dog sledding.  While there may indeed be a family out there that makes sledding one of their activities, this would be an oddity.  The MSP doesn't usually make main points out of oddities.

'Teamwork and training' or 'thrilling and dangerous' are the best options, then.  But the MSP often includes readings with a kind of moral element.  There are an unusual number of uplifting or inspiring stories.  Whether a little known figure gallant for service to others, or a determined soul who has surmounted obstacles to achieve something and/or (better yet) learned some important life lesson, MSP questions go through a vetting process that renders controversial or negative material unlikely to make the final cut.  

Thinking of it in this way, 'thrilling and dangerous' has just a hint of the selfish and irresponsible.  'Teamwork and training,' by contrast, is the kind of emphasis the MSP can and likes to support.  I'd probably go with that...and I'd turn out to be right.

According to "Excerpt from Iditarod Dream," why does Dusty decide to help the other racers build a fire?

1. He uses the fire light to see the trail markers
2. He thinks the fire will help him stay awake. 
3. He is following the rule of the wilderness. 
4. He needs to cook the dogs' frozen meat.

MSP can tend toward the 'unusual' option.  C jumped out immediately because it's of a different quality from the others, which are all specific and concrete things.  C, by contrast, is an interestingly oblique answer that hints of something 'higher' than the other three.  The combination of uniqueness and grandness makes C too hard to pass up, and doing so would yield a wrong answer--C is correct.

According to "Excerpt from Iditarod Dream," how would Dusty most likely react to entering another dog sled race?

1. He would be hopeful because he came so close to winning.
2. He would be nervous because he had trouble staying on the trail at night.
3. He would be excited because he knew how it felt to cross the finish line in the lead.
4. He would be anxious because he ran out of supplies and needed more for the next race.

On first blush, this ostensibly 'prediction' question seems unanswerable without reading the passage.  Indeed, how can we predict anything with such a dearth of knowledge of the situation.  Further, each question in this response contains a detail that we can only guess at, so we're left with a higher degree of uncertainty than in the previous questions.  But ultimately we are trying to get the correct test answer here, not predict something about Dusty, so things are not as hopeless as they seem.  

First, cover all the answers from the word 'because' onward.  You are left with a list of adjectives about how Dusty would feel.  The old advice to 'look for the stronger word,' and the current advice to think about uplift and inspiration could be of some help.  Granted, every test item won't work this way, but following these two 'rules,' C--excited--breaks out to an early lead in our race to decide.  Option A has the tinge of the overly competitive.  MSP probably tends to de-emphasize things like winning.  Just look how the test renders 'winning' in option C--'knew how it felt to cross the finish line in the lead.'  They seem to be at pains to avoid a word that sits uncomfortably in the social culture of collaborative education.  8th graders may not follow or care about the culture of education, but they do pick up on patterns, and the combination of that quirky way of saying 'win' and the most upbeat adjective--'excited'--make C a plausible option.  

Granted, this explanation is much more abstruse and convoluted, so do some more work by covering every answer from the adjective back to the beginning of the sentence and leave exposed what really are the first part of four conditional statements.   For instance, option A can rearrange to say "He came close to winning, so he will be hopeful."  

You'll note that not all the events can occur in the story.  How could Dusty come close to winning and cross the finish line in the lead? He can't, so either A or C is incorrect.  It's unlikely that both A and C are incorrect, as Dusty had to either win or not win, and the answer set would be strangely vexing if one of the causal elements (latter part of the statement) were true, but that answer were wrong. It would indeed be a more challenging test if readers had to actually make inferences about Dusty's feeling--by, say, dealing with several true statements.  But such are not as easily graded as the MSP needs to be.  

Using the 'deep' or 'serious' test, D is the least likely--it does not have the feel of high level of thinking. B is a contender, but its chances are reduced by the difficulty of both A and C then being incorrect.  I'm going with C, the odd wording for 'win' being too strong a pull to avoid.  

(At another time and place it would be worth considering how this unusual wording is really meant to distract some testers.  Some students will be vexed by the difference between 'win' and 'cross the finish line in the lead,' and so will not be sure if this is the right answer.  This vexation will help ensure that some students will get the answer wrong, thereby creating the necessary 'distribution' of answers and scores.  This opens a whole different problem--the effort to measure whether individual students are meeting a standard by misusing testing devices and procedures whose design actually distributes students across the outcome spectrum.)

Now, when you eventually do read the passage, all you really have to do is simply confirm which of the events described in the latter part of each sentence actually happened.  Did Dusty win?  If so, it's C.  Most of the time the option set will contain only one accurate description of an event which actually occurs in the story, making the corresponding answer option the obvious choice.   

The student taking the test does not really have to predict, s/he just has to look for which of the events described in the answer options really did happen.  Almost certainly only one occurred, but in the effort to make the test something more than matching (the story event to the correlated answer option) some slightly inaccurate permutation of one of the other events will appear as an answer.  In this case, the oddly inaccurate one is the 'going off the trail' option, and the correct answer is C.  (I confess, I've still not read the passage accompanying these questions, but my 7^th grade daughter confirmed these details.)

Interestingly, this question was categorized as 'comprehension,' which presumably ranks below 'analysis' on the intellectual spectrum.  The question is framed to look like a prediction question but really isn't.  The student's ability to comprehend which detail (from the latter half of the answer options) actually happened in the story is really what's getting tested.  They needn't predict anything.  This question was the hardest to answer without reading the passage, but many testers (including my daughter) were able to narrow it down to two answers and C was one of them. 

With the revelation of these patterns fresh in mind, I determined to figure out whether real test-taking students discerned the same patterns, or if I just unlocked my own odd, but ultimately individual, insight into the test.

I administered two different versions (one version is reproduced below) of 8-question, no reading passage tests to 95 8th grade students, and one 7th grader--my daughter.  I provided the title of the passage, followed by 4 questions on the passage, each question with 4 answer options.  Presumably, each student's average score would be in the area of 2 corrects (1 out of 4), as would the overall average of all students.  

Having identified and explained the test patterns to the students, I predicted that scores would be significantly higher than what random chance would expect.  Indeed, the average for the 95 students was 5.1 corrects, 2 ½ times the expected outcome from chance.  The standard deviation of the set was 1.38.  The t-test p value for these results 0.0001.  In other words, the probability that 95 testers would average 5.1 corrects when they should have averaged 2 (according to chance) is exceedingly low.

These findings raise a variety of questions about the MSP multiple choice questions, and none of the likely answers are good.  Fundamentally, is this as good a reading test as we hope and want it to be?  Or is it a less a reading test than a test on test-taking?  

If students can identify patterns of questions and their answers and get much better than expected scores just from knowing and seeing those patterns, then this particular standardized test is not really testing reading ability.  We could certainly claim that savvy (i.e., 'smart') students will more likely figure out the patterns and get the advantage on the test, and that such savvy is positively related to reading ability, but this adds another layer of uncertainty into the assessment process.  

If the test writers have corrected these patterns in the actual test items, the rest of us would never know, as gaining access to the test is not an easy process.  The screening process for test questions, with at least three phases of content, bias and sensitivity filtering, narrows the range of plausibly acceptable items, and increases the probability that the released (rejected) items are essentially similar to the actual test items.  When I described the patterns in the released items, my 8^th grade students certainly seemed familiar with them.  This indicates some compatibility between the rejected and the accepted test items.

Given all this—the substantially better than expected student scores, a test-writing process that probably generates a narrow range of question/answer design alternatives, the secrecy of the test production—we can only wonder at just how useful this test really is.  But I can say this, if my professional evaluation is going to be tied to such a test, I'm teaching every one of these tricks.  

Can you break the code of standardized test (at least WA's MSP) questions?

From the released items discussion below...

Explanations of how to decode the Questions and Answers without reading the passages.

1.   What is the main idea of “Excerpt from Iditarod Dream”?

A.   Sled dog racing is a thrilling and dangerous sport.

B.   Sled dog racing requires teamwork and training. 

C.   Sled dog racing requires specialized equipment. 

D.  Sled dog racing can be a family activity.

First, this is a main idea question, so we need to have a sentence that is ‘worthy’ of serving as a main idea.  It’s hard to explain, but ask a nearby 8th grader, he or she will understand that some of these just don’t ‘feel’ like MSP-type main idea answers.  They’re not serious or important or high-quality enough, or at least they’re not as serious as some other options. 

‘Specialized equipment’ isn’t as important a point as either ‘teamwork and training’ or ‘thrilling and dangerous.’  ‘Family activity’ is almost non-sensical in that it violates expectation of what we might think or hear about dog sledding.  While there may indeed be a family out there that makes sledding one of their activities, this would be an oddity.  The MSP doesn’t usually make main points out of oddities.

‘Teamwork and training’ or ‘thrilling and dangerous’ are the best options, then.  But the MSP often includes readings with a kind of moral element.  There are an unusual number of uplifting or inspiring stories.  Whether a little known figure gallant for service to others, or a determined soul who has surmounted obstacles to achieve something and/or (better yet) learned some important life lesson, MSP questions go through a vetting process that renders controversial or negative material unlikely to make the final cut. 

Thinking of it in this way, ‘thrilling and dangerous’ has just a hint of the selfish and irresponsible.  ‘Teamwork and training,’ by contrast, is the kind of emphasis the MSP can and likes to support.  I’d probably go with that...and I’d turn out to be right.

According to “Excerpt from Iditarod Dream,” why does Dusty decide to help the other racers build a fire?

A.   He uses the fire light to see the trail markers

B.   He thinks the fire will help him stay awake. 

C.   He is following the rule of the wilderness. 

D.  He needs to cook the dogs’ frozen meat.

MSP can tend toward the ‘unusual’ option.  C jumped out immediately because it’s of a different quality from the others, which are all specific and concrete things.  C, by contrast, is an interestingly oblique answer that hints of something ‘higher’ than the other three.  The combination of uniqueness and grandness makes C too hard to pass up, and doing so would yield a wrong answer--C is correct.

According to “Excerpt from Iditarod Dream,” how would Dusty most likely react to entering another dog sled race?

A.    He would be hopeful because he came so close to winning.

B.    He would be nervous because he had trouble staying on the trail at night.

C.    He would be excited because he knew how it felt to cross the finish line in the lead.

D.   He would be anxious because he ran out of supplies and needed more for the next race.

On first blush, this ostensibly ‘prediction’ question seems unanswerable without reading the passage.  Indeed, how can we predict anything with such a dearth of knowledge of the situation.  Further, each question in this response contains a detail that we can only guess at, so we’re left with a higher degree of uncertainty than in the previous questions.  But ultimately we are trying to get the correct test answer here, not predict something about Dusty, so things are not as hopeless as they seem. 

First, cover all the answers from the word ‘because’ onward.  You are left with a list of adjectives about how Dusty would feel.  The old advice to ‘look for the stronger word,’ and the current advice to think about uplift and inspiration could be of some help.  Granted, every test item won’t work this way, but following these two ‘rules,’ C--excited--breaks out to an early lead in our race to decide.  Option A has the tinge of the overly competitive.  MSP probably tends to de-emphasize things like winning.  Just look how the test renders ‘winning’ in option C--‘knew how it felt to cross the finish line in the lead.’  They seem to be at pains to avoid a word that sits uncomfortably in the social culture of collaborative education.  8th graders may not follow or care about the culture of education, but they do pick up on patterns, and the combination of that quirky way of saying ‘win’ and the most upbeat adjective--‘excited’--make C a plausible option. 

Granted, this explanation is much more abstruse and convoluted, so do some more work by covering every answer from the adjective back to the beginning of the sentence and leave exposed what really are the first part of four conditional statements.   For instance, option A can rearrange to say “He came close to winning, so he will be hopeful.” 

You’ll note that not all the events can occur in the story.  How could Dusty come close to winning and cross the finish line in the lead? He can’t, so either A or C is incorrect.  It’s unlikely that both A and C are incorrect, as Dusty had to either win or not win, and the answer set would be strangely vexing if one of the causal elements (latter part of the statement) were true, but that answer were wrong. It would indeed be a more challenging test if readers had to actually make inferences about Dusty’s feeling--by, say, dealing with several true statements.  But such are not as easily graded as the MSP needs to be. 

Using the ‘deep’ or ‘serious’ test, D is the least likely--it does not have the feel of high level of thinking. B is a contender, but its chances are reduced by the difficulty of both A and C then being incorrect.  I’m going with C, the odd wording for ‘win’ being too strong a pull to avoid. 

Now, when you eventually do read the passage, all you really have to do is simply confirm which of the events described in the latter part of each sentence actually happened.  Did Dusty win?  If so, it’s C.  Most of the time the option set will contain only one accurate description of an event which actually occurs in the story, making the corresponding answer option the obvious choice.  

The student taking the test does not really have to predict, s/he just has to look for which of the events described in the answer options really did happen.  Almost certainly only one occurred, but in the effort to make the test something more than matching (the story event to the correlated answer option) some slightly inaccurate permutation of one of the other events will appear as an answer.  In this case, the oddly inaccurate one is the ‘going off the trail’ option, and the correct answer is C.  (I confess, I’ve still not read the passage accompanying these questions, but my 7^th grade daughter confirmed these details.)

Interestingly, this question was categorized as ‘comprehension,’ which presumably ranks below ‘analysis’ on the intellectual spectrum.  The question is framed to look like a prediction question but really isn’t.  The student’s ability to comprehend which detail (from the latter half of the answer options) actually happened in the story is really what’s getting tested.  They needn’t predict anything.  This question was the hardest to answer without reading the passage, but many testers (including my daughter) were able to narrow it down to two answers and C was one of them.

Soon, I will post a new set of MSP released items....See if you can get them right.

Take the Measurement of Student Progress

Can you figure out the answers (or narrow it down to the top 2), just from the question and answer set?
In a later post, I'll explain how you can.  Give it a try for now.

Story 1--Excerpt from Iditarod Dream

1.  What is the main idea of “Excerpt from Iditarod Dream”?

1. Sled dog racing is a thrilling and dangerous sport. 
2. Sled dog racing requires teamwork and training. 
3. Sled dog racing requires specialized equipment. 
4. Sled dog racing can be a family activity.

2.  According to “Excerpt from Iditarod Dream,” why does Dusty decide to help the other racers build a fire?

1.         He uses the fire light to see the trail markers. 
2. He thinks the fire will help him stay awake. 
3. He is following the rule of the wilderness. 
4. He needs to cook the dogs’ frozen meat.

3.   According to “Excerpt from Iditarod Dream,” how would Dusty most likely react to entering another dog sled race?

A. He would be hopeful because he came so close to winning.

B. He would be nervous because he had trouble staying on the trail at night.

C. He would be excited because he knew how it felt to cross the finish line in the lead.

D.    He would be anxious because he ran out of supplies and needed more for the next race.

Story 2--Nurses in the Wilderness

4.  Which sentence tells how the Frontier Nursing Service and the Mary Breckinridge Hospital are similar?

1.   Both have modern supplies.
2.   Both provide rural medical services.
3.   Both are located in Wendover, Kentucky.
4.   Both train people from all over the world.

5. What is most likely the author’s main purpose for writing this selection?

1.   To inform the reader about the history of rural nursing 
2.   To describe the effects of diseases on rural children 
3.   To persuade the reader to support rural medicine 
4.   To explain the difficulty of travel in rural areas

6.  According to the captions in the selection, which statement is true?

1.   Nurses often called Mary an angel on horseback. 
2.   Patients were sometimes carried to clinics by neighbors. 
3.   Mary’s first clinic became the Mary Breckinridge Hospital. 
4.   Riding horseback was the only form of travel to the hospital.

7.   Which sentence best summarizes the selection?

1.   Mary and her family were always very generous to others.
2.   The Frontier Nursing Service reached far beyond Kentucky.
3.   Rural nurses found creative ways to transport supplies and patients.
4.   Mary and other nurses provided compassionate medical care to rural people.

Story 3-- A Touch of Genius

8.  Which sentence from the selection is an opinion?

1.     “The same process happens with me.”
2.     “He lost his sight and partial use of one hand.” 
3.     “I had no words to describe the emotion I felt.” 
4.     “Behind this statement lies a remarkable story.”

9. Which sentence best states the main idea of the selection?

1.     Michael served in Vietnam. 
2.     Michael is a talented artist. 
3.     Michael teaches sculpture in the pueblo. 
4.     Michael has displays in museums around the world.

10.   Which statement is the most important conclusion that may be drawn from the selection?

1.   War teaches people to be strong. 
2.   Art can make a difference in a child’s life. 
3.   Hardships can motivate a person to greatness. 
4.   Mothers have a great influence on their children.

11. After reading this selection, what generalization can you make about Michael?

1.   Michael is a motivated individual. 
2.   Michael is an excellent teacher.
3.   Michael likes military service. 
4.   Michael loves animals.

ANSWERS IN NEXT POST.