Last week, I sat on a panel of union members interviewing one of the candidates for state legislature in my area. I am a full member of the union, though with reservations. I found it hard to ask questions so laced with self-interest, and I ended up unsure whether I could support the candidate BECAUSE he was so stridently union. "Over my dead body," he assured us regarding any legislation that would undermine public employee unions.
It seems to me that the urgency of protecting the union is treating the symptom, rather than the disease. The social climate regarding schools is poisonous--distrust fouls the air. Those who distrust teachers want accountability. Those same are distrusted by teachers, who respond by seeking mechanisms of protection.
If the reformers spent some more time in class rooms and worked along with teachers, rather than mandating at them, we might have a chance at real trust-building that minimized the need for convoluted and ineffective instruments of accountability and protection.
Short of that, I may need to invoke a stylization of Groucho and get out of this club that wants me as a member.
Thursday, April 26, 2012
Monday, April 23, 2012
How does this affect the standardized test score?
A colleague related a story of some people she was with who knowingly parked their cars in a business' parking lot to attend an event not associated with the business, under signs warning that such conduct would result in the vehicle's getting towed. When they returned, one of the cars was being towed. They were able to get the car back on the spot, but still had to pay a 3-figure fee for its having begun the towing journey.
They were furious enough that they complained to the business manager--offering some sort of lame explanation about it having been a community event, after all--and were reimbursed the towing fee and given a sizable gift card to the business.
How in the world, you may ask, does this have anything to do with education?
Two things. First, standardized test scores can go hang if our education system doesn't do something about this kind of corrosive self-absorption. Second, and more importantly, how can "education" (the system, individual teachers, etc.) do anything about such corrosive self-absorption when our society simply swims in such ridiculous justifications for insanely selfish behavior?
How in the world are a bunch of maligned and beleaguered teachers going to do anything about the teenage children of people who do such a thing as this? And yes, this does affect standardized test scores. If a self-absorbed school student doesn't see a need to apply him or herself to school work and test readiness, then s/he won't, and too often there's little available to compel greater attentiveness, especially in the face of such well-honed self-justification machinery.
Ask any teacher you know. They can tell you any number of stories of such selfishness, responsibility-avoidance, and blame-deflection.
They were furious enough that they complained to the business manager--offering some sort of lame explanation about it having been a community event, after all--and were reimbursed the towing fee and given a sizable gift card to the business.
How in the world, you may ask, does this have anything to do with education?
Two things. First, standardized test scores can go hang if our education system doesn't do something about this kind of corrosive self-absorption. Second, and more importantly, how can "education" (the system, individual teachers, etc.) do anything about such corrosive self-absorption when our society simply swims in such ridiculous justifications for insanely selfish behavior?
How in the world are a bunch of maligned and beleaguered teachers going to do anything about the teenage children of people who do such a thing as this? And yes, this does affect standardized test scores. If a self-absorbed school student doesn't see a need to apply him or herself to school work and test readiness, then s/he won't, and too often there's little available to compel greater attentiveness, especially in the face of such well-honed self-justification machinery.
Ask any teacher you know. They can tell you any number of stories of such selfishness, responsibility-avoidance, and blame-deflection.
Wednesday, April 11, 2012
Another Update to the Test?
State Superintendent Randy Dorn is apparently imagining yet another change to Washington's test. It seems the Legislature "funded a report on the merits of formative testing"--tests done at the beginning of the learning. So he wants to do more smaller tests through the year.
It took a study to figure out this was worth doing. If you want to evaluate teachers based on scores, the only study necessary would have been to ask teachers. They are desperate to have a baseline by which their impact on a student can be more accurately and usefully measured.
My district used to do this. A one-day computer-based test that correlated pretty well with WASL/MSP strand content and score outcomes. I could project MSP results based on our internal test results, and I could identify what students needed to work on by their results of early tests. We abandoned that for lack of money.
Another aspect of this--we keep changing the test, always presuming it's the 'same' (if we're going to do a lot evaluating of teachers under the current testing structure, it will be across test years) throughout all the changes. Consistency of the test instrument is part of 'standardized' also. Not sure how safe it is to assume consistency will be maintained, though.
It took a study to figure out this was worth doing. If you want to evaluate teachers based on scores, the only study necessary would have been to ask teachers. They are desperate to have a baseline by which their impact on a student can be more accurately and usefully measured.
My district used to do this. A one-day computer-based test that correlated pretty well with WASL/MSP strand content and score outcomes. I could project MSP results based on our internal test results, and I could identify what students needed to work on by their results of early tests. We abandoned that for lack of money.
Another aspect of this--we keep changing the test, always presuming it's the 'same' (if we're going to do a lot evaluating of teachers under the current testing structure, it will be across test years) throughout all the changes. Consistency of the test instrument is part of 'standardized' also. Not sure how safe it is to assume consistency will be maintained, though.
Wednesday, April 4, 2012
Tests with High Stakes...but for whom?
Most of the discussion of high stakes testing of public school students tends to overlook some fundamental, but crucial, aspects of the whole notion of such tests, especially their connection to teacher evaluations.
The test outcomes have no impact on the students' academic lives, so the stakes are not high for them. Rather, teachers feel the pressure of the high stakes, and increasingly so when student performance on those tests is going to be a factor in teacher evaluations, as it will undoubtedly be almost everywhere soon.
But any notion that we can rightly and wisely connect scores to evaluations suffers from several serious logical errors or flaws.
First, and most importantly, we are, to be logic of 'sciencey' about it, affirming the consequent--using test score changes as proof of teacher quality. The whole issue becomes tautological, as the measure of the independent variable (Teacher Impact) is observed as the result on the dependent variable (Test Score). This an invalid form of argument. To make the Teachers --> Test Score claims valid we must define measures of Teacher Impact prior to and separate from Test Scores, then hypothesize that Higher Teacher Impact scores will cause Higher Test Scores. Sadly, I don't see that happening. It's far too easy to simply define Teacher quality by Test Score results.
Second, connecting scores and evaluations implicitly looks upon the students as neutral actors in the whole scenario. By that I mean we must assume students are merely objects being turned or maneuvered by teachers and that whatever teachers do well or badly transmits fairly directly to students and shows up in their test scores. If student scores go up, that teacher did a good job. If not, not.
Or think of it this way. If something intervenes between the causal agent Teacher Impact and the outcome Student Performance which causes the measurement device Test Score to register something more or less than actual Teacher Impact on Student Performance, then Test Score is a less than fully accurate accounting of Teacher Impact.
We're assuming, in still other words, that Student Performance measured as Test Scores is actually an accurate (if logically invalid) measure of Teacher Impact.
Dubious assumption, as stuff intervenes, no doubt, between Teacher and Test Score. The question is, how much stuff, and how do we tell what effect it has?
But even if we could figure all that out, we have a third concern, this time about motivation. The students are the ones taking the tests, and their scores are a measure of accountability for the teacher, not for the student. The study of economics teaching us nothing if not this: You have to watch the incentives. Pay attention to who has incentive to do what things.
As we are constructing the situation, the teacher has a lot of incentive to make sure students do well. And I assume that's the hope. Motivate teachers and they'll go motivate and teach students. But the students don't have much incentive beyond some amorphously constructed internal drive to do well on the test.
To put it in a social sciencey kind of way, the agents (students) whose practical ability we are hoping to see (registered as test score outcomes) don't have appropriate incentives (or not as much as some other actors--teachers--who lack practical ability) to necessarily maximize performance. Nor is a legal authority available that could compel students to seek maximal outcomes against their own preference to do so or not.
The so-called strategic triangle of compliance (thanks to Ron Mitchell, p. 14 for teaching me that one), which in this case is test score maximization, does not get rightly made in this situation. The actors with incentives (Teachers) have some, but limited impact on the actors with the practical capacity (Students) to maximize scores, and the actor with 'legal' authority (Parents) to compel the exercise of greater student capacity has been dropped out of the scenario.
Unfortunately, I do not expect these concerns to derail a train with so much steam built up. Bring on the Test Tricks.
The test outcomes have no impact on the students' academic lives, so the stakes are not high for them. Rather, teachers feel the pressure of the high stakes, and increasingly so when student performance on those tests is going to be a factor in teacher evaluations, as it will undoubtedly be almost everywhere soon.
But any notion that we can rightly and wisely connect scores to evaluations suffers from several serious logical errors or flaws.
First, and most importantly, we are, to be logic of 'sciencey' about it, affirming the consequent--using test score changes as proof of teacher quality. The whole issue becomes tautological, as the measure of the independent variable (Teacher Impact) is observed as the result on the dependent variable (Test Score). This an invalid form of argument. To make the Teachers --> Test Score claims valid we must define measures of Teacher Impact prior to and separate from Test Scores, then hypothesize that Higher Teacher Impact scores will cause Higher Test Scores. Sadly, I don't see that happening. It's far too easy to simply define Teacher quality by Test Score results.
Second, connecting scores and evaluations implicitly looks upon the students as neutral actors in the whole scenario. By that I mean we must assume students are merely objects being turned or maneuvered by teachers and that whatever teachers do well or badly transmits fairly directly to students and shows up in their test scores. If student scores go up, that teacher did a good job. If not, not.
Or think of it this way. If something intervenes between the causal agent Teacher Impact and the outcome Student Performance which causes the measurement device Test Score to register something more or less than actual Teacher Impact on Student Performance, then Test Score is a less than fully accurate accounting of Teacher Impact.
We're assuming, in still other words, that Student Performance measured as Test Scores is actually an accurate (if logically invalid) measure of Teacher Impact.
Dubious assumption, as stuff intervenes, no doubt, between Teacher and Test Score. The question is, how much stuff, and how do we tell what effect it has?
But even if we could figure all that out, we have a third concern, this time about motivation. The students are the ones taking the tests, and their scores are a measure of accountability for the teacher, not for the student. The study of economics teaching us nothing if not this: You have to watch the incentives. Pay attention to who has incentive to do what things.
As we are constructing the situation, the teacher has a lot of incentive to make sure students do well. And I assume that's the hope. Motivate teachers and they'll go motivate and teach students. But the students don't have much incentive beyond some amorphously constructed internal drive to do well on the test.
To put it in a social sciencey kind of way, the agents (students) whose practical ability we are hoping to see (registered as test score outcomes) don't have appropriate incentives (or not as much as some other actors--teachers--who lack practical ability) to necessarily maximize performance. Nor is a legal authority available that could compel students to seek maximal outcomes against their own preference to do so or not.
The so-called strategic triangle of compliance (thanks to Ron Mitchell, p. 14 for teaching me that one), which in this case is test score maximization, does not get rightly made in this situation. The actors with incentives (Teachers) have some, but limited impact on the actors with the practical capacity (Students) to maximize scores, and the actor with 'legal' authority (Parents) to compel the exercise of greater student capacity has been dropped out of the scenario.
Unfortunately, I do not expect these concerns to derail a train with so much steam built up. Bring on the Test Tricks.
Subscribe to:
Posts (Atom)