My card...
There's some interesting stuff.
Thursday, January 29, 2015
Wednesday, January 21, 2015
Tuesday, January 20, 2015
Seahawks undermine standardized testing even more than Oregon or Florida State
For two weeks now I've been musing about how the awful performances of first FSU, then Oregon proved that standardized testing is dubious. The Seattle Seahawks prove it all the more.
The point is this--We all agree that football games will generate a decisive outcome. By definition, that outcome is based on the number of points each team scores. But that outcome that day proves nothing more than who had more points on that particular occasion. It doesn't prove that one team is actually "better" than the other, except by definition--they won.
The same is true of standardized tests. They most clearly show what a student does on that particular test. We make inferential assumptions from the test scores, but those are more dubious than the one reliable observation about actual scores on the specific test.
Take the Seahawks beating the Packers in the NFC championship game. That particular outcome (Seattle winning) was predicted before, and probably "should have" held. And the chances were very slight that Green Bay would so dominate so much of the game, yet they did. Further, the chances that the Seahawks would still recover to win as late as they did were also very small. And yet they did. The chances that Seattle could do that again are so negligible as to be zero. And yet they did.
They did win that particular game, though at 55 minutes they probably still shouldn't have. But the thinking prior to the game predicted--in general--a Seattle win, in "the next, but yet-to-be-played" contest. But that prediction means nothing against the actual outcome--the concrete result we all agree to observe.
The point is that snapshots in time are, more than anything, measures of what is happening precisely at that moment. The further in time you move from that snapshot, the farther out--toward the tails--along a normal distribution curve you move. It's not quite how you use these curves, but think of the middle as the place where we discern the the actually observed outcome--the location of the snapshot called today's game. Accept this visual image, because the outcome of the game is--by definition--the most likely outcome (at the top of the curve). The winner is agreed upon at that time and place. Defining something called "better" team, over time or in the abstract, is speculative and therefore moves away from the center of the curve.
Sunday's game gives almost no insight into what happened 10 years ago, or what will happen in 10 years. While the "evidence" from the Seahawk-Packer contest earlier this season is closer in time, it didn't hold much predictive value for the Sunday's game--indeed, it doesn't even hold much retrodictive value, either. Likewise, this year's games will hold little predictive power for next year's games. The snapshot called "next year's game" will show what was happening at that time--what will be, or will have been happening....
Of course, the even more fundamental issue is that a football game creates clarity--one side wins and one side loses. And we accept that as defined by the rules. What such outcomes do not prove is who is the "better" team, in more subtle ways--or ways beyond that manner which is defined by the rules--than that binary result.
Standardized tests--unintentionally, no doubt--try to do the same thing. Students are rendered as either passed or failed, according to a snapshot taken in time. Using 8th grade standardized test scores to evaluate anything beyond what happened for that student on that day becomes increasingly speculative the further in time and farther in space that student moves away from that day's test.
This is probably a function (at least in some degree) of the limitations of such a binary measure of a person, who, by his/her very nature, fluctuates at least modestly in the performance of tasks. Furthermore, people are good at different things, some of which are not assessed by the current standardized tests. Like pointing out that the Seahawks have the more colorful uniforms, though, if we are choosing to focus on a set of specific indicators, those other measures of value and performance won't weigh in a student's or team's favor.
The point is that the standardized test process tries to reify something intangible and variable (learning) into a concrete "thing" that can measured and that we can call education. That's like saying that the Seahawks are better people, kinder, cooler, nicer, whatever, because they won on the particular measure of importance on Sunday. You know such things can't be true...just look at all those winning Raider teams from the 1970s and 1980s.
The point is this--We all agree that football games will generate a decisive outcome. By definition, that outcome is based on the number of points each team scores. But that outcome that day proves nothing more than who had more points on that particular occasion. It doesn't prove that one team is actually "better" than the other, except by definition--they won.
The same is true of standardized tests. They most clearly show what a student does on that particular test. We make inferential assumptions from the test scores, but those are more dubious than the one reliable observation about actual scores on the specific test.
Take the Seahawks beating the Packers in the NFC championship game. That particular outcome (Seattle winning) was predicted before, and probably "should have" held. And the chances were very slight that Green Bay would so dominate so much of the game, yet they did. Further, the chances that the Seahawks would still recover to win as late as they did were also very small. And yet they did. The chances that Seattle could do that again are so negligible as to be zero. And yet they did.
They did win that particular game, though at 55 minutes they probably still shouldn't have. But the thinking prior to the game predicted--in general--a Seattle win, in "the next, but yet-to-be-played" contest. But that prediction means nothing against the actual outcome--the concrete result we all agree to observe.
The point is that snapshots in time are, more than anything, measures of what is happening precisely at that moment. The further in time you move from that snapshot, the farther out--toward the tails--along a normal distribution curve you move. It's not quite how you use these curves, but think of the middle as the place where we discern the the actually observed outcome--the location of the snapshot called today's game. Accept this visual image, because the outcome of the game is--by definition--the most likely outcome (at the top of the curve). The winner is agreed upon at that time and place. Defining something called "better" team, over time or in the abstract, is speculative and therefore moves away from the center of the curve.
Sunday's game gives almost no insight into what happened 10 years ago, or what will happen in 10 years. While the "evidence" from the Seahawk-Packer contest earlier this season is closer in time, it didn't hold much predictive value for the Sunday's game--indeed, it doesn't even hold much retrodictive value, either. Likewise, this year's games will hold little predictive power for next year's games. The snapshot called "next year's game" will show what was happening at that time--what will be, or will have been happening....
Of course, the even more fundamental issue is that a football game creates clarity--one side wins and one side loses. And we accept that as defined by the rules. What such outcomes do not prove is who is the "better" team, in more subtle ways--or ways beyond that manner which is defined by the rules--than that binary result.
Standardized tests--unintentionally, no doubt--try to do the same thing. Students are rendered as either passed or failed, according to a snapshot taken in time. Using 8th grade standardized test scores to evaluate anything beyond what happened for that student on that day becomes increasingly speculative the further in time and farther in space that student moves away from that day's test.
This is probably a function (at least in some degree) of the limitations of such a binary measure of a person, who, by his/her very nature, fluctuates at least modestly in the performance of tasks. Furthermore, people are good at different things, some of which are not assessed by the current standardized tests. Like pointing out that the Seahawks have the more colorful uniforms, though, if we are choosing to focus on a set of specific indicators, those other measures of value and performance won't weigh in a student's or team's favor.
The point is that the standardized test process tries to reify something intangible and variable (learning) into a concrete "thing" that can measured and that we can call education. That's like saying that the Seahawks are better people, kinder, cooler, nicer, whatever, because they won on the particular measure of importance on Sunday. You know such things can't be true...just look at all those winning Raider teams from the 1970s and 1980s.
Monday, January 19, 2015
Commemorating the Civil Rights Movement
You could do a lot worse than to read and ponder (and contribute to?) our fellow Americans' thoughts on race at The Race Card Project.
My six words:
Adopted black son; race MORE confusing
My six words:
Adopted black son; race MORE confusing
Tuesday, January 13, 2015
Oregon is the New Florida State
Last week, I mused that Florida State's awful performance in the Rose Bowl both a) raised concern about TCU's banishment (especially given their utter flattening of Ole Miss in the Peach Bowl) from the first college football playoff, and--more importantly-- b) demonstrated that our hopes for standardized testing of students are probably misguided.
This week, Oregon's collapse also proves B above. Oregon is this week's FSU, by which I mean that data and the statistical manipulation of the same will show that Oregon would do far better should they play Ohio State again. This was what the data showed last week about FSU's mauling by Oregon.
While it may be "fun" to muse over the data in this way, we have to admit that such parlor games are highly speculative. We have to actually play the games to see what would happen, and when we do play them, we grant to them a finality and definitiveness--by definition--of meaning. Ohio State won, they are the national champions, and any claim that Oregon is actually a better team that just happened to lose is merely speculative. We grant to the single data point called "the championship game" a supreme authority that, though it is really only proof of who won that night, bestows the distinction of accomplishment that lasts all year--until the next champion is determined.
In short, we invest a lot authority in what really are one-shot, or one-time, games. For college football, we can accept the snapshot as the final outcome, for football is a game, without much relevance (unless you're talking about Alabama) to much else.
Is such reductionism really a good idea when it comes to thinking about and evaluating students? By so much emphasis on the standardized test, we are saying "Yes" to that question.
Just say, "NO!"
This week, Oregon's collapse also proves B above. Oregon is this week's FSU, by which I mean that data and the statistical manipulation of the same will show that Oregon would do far better should they play Ohio State again. This was what the data showed last week about FSU's mauling by Oregon.
While it may be "fun" to muse over the data in this way, we have to admit that such parlor games are highly speculative. We have to actually play the games to see what would happen, and when we do play them, we grant to them a finality and definitiveness--by definition--of meaning. Ohio State won, they are the national champions, and any claim that Oregon is actually a better team that just happened to lose is merely speculative. We grant to the single data point called "the championship game" a supreme authority that, though it is really only proof of who won that night, bestows the distinction of accomplishment that lasts all year--until the next champion is determined.
In short, we invest a lot authority in what really are one-shot, or one-time, games. For college football, we can accept the snapshot as the final outcome, for football is a game, without much relevance (unless you're talking about Alabama) to much else.
Is such reductionism really a good idea when it comes to thinking about and evaluating students? By so much emphasis on the standardized test, we are saying "Yes" to that question.
Just say, "NO!"
Thursday, January 8, 2015
Second Installment on my 15 minutes (of fame)!?
Here's the link to the video of my talk about The Normal Accident Theory of Education at Pacific Lutheran University.
Monday, January 5, 2015
TCU should have played Oregon....Proves that standardized tests are sketchy!
A college football game that didn't happen is a lot like standardized testing of students.
By that I mean, if you read the linked article about whether TCU would have given Oregon a better game in the Rose Bowl than did Florida State, you can see the speculative nature of data.
It seems that all sorts of newfangled (and mind numbingly uninteresting) data analyses seem to suggest that one could argue that the Horned Frogs would indeed have been a better opponent than than the Seminoles, but nobody can definitively say so without the two having actually played. And, if they had (instead of Oregon and Florida State), arguments (and data) would undoubtedly have been mustered that FSU would have been better instead of TCU. Absent actual contests, with definitive results, we will never know.
Of course, even when they play, the only thing that is clearly established is who won that particular contest. As the article points out, if Florida State played Oregon again, the data show that there's nearly a 100% chance they would perform better.
So, claims about who is the "better" team still have something of a speculative nature. One contest is a discrete event, whose outcome we accept as definitive, by definition. But certainly there have been single contests in which the lesser team won. (I know...I've participated in many--as both winner and loser.) A variety of intervening non-football variables affect outcomes, especially in one-time contests.
Apply the same thinking to standardized tests. They are one-time events, which means other non-test factors can intervene. More importantly, this one-shot game--whose outcome may or may not accurately reflect a student's "quality"--supposedly indicates whether or not a student is succeeding--"getting educated," and whether teachers are performing adequately.
But can the test results really validate such claims? Or, like football games, is the most that we can say that a student got this specific score on the that particular test, and no more?
Like arguing over whether TCU or FSU would have played Oregon better, saying that a one-shot discrete event called a standardized test signifies anything other than performance on that test is an arbitrary exercise. Just as saying, by definition, the team with the most points (rather than, say, the most yards, or the cleanest uniforms, or anything else we might value) is named the winner and the "better" team, we impose a definition and marker of "successful" or "at standard" student that may or may not reflect anything worth really knowing about that student.
Indeed, the process is somewhat arbitrary. And, of course, arbitrary invites capricious, so let's get sensible about what we're doing and keep a proper perspective on what these one-shot games--called standardized tests--can really do. And it ain't much--pardon my sub-standard English.
By that I mean, if you read the linked article about whether TCU would have given Oregon a better game in the Rose Bowl than did Florida State, you can see the speculative nature of data.
It seems that all sorts of newfangled (and mind numbingly uninteresting) data analyses seem to suggest that one could argue that the Horned Frogs would indeed have been a better opponent than than the Seminoles, but nobody can definitively say so without the two having actually played. And, if they had (instead of Oregon and Florida State), arguments (and data) would undoubtedly have been mustered that FSU would have been better instead of TCU. Absent actual contests, with definitive results, we will never know.
Of course, even when they play, the only thing that is clearly established is who won that particular contest. As the article points out, if Florida State played Oregon again, the data show that there's nearly a 100% chance they would perform better.
So, claims about who is the "better" team still have something of a speculative nature. One contest is a discrete event, whose outcome we accept as definitive, by definition. But certainly there have been single contests in which the lesser team won. (I know...I've participated in many--as both winner and loser.) A variety of intervening non-football variables affect outcomes, especially in one-time contests.
Apply the same thinking to standardized tests. They are one-time events, which means other non-test factors can intervene. More importantly, this one-shot game--whose outcome may or may not accurately reflect a student's "quality"--supposedly indicates whether or not a student is succeeding--"getting educated," and whether teachers are performing adequately.
But can the test results really validate such claims? Or, like football games, is the most that we can say that a student got this specific score on the that particular test, and no more?
Like arguing over whether TCU or FSU would have played Oregon better, saying that a one-shot discrete event called a standardized test signifies anything other than performance on that test is an arbitrary exercise. Just as saying, by definition, the team with the most points (rather than, say, the most yards, or the cleanest uniforms, or anything else we might value) is named the winner and the "better" team, we impose a definition and marker of "successful" or "at standard" student that may or may not reflect anything worth really knowing about that student.
Indeed, the process is somewhat arbitrary. And, of course, arbitrary invites capricious, so let's get sensible about what we're doing and keep a proper perspective on what these one-shot games--called standardized tests--can really do. And it ain't much--pardon my sub-standard English.
What if we asked better questions?
Education Secretary Arne Duncan asks the question that assumes schools--their design, structure, programs--are the answer, and simply drops out all sociological realities.
The opportunity most desirable in the most successful schools is likely something a school can't replicate--the family and socio-economic circumstances of the student body/neighborhood. The opportunities that matter are not a better curriculum or more computers or a particular pedagogy. Stable families with enough resources (not just money, but time and willingness) to engage in a child's learning do a lot more than those things.
But schools, districts, bureaucrats, etc., have to do something, so they grasp for things that are possible within the organizational capacities available to them. For bureaucrats who live and work far away from the schools they're governing that something might be to ask vapid questions like this.
Education is complicated...
Free Pre-K doesn't make it less so...
Obviously, the immediate benefits of intensive support are clear. But this story does not discuss (likely because there really isn't so much available to "do" about it) the lack of subsequent family or institutional support down the years. An early foundation is critical, but on-going support of the work to build on that foundation is also required.
This is why the whole "get them reading by 3rd grade or they'll have trouble forever" discussion tends to miss the point. The answer isn't to give them all kinds of extra support to read by 3rd grade (then withdraw that extra support, thinking that everything is now fine). It's not that children's brains couldn't learn after 3rd grade, but that the engagement, support, involvement by adults in learning activities outside of school (reading, talking, singing, asking, wondering, solving, etc....) that was missing before third grade will likley still be missing after third grade. And institutional school--perhaps even with the extra support programs--just isn't enough to make up all the deficits.
In other words, a student's inability to read by 3rd grade might (likely) be an indication of a deeper systemic problem in that student's academic environment. Extra support to get to reading is treating a symptom rather than the root issue. On-going treatment of symptoms isn't really a long-term strategy.
The problem is that learning is a complicated and variable, and programmatic responses tend to focus on the effects of the program, measured more in the short-term than long.
The problem is that learning is a complicated and variable, and programmatic responses tend to focus on the effects of the program, measured more in the short-term than long.
Subscribe to:
Posts (Atom)