TCU should have played Oregon....Proves that standardized tests are sketchy!

A college football game that didn't happen is a lot like standardized testing of students.

By that I mean, if you read the linked article about whether TCU would have given Oregon a better game in the Rose Bowl than did Florida State, you can see the speculative nature of data.

It seems that all sorts of newfangled (and mind numbingly uninteresting) data analyses seem to suggest that one could argue that the Horned Frogs would indeed have been a better opponent than than the Seminoles, but nobody can definitively say so without the two having actually played. And, if they had (instead of Oregon and Florida State), arguments (and data) would undoubtedly have been mustered that FSU would have been better instead of TCU.  Absent actual contests, with definitive results, we will never know.

Of course, even when they play, the only thing that is clearly established is who won that particular contest. As the article points out, if Florida State played Oregon again, the data show that there's nearly a 100% chance they would perform better.

So, claims about who is the "better" team still have something of a speculative nature. One contest is a discrete event, whose outcome we accept as definitive, by definition. But certainly there have been single contests in which the lesser team won.  (I know...I've participated in many--as both winner and loser.)  A variety of intervening non-football variables affect outcomes, especially in one-time contests.

Apply the same thinking to standardized tests. They are one-time events, which means other non-test factors can intervene. More importantly, this one-shot game--whose outcome may or may not accurately reflect a student's "quality"--supposedly indicates whether or not a student is  succeeding--"getting educated," and whether teachers are performing adequately.

But can the test results really validate such claims?  Or, like football games, is the most that we can say that a student got this specific score on the that particular test, and no more?

Like arguing over whether TCU or FSU would have played Oregon better, saying that a one-shot discrete event called a standardized test signifies anything other than performance on that test is an arbitrary exercise.  Just as saying, by definition, the team with the most points (rather than, say, the most yards, or the cleanest uniforms, or anything else we might value) is named the winner and the "better" team, we impose a definition and marker of "successful" or "at standard" student that may or may not reflect anything worth really knowing about that student.


Indeed, the process is somewhat arbitrary.  And, of course, arbitrary invites capricious, so let's get sensible about what we're doing and keep a proper perspective on what these one-shot games--called standardized tests--can really do.  And it ain't much--pardon my sub-standard English.