What's middle school like...after coming back from remote learning? Well let me tell you...it's different. (If you were reading this with standardized test eyes, that's the thesis statement. Just didn't want you to miss it.) The rest of the blog will explain "different."
Friday, May 13, 2011
Servant Leadership instead of Leading the Servile
Wednesday, May 11, 2011
Test Results...what to do with them.
So, the last two years' scores.
Level 3 and 4 constitute 'passing.' Level 1 and 2 are 'not meeting standard.'
2010 8th grade testers | 8th grade results | 7th grade reading results | 7th grade writing results | 6th grade results |
Overall at Standard | 82.8 (69.4) | 74.2 (59.3) | 82.5 (69.8) | 84.9 (68.9) |
Level 4 | 51.2 (43.1) | 38.1 (27.2) | 37.1 (25.1) | 40.2 (25.6) |
Level 3 | 31.5 (25.6) | 36.1 (31.2) | 45.4 (43.8) | 44.7 (42.1) |
Level 2 | 17.2 (18.8) | 23.2 (29.1) | 17.5 (18.0) | 14.0 (23.1) |
Level 1 | 13.3 (10.7) | 2.6 (10.2) | 5.2 (10.1) | 1.1 (7.2) |
2009 8th grade testers | 8th grade results | 7th grade reading results | 7th grade writing results | 6th grade results |
Overall at Standard | 79.6 (67.5) | 73.2 (63.1) | 77.8 (70.0) | 74.2 (68.0) |
Level 4 | 48.4 (34.3) | 40.4 (34.3) | 22.2 (24.3) | 26.3 (25.9) |
Level 3 | 31.1 (32.5) | 32.8 (27.9) | 55.6 (44.8) | 47.9 (41.0) |
Level 2 | 14.2 (21.1) | 20.2 (26.5) | 18.2 (19.2) | 20.5 (24.0) |
Level 1 | 5.3 (9.8) | 5.1 (9.4) | 3.0 (9.3) | 5.3 (6.9) |
I still don't know what to read in these figures. It's great that we're above our state average. I don't know how much that has to do with me, or any other teacher, though.
(Note that most of the time we're also lowering than the state average on the 'not passing' scores.)
I was also pleased to see higher numbers passing at Level 4. Then again, I don't know if anybody has talked of making that a measure of teacher performance. What about 2010 in 8th grade? High Level 4 pass rate, but also higher than the state on Level 1 scores. Is that good, bad, or what?
What do we do with 7th grade, which tests on two language arts related subjects? Do we evaluate their test outcomes differently?
I hope this becomes clearer, and more open, than the test development process.
Friday, May 6, 2011
Not so Standardized Test
Thursday, May 5, 2011
The Civics Test may not capture what we're hoping for
Well, more data...this time that our students are pathetic in civics, too.
I wondered, so I wandered through the "report card" web site.
I came across this sample question.
The following question refers to the statement below.
The Second World War marked the most substantial change ever in the context in which United States foreign policy is made. The world that emerged after the war had fundamentally changed in economic, political, and military ways. These changes made the world a more dangerous place, and altered the demands placed on foreign policy.
The statement calls the world after the Second World War "a more dangerous place." What specific change could one cite to support this claim?
- The rise of the European Union (EU)
- The signing of the General Agreement on Trade and Tariffs (GATT)
- The decline of German military power
- The development and spread of nuclear weapons
Tuesday, May 3, 2011
Test Exhaustion...and we haven't even started yet
Explain how Michael became an award winning artist. Include two details from the selection in your answer.
The teacher booklet goes on to explain what respondents might put in their answers:
Text-based details may include, but are not limited to:
A. His mother was a potter / he helped her fix her clay
B. Playing with clay as a child
C. “I knew that what I wanted to do was be an artist someday.”
D. Making small animals in hospital after blindness/Vietnam, continued to
sculpt
E. Sculptures were photographed by newspapers
F. I get a picture in my mind / make his memories come to life
G. Inspires / leads sculpture workshops
H. Sculptures can be seen in museums/public buildings/Vatican /White House
I. People collect his work
Now, it doesn't matter which two items a test-taker lists in his/her answer. All items are equal, even those--like H, I, and maybe E and G--which don't really 'explain how he became an award-winning artist.'
H and I seem more the consequence of his becoming an award-winning artist, not an explanation of how he became one. I guess 'how he became' does not mean causality. I would have thought it did. But, by their answers, I gather they really mean something more like the event sequence of his rising to 'award-winning artist status.'
I hope this passage and questions got rejected for this flaccidity. I hope it wasn't something else, and that we really are (mis)testing comprehension with items like this.
Saturday, April 30, 2011
Character
Wednesday, April 27, 2011
Normal Accident Theory of Education, continued
Earlier, I started my Normal Accident Theory of Education. Here are a few key ideas necessary to advance the idea that education is rife with normal accidents or normal failures.
A brief but good overview of some of the definitions of key terms that make up Perrow's theory about Normal Accidents, by Professor Piccard of Ohio University.
High-Risk Systems
- This term encompasses risks "for the operators, passengers, innocent bystanders, and for future generations." He applies it to "enterprises [that] have catastrophic potential, the ability to take the lives of hundreds of people in one blow, or to shorten or cripple the lives of thousands or millions more." This means that although he does include chemical plant and refinery accidents, he is explicitly excluding from his focus the primary harmful impacts of fossil-fuel burning (greenhouse gases and toxic combustion products released into the atmosphere), since those effects are diffuse and happen by design, not as an accident. Education and schools are a bit different. Their accidents are not deadly, but they do have lasting impact. Today especially, we look upon school processes and their outcomes as high-stakes. The consequences of failure, then, are serious, though not deadly.
Normal Accidents
- Perrow uses this term in part as a synonym for "inevitable accidents." This categorization is based on a combination of features of such systems: interactive complexity and tight coupling. Normal accidents in a particular system may be common or rare, but the system's characteristics make it inherently vulnerable to such accidents, hence their description as "normal."
- Education accidents (or failures) are common, though lower intensity than the fatalities that would be associated with a nuclear meltdown or a plane crash.
Discrete Failures
- A single, specific, isolated failure is referred to as a "discrete" failure.
- A student's failure of one course, or one standardized test, would be a discrete failure. So, where a nuclear plant might have a discrete failure occasionally, the education system produces its smaller discrete failures routinely.
Redundant Sub-systems
- Redundant sub-systems provide a backup, an alternate way to control a process or accomplish a task, that will work in the event that the primary method fails. This avoids the "single-point" failure modes.
Interactive Complexity
- A system in which two or more discrete failures can interact in unexpected ways is described as "interactively complex." In many cases, these unexpected interactions can affect supposedly redundant sub-systems. A sufficiently complex system can be expected to have many such unanticipated failure mode interactions, making it vulnerable to normal accidents. A student's failure to 'get an education' would be an interactively complex failure. A student must systematically fail courses, annual tests, etc., though these discrete failures may not necessarily amount to not getting an education. It is difficult to determine 'getting an education,' and we have increasingly defined that as passing all the classes and annual tests (or at least the last of these tests, 10th grade in WA state).
Tight Coupling
- The sub-components of a tightly coupled system have prompt and major impacts on each other. If what happens in one part has little impact on another part, or if everything happens slowly (in particular, slowly on the scale of human thinking times), the system is not described as "tightly coupled." Tight coupling also raises the odds that operator intervention will make things worse, since the true nature of the problem may well not be understood correctly. The education system is loosely coupled, which means discrete failures here and there can be overcome. Persistence of discrete failure, however, which is, definitionally, evidence of interactive failure, can occur...with everyone's knowledge (so these failures are not incomprehensible--see below). One point of concern here. Perrow argues for decentralization of loosely coupled systems in order to allow flexible decision-making in the performance of the system. School reform portends increased centralization, though. This will increase the likelihood of normal accidents.
Incomprehensibility
- A normal accident typically involves interactions that are "not only unexpected, but are incomprehensible for some critical period of time." The people involved just don't figure out quickly enough what is really going wrong. A normal accident occurs in a complex system, one that has so many parts that it is likely that something is wrong with more than one of them at any given time. A well-designed complex system will include redundancy, so that each fault by itself does not prevent proper operation. However, unexpected interactions, especially with tight coupling, may lead to system failure. System operators must make decisions, even with ambiguous information. The process of making a tentative choice also creates a mental model of the situation. When following through on the initial choice, the visible results are compared to those expected on the basis of that initial mental model. Provided that the first few steps' results are consistent, the fact that the mental model was tentative is likely to be forgotten, even if later results contradict it. They become "mysterious" or "incomprehensible" rather than functioning as clues to the falsity of the earlier tentative choice. This is simply the way the human mind works, and systems designed with contrary expectations of their operators are especially vulnerable to system accidents.
Operator Error
- It is indeed the case that people sometimes do really stupid things, but when most of the accidents in a particular type of system (airplane, chemical plant, etc.) are blamed on the operator, that is a symptom that the operators may be confronted with an impossible task, that there is a system design problem. In a typical normal accident, the operator's actions may contribute to the problem, or even initiate the sequence of events, but the characteristics of tight coupling and interactive complexity also make their contributions. This brings us to the question--what is 'the education system' and who are the operators? If the system is the parents, home, community and school, then the operator is the individual student. If the system is just the schools, then the operators are the teachers and other school personnel. If we call the schools the system, we in effect leave 'incomprehensible' the functioning of the other parts of the student's educational life, and leave the student as something of a cipher. an entity being acted upon rather than an principal-operator of an educational system created--by his/her parents AND the schools--to make an education available to him/her.
For want of a nail ...
- The old parable about the kingdom lost because of a thrown horseshoe has its parallel in many normal accidents: the initiating event is often, taken by itself, seemingly quite trivial. Because of the system's complexity and tight coupling, however, events cascade out of control to create a catastrophic outcome. So, a cascade can come from a string of bad or inattentive teachers. Or, a cascade can come from a school and teachers monitoring and intervening, but other parts of the whole system (parents, student, etc.) not responding. I'm sure the former happens. I see several cases of the latter every year. Most common form--we identify a student who needs an intervention in math or reading, and suggest a one-quarter class for this, and the parent opts to keep their student in band or art. Parents' choice; school will be held responsible.
Organizations
- Organizational issues routinely confront the analyst of normal accidents. Because the interactions among subsystems are not predictable, the operators must be able to take prompt and independent action. Because of the tight coupling, in which operators of one part of the system influence the tasks confronting operators of other parts of the system, centralized control is required. These two conflicting requirements cannot be readily resolved, and the organizational attributes that work well for one side are likely to be dysfunctional for the other.
- Schools and school personnel are increasingly bound, not independent, in their actions. There are school districts that tell teachers exactly what they'll be teaching on a given day. And, of course, as soon as the stakes are raised on the standardized tests, we incentivize teachers to focus more on the test, and, for some, to even cheat.
- Okay, more later.