Saturday, April 30, 2011
Wednesday, April 27, 2011
Earlier, I started my Normal Accident Theory of Education. Here are a few key ideas necessary to advance the idea that education is rife with normal accidents or normal failures.
A brief but good overview of some of the definitions of key terms that make up Perrow's theory about Normal Accidents, by Professor Piccard of Ohio University.
- This term encompasses risks "for the operators, passengers, innocent bystanders, and for future generations." He applies it to "enterprises [that] have catastrophic potential, the ability to take the lives of hundreds of people in one blow, or to shorten or cripple the lives of thousands or millions more." This means that although he does include chemical plant and refinery accidents, he is explicitly excluding from his focus the primary harmful impacts of fossil-fuel burning (greenhouse gases and toxic combustion products released into the atmosphere), since those effects are diffuse and happen by design, not as an accident. Education and schools are a bit different. Their accidents are not deadly, but they do have lasting impact. Today especially, we look upon school processes and their outcomes as high-stakes. The consequences of failure, then, are serious, though not deadly.
- Perrow uses this term in part as a synonym for "inevitable accidents." This categorization is based on a combination of features of such systems: interactive complexity and tight coupling. Normal accidents in a particular system may be common or rare, but the system's characteristics make it inherently vulnerable to such accidents, hence their description as "normal."
- Education accidents (or failures) are common, though lower intensity than the fatalities that would be associated with a nuclear meltdown or a plane crash.
- A single, specific, isolated failure is referred to as a "discrete" failure.
- A student's failure of one course, or one standardized test, would be a discrete failure. So, where a nuclear plant might have a discrete failure occasionally, the education system produces its smaller discrete failures routinely.
- Redundant sub-systems provide a backup, an alternate way to control a process or accomplish a task, that will work in the event that the primary method fails. This avoids the "single-point" failure modes.
- A system in which two or more discrete failures can interact in unexpected ways is described as "interactively complex." In many cases, these unexpected interactions can affect supposedly redundant sub-systems. A sufficiently complex system can be expected to have many such unanticipated failure mode interactions, making it vulnerable to normal accidents. A student's failure to 'get an education' would be an interactively complex failure. A student must systematically fail courses, annual tests, etc., though these discrete failures may not necessarily amount to not getting an education. It is difficult to determine 'getting an education,' and we have increasingly defined that as passing all the classes and annual tests (or at least the last of these tests, 10th grade in WA state).
- The sub-components of a tightly coupled system have prompt and major impacts on each other. If what happens in one part has little impact on another part, or if everything happens slowly (in particular, slowly on the scale of human thinking times), the system is not described as "tightly coupled." Tight coupling also raises the odds that operator intervention will make things worse, since the true nature of the problem may well not be understood correctly. The education system is loosely coupled, which means discrete failures here and there can be overcome. Persistence of discrete failure, however, which is, definitionally, evidence of interactive failure, can occur...with everyone's knowledge (so these failures are not incomprehensible--see below). One point of concern here. Perrow argues for decentralization of loosely coupled systems in order to allow flexible decision-making in the performance of the system. School reform portends increased centralization, though. This will increase the likelihood of normal accidents.
- A normal accident typically involves interactions that are "not only unexpected, but are incomprehensible for some critical period of time." The people involved just don't figure out quickly enough what is really going wrong. A normal accident occurs in a complex system, one that has so many parts that it is likely that something is wrong with more than one of them at any given time. A well-designed complex system will include redundancy, so that each fault by itself does not prevent proper operation. However, unexpected interactions, especially with tight coupling, may lead to system failure. System operators must make decisions, even with ambiguous information. The process of making a tentative choice also creates a mental model of the situation. When following through on the initial choice, the visible results are compared to those expected on the basis of that initial mental model. Provided that the first few steps' results are consistent, the fact that the mental model was tentative is likely to be forgotten, even if later results contradict it. They become "mysterious" or "incomprehensible" rather than functioning as clues to the falsity of the earlier tentative choice. This is simply the way the human mind works, and systems designed with contrary expectations of their operators are especially vulnerable to system accidents.
- It is indeed the case that people sometimes do really stupid things, but when most of the accidents in a particular type of system (airplane, chemical plant, etc.) are blamed on the operator, that is a symptom that the operators may be confronted with an impossible task, that there is a system design problem. In a typical normal accident, the operator's actions may contribute to the problem, or even initiate the sequence of events, but the characteristics of tight coupling and interactive complexity also make their contributions. This brings us to the question--what is 'the education system' and who are the operators? If the system is the parents, home, community and school, then the operator is the individual student. If the system is just the schools, then the operators are the teachers and other school personnel. If we call the schools the system, we in effect leave 'incomprehensible' the functioning of the other parts of the student's educational life, and leave the student as something of a cipher. an entity being acted upon rather than an principal-operator of an educational system created--by his/her parents AND the schools--to make an education available to him/her.
For want of a nail ...
- The old parable about the kingdom lost because of a thrown horseshoe has its parallel in many normal accidents: the initiating event is often, taken by itself, seemingly quite trivial. Because of the system's complexity and tight coupling, however, events cascade out of control to create a catastrophic outcome. So, a cascade can come from a string of bad or inattentive teachers. Or, a cascade can come from a school and teachers monitoring and intervening, but other parts of the whole system (parents, student, etc.) not responding. I'm sure the former happens. I see several cases of the latter every year. Most common form--we identify a student who needs an intervention in math or reading, and suggest a one-quarter class for this, and the parent opts to keep their student in band or art. Parents' choice; school will be held responsible.
- Organizational issues routinely confront the analyst of normal accidents. Because the interactions among subsystems are not predictable, the operators must be able to take prompt and independent action. Because of the tight coupling, in which operators of one part of the system influence the tasks confronting operators of other parts of the system, centralized control is required. These two conflicting requirements cannot be readily resolved, and the organizational attributes that work well for one side are likely to be dysfunctional for the other.
- Schools and school personnel are increasingly bound, not independent, in their actions. There are school districts that tell teachers exactly what they'll be teaching on a given day. And, of course, as soon as the stakes are raised on the standardized tests, we incentivize teachers to focus more on the test, and, for some, to even cheat.
- Okay, more later.
Saturday, April 23, 2011
Friday, April 22, 2011
Take 'democracy,' for instance.
Democracy has two meanings. One is the process by which we all access and participate in the machinery by which we make decisions about how we live together. To a political scientist this means the instruments that are available to express and aggregate our opinion, then deliver it to the political leadership, and participate in selecting who that leadership is. Note that voting is a relatively small aspect of the available instruments.
The second is about the fairness and justice of the outputs of that machinery. Do we have equity, equality, human rights and needs met reasonably?
The first is about equality of access, the second is about equality of outcome...and these are different.
We can have a functioning democracy that generates very inequitable outcomes. In fact, we do.
In fact, the variable levels of 'social entrepreneurialism' (I stole that euphemism) in people means that a participatory system will generate differential outcomes for people.
Market forces, then, are not solutions before they are simply a variation in the participatory arrangements.
So, while I am a righty who likes markets, I am also aware that whatever market elements we embrace, in 5 years we'll be arguing over their deficiencies and skewed outcomes.
Wednesday, April 20, 2011
Tightly coupled complex systems are subject to normal accidents. Unexpected interactions (of the system parts) generate unpredicted and (in a complex system) often unnoticed effects on the performance of other parts of the system. If a complication arises, it may be too late before anybody even realizes the problem, let alone does anything about it. Nuclear power plants are a tightly coupled complex system. The Valujet flight that fell into the Florida Everglades did so because of a normal accident in oxygen canister management.
So, education is a normal accident (if we mean 'failing to meet' not just standard but a variety of other expectations beyond the testing) waiting to happen in that the whole system—from home and parents to the school house and class rooms--must meet the needs of an incredibly wide variety of students with variant levels of preparation, commitment and support.
Schools are then a normal accident waiting to happen in the way we neglect the reality of the above and choose to redefine the organizational goals as 'meeting standard' in some few test areas for all students. This narrows the work of the organization, and then creates a situation in which we miss or minimize other needs a student might have.
And, well, there's more. But I'll save the elaboration for another post.
Tuesday, April 19, 2011
Yes, a story, but a true one, and, as we say, quite good experiential learning.
The whole thing started strangely.
"Good morning, Mr. Milton. This is Tiffannee." (I'm sure that's how it was spelled.)
"I'm calling from Opinion Research Associates (or some such) in suburban Knoxville" (or some place whose name and general location I know, but whose specifics—character, place, theme, etc.— escape me).
"I'm not trying to sell anything. I'm calling because you've been selected to participate in an opinion research survey. Do you have a few minutes?"
I like these. It's fun to try to outmaneuver the questions while not demeaning the questioner, an innocent hourly employee who just wants to get the calls done. I was in bliss during one presidential election season in which I had registered as "undeclared." The parties' pollsters must have thought I was the elusive and all-important swing voter, so in the course of a few months, I got to play the evasive swing voter no fewer than 4 times each with both the big parties. I could so easily figure out which party was sponsoring the "survey," that I lost interest in asking, and eventually, in answering.
In my favorite call, I was asked whether I would tell the President, then a Democrat, if I supported Democratic Congressman Whozit's bill restricting handguns. (Democrats calling.) When I said 'No,' I'm certain, given where I lived, they ticked the 'Republican/NRA/Whacko' box for me. I told the questioner that I didn't like it because it was way too mild a bill, but since a stronger bill really isn't part of the landscape of discussion in our society, I knew they'd think I was a pro-gun type.
"Yeah, ok," was basically the response I got. You know, the sort of 'Hey, they don't pay me enough to deal with this malarkey' kind of 'yeah, ok.'
"Sorry, this is what you get for calling a political science PhD candidate who spends a lot of time at home pushing his toddler son around on his Tommy the Tooting Train."
Woo, woo. Chug, chug, chug…
…All Aboard! "Sure, Tiffannee." I'll take the bait, I think to myself.
"First," she interjects, "I have to ask if you're available next Saturday from 8:45 to 4:30. That's when the"—and now the word escapes me, but it is important, as the crux of the strangeness resides here—"[whatever it is] will take place."
I was available, so she asked if I knew these and those people (parties to the case). I didn't, so we could proceed. What followed was not one of those fun surveys, but a description of something to come next Saturday, something I could only construe as part market research, part mock trial, part lawyerly information gathering. I wasn't quite sure what it was going to be, but I persuaded myself it would be even more fun than outguessing the major political survey takers, and they were going to pay me $150.
Later, I would find that my comrades were equally vexed. The letter we had all gotten said "jury research project," though "market research" was splashed generously through the letter's several paragraphs. Rereading the letter now throws no brighter illumination of what I just finished participating in.
The same day the letter came, I got a second phone call, which I remember better, except for the caller's name, so let's call him Philip. Philip was calling to make sure I was still coming.
"Yeah, it seems like teachers, like yourself, understand better. You all seem to know the idea of market research." Market research, I remember those words clearly because they befuddled me. I had spent the past several days imagining that one side or the other in a pending lawsuit wanted to do a mock trial and examine our responses to various aspects of their case. If politics has its 'trial balloons' and 'focus groups,' litigation must have its 'market research,' I figured. I convinced myself satisfactorily enough that I didn't think about it again.
"Be sure to be there by 8:45," both Tiffannee and then Philip reminded me over and over. "You have to be there by 8:45 in order to participate, and you have to stay to 4:30 to get the $150."
"Look," I said, "I'm a prompt person, and I hate getting there at the appointed time only to find that this was the front end of the arrival window and everyone is actually allowed to trickle in for the real start time, which is some distant hour." So 15 minutes isn't all that distant, but it's the principle.
"Just be sure to be there by 8:45."
By now, I guess, I'd overlooked several unusual things—the unusualest being this: Why are law firms from the Puget Sound region hiring an opinion research firm from somewhere in the Deep South to secure participants in a mock trial? Must be a pretty big case, I thought, but not too hard or for too long. We had plans for the 150 bucks, and I was kind of pondering that bonus drawing for an extra 100 if you, in fact, participate for the whole day. The money, and the chance to slyly deploy my prodigious analytical skills, was more than adequate to distract from any further wondering about just what I was getting myself into.
Saturday, 8:40. Five minutes to spare. Sign in, get my ID tag (Juror #14), and a few forms to fill out. Squeeze in to one of the chairs lined up, tightly, under and along a richly paneled wall with elegantly framed windows…all with blinds drawn.
Stuck all day, in what looks like a mini-courtroom. We jurors face a set of desks, which sit at right angles to us. On our right, a stylish old partner's desk, flanked by the state and US flags. The "judge" must sit there. Across from the judge, on our left, counsels' tables. Their chairs look an awful lot more comfortable than the one I'm in.
And, what kind of opinion or market research will they be getting from us in such an officious arrangement? Even for a mock trial, the whole thing feels eerily overbearing.
Not much time to think about it…forms to complete. What kind of person am I? What's our household income? Do I know these and those people—same list?
8:55, done. Hand back the forms. The sign-in list shows that one juror didn't check in until 8:50. The threat of being excluded is going unfulfilled, but, I presume, there'll be one fewer person for the bonus drawing....And we're ready to go.
9:00, at least one juror colleague is still filling out paper work. So it's going to be more than 15 minutes.
9:05, Nurcan, the facilitator of the day's activities assures us we'll start in a few minutes, and invites waiting jurors to enjoy the Danish and coffee. Several of my colleagues move to the other room, and sit down with their refreshments. We're not starting any time soon, so I ask my neighbor about the Steven King book she's reading. Never having read one myself—indeed, I've probably read 10 fiction books in the last 10 years, all I can think to say is, "Did you know that he wrote The Running Man under a pseudonym? You know…the Schwarzenegger movie? With Richard Dawson, too?"
Ways Andrew Should Wait More Patiently. SURVEY SAYS!... Don't talk to neighboring jurors.
9:23, we're going to start. Oh, Nurcan—the founder of this legal process facilitating firm, which presumably does this routinely—realizes we're one chair short. Wait some more.
9:25….STARTING! Nurcan first assures us this is a real case and our decisions will have real consequences for real people. Lots of talk about our responsibilities, threats of being hauled before a judge if we talk about this case outside this room, reminders not to speak about it during breaks, etc. Our brief instructions about 'standards of proof and evidence' are elaborated with a reminder about how it's just "like what you hear on TV shows like CSI."
Now, I'm mad. Invoking a TV drama to explicate the civic requirements of the justice process means they're wanting me to take this more seriously than they take it. So with a boldness that later earns me immediate and spontaneous and consensus nomination to be foreman of our jury, I ask what "real consequences for real people" means.
"Will our decision today have some sort of legal standing," I ask. "I was told we were doing opinion research." Murmurings from Jurors 1-13 confirm that everybody understood something different from what we were now experiencing.
Nurcan coolly put us off. "I don't know why they told you that….I'll have to look into that." This, even though she had just borrowed, from one of my fellow jurors, a copy of the opinion research firm's letter to us.
"So, is this a trial," another juror asks.
"Your decision will have a real effect on real people, and that is enough of an answer." A slightly foreign bearing in Nurcan's English serves to make it a little easier to dismiss us. A mildly aloof demeanor that seems partly constituted of an "I don't quite understand you" expression requires us to just let it pass.
All at once an Agatha Christie-ish, 10 Little Indians type of feel swept over me. We're in an odd house, drawn here under different pretense than what is really happening. Told we must do certain things but not told why we're doing them. Is there a record player around? To play the recording about all the things I've done wrong? 14 ceramic figurines to knock off, one at time, as we all succumb?
And, look at the ceiling—the ubiquity of the video cameras is pressing in on me…literally, as we must—MUST—sit hip to hip and knees in the backs of the row ahead in order to all fit in the frame. Wait a minute…they never asked my permission to video tape me. Keyword alternatives for a YouTube search race through my mind.
'Fake Trial' 'Jury Fail' 'Dumb Jury Subjects' 'Fake Jury Study'
I'm convinced that by Sunday morning any of these will retrieve what's about to become the "snuff film" of my dying dignity. But as it turns out, I wouldn't have to wait that long. Nurcan brought it all about much quicker.
We heard a brief presentation by both sides…what seemed like opening statements with a little bit of evidence. Then off to lunch. No, wait…again. Nurcan forgot the 45 minutes of video excerpts of the depositions of the two principles. First the video, then, at 11:45, 45 minutes for lunch. At 12:30, ready to go, Nurcan's assistant announces, "20 more minutes."
After 65 minutes of break—and more warnings not to talk about the case, ever—we're off to review a few documents and deliberate, with all of us crammed on one side of the table, faces pointed at a camera on the ceiling. On pain of death, I can't say anything about what we did or didn't decide.
Debriefing with the two attorneys, and questions like, who did you think was behind this?
"The defendant," (a big organization) we all agreed. No indication if we were right.
"Did you think this was making a mountain out of a molehill," they enquired.
"Yes," we again all agreed.
"Did any of you think the defendant was lying?"
"Did any of you think the plaintiff's testimony had problems?"
Never at any point did we get even a dribble of information about what was "really" going on. We were left guessing or wondering…unless of course the $150 was enough to buy a diverting quiescence in the hearts of the individual jurors. It didn't in mine, but not because of the amount as much as the nature of the money.
One last warning before we go.
"Let me tell you a story," Nurcan says. "You think Pierce County is a small place—I mean big. It's not. You can't talk to anybody about this case. Not your spouse, not at work." Odd story—started slow, and never really got better.
But she'd said the confidentiality worked both ways. So, since I didn't give any contact information on Saturday, she'd have to get it from the opinion research firm. And the lawyers would have to start even further back, asking Nurcan to go the research firm, etc. Now, do I think that if they really needed to find me they'd stop themselves from violating our confidentiality agreement? I don't, but if I black out all detail that could identify me, or that they could use in conjunction with watching that videotape they never said they were taking, and since there really wasn't a juror 14, they won't be able to figure out who I am.
As I revel in the bizarre strangeness of the whole day, and delight at spinning out this little yarn, an impulse almost imperceptible and now only faintly remembered sets me in front of the computer. The web is a phenomenal thing.
Within about 10 minutes of browsing I discover I'd gotten it all wrong. I couldn't have been wronger, in fact. A Google search by the plaintiff's name retrieved a story from a local newspaper which reveals not only the fact that this lawsuit was filed two years ago, but her attorney's name, and the fact that this same attorney had recently won a similar case for an amount of money half way between 6 and 7 figures.
A Google search by the attorney's name leads to the web site of a sizable (and aggressive) local law firm. Reading through the list of attorneys at the firm, I recognized another name from our "no-know" list. The firm is kind enough to provide photos of their attorneys, so there it is for me to see—our two unnamed presenters from Saturday are, in fact, counsel for the plaintiff.
If you've ever discovered something dramatic and secret and surprisingly unpleasant about someone, you've had that "did I even really know this person?" feeling, the kind that makes you reconsider all your general history with that person, and some of the specific events they were involved in, and which compels you to rewrite the tone and mood of that history and revise your understanding of the outcomes of those events.
If you've ever had that happen, then you know precisely what I went through at 11:30 Sunday night. Everything was different now, darker, more banal and yet more sinister at the same time. What exactly had I signed on the confidentiality agreement that they did not give me a copy of? Hadn't I always gotten to keep a copy of those kinds of legal documents before?
And why did they never tell us verbally or in writing (if it's on the confidentiality agreement, I don't remember) that we were being videotaped? Unless it's buried in the confidentiality agreement, I don't remember signing a consent to be taped, and everyone agrees that written consent—on a specific video consent form—is the best protection for all involved. How many times have I been annoyed with the abundance of forms for just such purposes?
Could they now deny ever taping us? I don't have the tape to show it. But hadn't Nurcan lined up our chairs just so (leaving us uncomfortably close, at that) under a camera? And firmly said NO—without explanation, when one juror asked to move around the table? And as I left hadn't I seen a large TV monitor—then showing a snow pattern—upstairs in the same room where one of the attorneys was gathering his coat and bag?
Then I remembered reading about the ethical requirements of studying human subjects. As The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research describes it, they were supposed to get my informed consent, which involves giving me, among other things, "sufficient information on which to decide whether or not to participate, including the research procedure(s), [and] their purposes."
Why, I even remember conducting a study once in which we had to tell the subjects verbally and in writing that they were free to leave the study at any time, and they had to sign a paper saying they understood this. Telling us we wouldn't get the $150 if we didn't participate in the whole process isn't exactly the same as having us sign a paper saying we understand we could leave.
I didn't win the bonus drawing, and I'm almost glad I didn't. I did sign one last paper, though, and here the tale ends as strangely as it started. On completion we signed out. Nurcan's assistant made a mark in the last column of the row with my name. It was to note "Envelope Received." Not "Payment Received." Not" $150." Envelope Received. Inside, three 50 dollar bills.
It may as well have read, "Juror 14, Compromising Completed."
Monday, April 18, 2011
Friday, April 15, 2011
I read another claim about programs that make "significant impact on student achievement."
I'll presume positive impact, but the bigger question is 'achievement of what?' I'm not being sassy here. If achievement of higher test scores, that's somewhat immaterial to whether the student is getting a good education. If it's achievement of self-esteem, I need to see the data that shows such achievement is 'best practice.' If something else, please specify.
The What Works Clearinghouse says it is a "trusted source of scientific evidence of what works in education."
Similar question--what works...for?
I don't remember who it was I read that observed that every discipline or profession needs its specialized language, in part to reserve participation in the conversation to the initiates only. If this is any indication, then the least specialized groups seem to come up with the most awkward language of all.
Wednesday, April 13, 2011
The discussion of all that's new in education, especially technology, is exhausting—tiresome?—for the imperiousness. I read an article about so-called digital writing. The blurb/sub-title said something about 'innovative teachers' using technology to engage students more in writing.
The first sentence of the essay claims, "The nature of writing has shifted in recent years."
It has? I wish somebody had alerted me sooner, because I missed it.
"Digital writing," the article points out, "is writing created or read on a computer or other Internet-connected device."
I guess the article I was reading--which was created on a computer and posted on the Internet itself—was therefore this newly "re-natured" writing, even though it seemed strangely similar to "traditional" writing.
The article points out that "traditional writing formats, such as journaling, are frequently used for private reflection, digital writing is almost always meant for an audience."
Well, if you pick one specific traditional format to compare with digital, yes. But writing teachers have long taught writers to think about their audience.
--- Give me a break, just because college applications might be online doesn't mean the nature of writing has changed. You do the same tasks as before, but on a different platform. That's not a changed nature, but rather reflects that the fundamental purpose of writing is nearly immutable.
Seemingly innocuous claims like the changed nature of writing are dangerous. Such brazenness invokes the risk of diminishing the seriousness of our approach to issues. Digital realities are important. I'm sure, for instance, that the increasing digitization of reading and writing (and the greater variety and brevity of material that people read and write) affects the neural wiring of the brain, which then establishes/affects the prospects and constraints for further reading and writing activity. This is an important thing. But claiming that digitization has changed the nature of writing, when it hasn't, raises the prospect that we miss or diminish this other actually important situation.
Of course, brazen claims get attention. It's easier to get on TV or make a blogging name for yourself with attention-getting statements.
Imagine reframing the story. "Some teachers have used digital tools to try to stimulate more enthusiasm for writing. There are benefits as well as costs to doing this. Overall, some teachers have found it somewhat helpful."
Wouldn't get as much attention, even though it's more accurate.