Category: Education policy

Posted in Education policy, Educational Research, Teacher education, Teaching

Targeting Teachers

Posted on by David Labaree

In this piece, I explore a major problem I have with recent educational policy discourse — the way we have turned teachers from the heroes of the public school story to its villains.  If students are failing, we now hear, it is the fault of teachers.  This targeting of teachers employs a new form of educational firepower, value-added measures.  I show how this measure misses the mark by profoundly misunderstanding the nature of teaching as a professional practice, which has the following core characteristics:

  • Teaching is hard

    • Teachers depend on their students for the professional success

    • Students are conscripts in the classroom

    • Teachers need to develop a complex teacher persona in order to manage their relationship with students

    • Teachers need to carry out their practice  under conditions of high uncertainty

  • Teaching looks easy

    • It looks like an extension of child raising

    • It is widely familiar to anyone who has been a student

    • The knowledge and skills that teachers teach are ones that most competent adults have

    • Unlike any other professionals, teachers give away their expertise instead of renting it to the client, so success means your students no longer need you

  • Teachers are an easy target

    • Teachers are too visible to be inscrutable and too numerous to be elite

    • They don’t have the distance, obscurity, and selectivity of the high professions — so no one is willing to bow to their authority or yield to their expertise

This piece originally appeared in Dissent in 2011.  Here’s a link to the original.

 

Targeting Teachers

David F. Labaree

The mantra of the current school reform movement in the U.S. is that high quality teachers produce high achieving students.  As a result, we should hold teachers accountable for student outcomes, offering the most effective teachers bonus pay and shoving the least effective ones out the door.  Of course, in order to implement such a policy you need a valid and reliable measure of teacher quality, and the reformers have zeroed in on one such measure, which is known as the value-added approach.  According to this method, you calculate the effectiveness of individual teachers by the increase in test scores that students demonstrate after a year in their classroom.

Propelling this trend forward is a flood of research purporting to show that differences in teacher quality can lead to huge differences in the outcomes of schooling, both for students and for society.  For example, in a 2010 study for the National Bureau of Economic Research, Eric Hanushek argues that a strong teacher by the value-added measure (one standard deviation above the mean) might raise the lifetime earnings of a student by $20,000.  From this perspective, improving the quality of teaching promises to increase individual opportunity for the disadvantaged, which will reduce social inequality, and at the same time to increase human capital, which will promote economic growth and national competitiveness.  Sounds great.  Of course, this calculation is based on the assumption that test scores measure the economically useful knowledge of the future worker, which is far from obvious.  But arguments like these provide a big incentive to generate actionable data on who’s a good teacher and who’s not.

All of this makes the current effort to develop a simple and statistically sound measure for good teaching quite understandable.  But that doesn’t make it justifiable.  The problem with this approach is that teaching is in fact an extraordinarily complex and demanding form of professional practice, whose quality is impossible to capture accurately in a simple metric.  The push to develop such a metric threatens to reduce good teaching – and good education – to whatever produces higher scores on a standardized test.  As a result, the value-added measure of teacher quality may end up promoting both the wrong kind of teaching and the wrong kind of schooling.

In this article, I explore three major questions that arise from this development.  Why did the value-added measure of teaching emerge at this point in the history of American education?  What are the core characteristics of teaching as a professional practice that makes it so hard to perform effectively and so hard to measure accurately?  And under these circumstances, what are the likely consequences of using the value-added measure of teaching?

Roots of the Value-Added Measure of Teacher Quality

Until the last 30 years, Americans have been comfortable measuring the effectiveness of their schools by their broad social outcomes.  As long as graduates have tended to find jobs at a higher level than the jobs their parents had, then schools must be effectively promoting social opportunity.  And as long as the economy has been growing in size and productivity, then schools must be effectively producing human capital and spurring economic prosperity.  Under these circumstances, which lasted from the emergence of the common school in the early 19th century until the 1980s, there was little reason to seek out hard data about how much students were actually learning in school.

In the 1980s, however, this began to change with the emergence of a new kind of educational reform movement, which focused on raising the standards for student achievement.  Starting with the report A Nation at Risk in 1983, the idea was to set strict curriculum standards and enforce them with high-stakes tests in order to shore up the American economy with higher achievement.  Then came the No Child Left Behind law in 2002, which required schools to demonstrate that they were distributing educational and social opportunity more equally.

This radical shift to measuring learning outcomes in schooling came about in the late 20th century because of two converging changes in the politics of education: growing fiscal constraints and growing educational inequality.  For one thing, the rising cost of financing the expansion of schooling was beginning to run into severe fiscal limits.  By the end of the 20th century, state and local governments in the United States were spending about 30 percent of their total budgets on education, at an aggregate cost of about $400 billion.  Exacerbating this cost rise was the rise in educational level of the population.  From 1900 to 1975 the average education level of a 24 year-old rose from 8 years of elementary school to two years of college.  The problem is that the per-student cost of education is markedly higher as you move up the system, from elementary to secondary to college to graduate school.  As a result, schools at all levels came under pressure to demonstrate that they were producing learning outcomes that would justify the cost.

At the same time, a parallel concern emerged about radical differences in educational quality and outcomes for different groups in the population, sharply undercutting the hoary fiction that all high school or college diplomas were the same.  Middle class parents have long shown an acute awareness of this distinction and have had the means to pursue the best schools for their children.  Parents with more limited resources, however, have been stuck with their local schools, which were too often dirty, dangerous, and dysfunctional.

Under these circumstances, value-added measures of education have obvious value in potentially helping us zero in on the contribution that a school makes to the educational and social outcomes of its students.  The value-added approach seeks to take into account the educational achievement of students coming into a school or a classroom, in order to measure what added contribution the school or teacher makes to student achievement.  By controlling for the selection effect, this technique seeks to focus on the school’s socialization effect.

The Bill and Melinda Gates Foundation has plunged $355 million into the effort to measure teacher effectiveness.  Grounded in the value-added approach, this effort is using analysis of videos of teaching in individual classrooms to establish which teacher behaviors are most strongly associated with the highest value-added scores for students.  And the Brookings Institution published a study in 2010 that provided support for the value-added approach.  But, as Kevin Welner points out in the previous issue of Dissent, the evidence for the validity of the Gates value-added measures is weak.  In a recently published review, economist Jesse Rothstein from University of California, Berkeley performed an analysis of Gates data, which shows that 40 percent of teachers whose performance placed them in the bottom quartile using the value-added measure scored in the top half by an alternative measure of student achievement.  In short, the value-added approach is hardly the gold standard for measuring teacher effectiveness that its supporters claim it is.

Why This Measure Misses the Mark

So far I’ve been explaining where this new measure of teaching effectiveness came from and why it emerged when it did.  But I haven’t addressed why it fails to capture the elements of good teaching and why school reformers are so willing to deploy it anyway in formulating school policy.

The nature of American teaching arose from the structure of the American school system that was established before the Civil War, a system whose primary mission was political.  Founders wanted these schools to solve the core problem of a liberal democracy: to reconcile the self-interested pursuit of personal advantage demanded by a market economy with the civic commitment to community required by a republic.  In the second quarter of the nineteenth century, this problem was particularly acute, since the market was expanding at an extraordinary clip and the republic was young and fragile.  The idea was to create community schools that would instill republican principles in the young while also giving them a shared experience that might ameliorate growing class divisions.  To accommodate the huge influx of students, and to provide a setting in which students could be taught as a group and ranked by ability, they established the self-contained classroom, graded by age.  And to make sure that the school community was inclusive, they gradually made school attendance compulsory.

From this structure, emerge three core characteristics of teaching in the U.S.:    Teaching is hard; teaching looks easy; and teachers are an easy target.  Let me say a little about each.

Teaching Is Hard

In many ways, teaching is the most difficult of professions.  In other professions, professional success lies in the skills and knowledge of the practitioner and outcomes are relatively predictable.  Not so with teaching.  Why?

Teachers Depend on Students for their Success:  Teachers can only be successful if students choose to learn.  This is the core problem facing every teacher every day in every classroom.  Surgeons operate on clients who are unconscious; lawyers represent clients who remain mute; but teachers need to find a way to motivate students to learn the curriculum.  The teacher’s knowledge of the subject and skill at explaining this knowledge amount to nothing if students choose not to learn what they’re taught.  Student resistance to learning can come from a wide variety of sources.  Maybe students don’t like the subject or the teacher.  Maybe they don’t want to be in school at all.  Maybe they’re distracted by fear of a bully, hunger in the belly, or lust for the student in the next seat.  Maybe they’re bored to death.  The reasons for not learning are endless, and the teacher’s job is to find a way to understand these reasons and work around them, one student at a time.

What makes this even more difficult is that the teacher’s task extends beyond just getting students to learn the subject.  Teaching is a people-changing profession.  Education involves more than acquiring knowledge, since we ask it to take students and turn them into something else:  law abiding citizens, productive workers, ambitious achievers.  Changing people’s behavior and attitude and character and cultural yearnings is a lot harder than fixing a technical problem within the human body.  A surgeon can remove a diseased appendix, a physician can prescribe a pill to cure an infection.  But teaching is less like these highly esteemed and technically advanced arenas of medicine and more like the less prestigious and less certain practice of psychotherapy.  For therapists, the problem is getting patients to abandon a set of practices that they are unwilling or unable to manage on their own – like countering negative thoughts or calming anxiety.  Changing people in these nether realms of medicine is very difficult, but these practitioners do enjoy one advantage:  the patient approaches the therapist asking for help in making the change.  But this is not the case with teachers, where students enter class under duress.

Students Are Conscripts in the Classroom:  Students are in the classroom for a variety of reasons that often have nothing to do with wanting to learn.  They are compelled by strong pressures from their parents, the job market, cultural norms, and truant officers.  Also all of their friends are there, so what would they do if they stayed home?  Except for the rare case, however, one thing that does not bring them to the classroom is a burning desire to learn the formal curriculum.  As a result, unlike the clients of nearly all other professionals, they are not volunteers asking for a professional service but conscripts who have little reason to cooperate with, much less actively pursue, the process of learning that teachers are trying to facilitate.

The problem is that teachers don’t have much ability to impose their will on students in order to make them learn.  They have weak disciplinary tools, they are vastly outnumbered, and they have to deal with their students behind the doors of the self contained classroom, without the help of colleagues.  In the end, all that strict discipline can achieve is to maintain classroom order; inducing learning is another thing entirely.  The result is that teachers have to develop a complex mechanism for motivating their students to learn.

Teachers Need to Develop a Teaching Persona to Manage the Relationship with Their Students:  Teaching means finding a way to get students to want to learn the curriculum.  And this requires the teacher to develop a highly personalized and professionally essential teaching persona.  That persona needs to incorporate a judicious and delicately balanced mix of qualities.  You want students to like you, so they look forward to seeing you in class and want to please you.  You want them to fear you, so they studiously avoid getting on your bad side and can be stopped dead in their tracks with the dreaded “teacher look.”  You want them to find your enthusiasm for learning the subject matter infectious, so they can’t help getting caught up in the process and lured into learning.

Constructing such a persona is a complex task, which takes years of development.  It’s part of why the first years of teaching are so difficult, until the persona has fallen in place and becomes second nature.  The problem is that there is no standard way of doing this.  The persona has to be a combination of what the situation demands – the grade level, subject matter, cultural and personal characteristics of the students – and what the teacher can pull together from the pieces of his or her own character, personality, and interests.  It can’t be an obvious disguise, since students have an eye toward the fake and place high value on authenticity, and since it has to be maintained day in and day out over the years of a teaching career.  So the persona has to be a mix of who you are as a person and what you need and want to be as a teacher.

When it all comes together, it’s a marvel to behold.  In his book Small Victories, Samuel Freedman provides a vivid portrait of the teaching persona of a New York high school English teacher named Jessica Siegel.  She wears eye-catching clothing (one student asks, “Miss Siegel, do you water that dress?”), moves effortlessly between captivating and controlling her students, making wisecracks out of the corner of her mouth (“Gimme a break.”).  He calls this persona The Tough Cookie.  That works for her, but all successful teachers need to find their own right persona.  Think about it:  How can you measure this?  Measurement is particularly difficult because the criteria for defining a successful professional performance are up in the air.

Teachers Need to Carry Out Their Practices Under Conditions of High Uncertainty:  The problem is that there is no definitive code for effective teaching practice to parallel the kinds of codes that exist in other professions.  In general, professionals can defend themselves against malpractice by demonstrating that they were following standard professional practice.  The patient died but the physician was doing her job appropriately.  Teaching has no guide for optimal professional standards.  Instead there are rules about minimum criteria of acceptable behavior:  Don’t hit kids, show up for class.

One reason for the absence of such a code of professional practice for teachers is that, as I have been showing, the task of teaching involves the effort to manage a complex process of motivating learning in your students through the construction of a unique teaching persona.  Another is the problem of trying to identify what constitutes a definitive measure of teaching success.  The things that are easiest to measure are the most trivial:  number of right answers on a Friday quiz, a homework assignment, or – I might add – what’s represented in value-added test scores.  These things may show something about what information students retained at that point, but they don’t say anything about the long-term benefits of the class on these students.  Did the teacher make students better citizens, more productive workers, life-long learners, innovative entrepreneurs?  These are the outcomes we care about, but how can you measure them?  Even if you could find a way to measure such outcomes later in life, how could you trace back the impact that the student’s fourth grade teacher had on those outcomes?

This suggests another problem that raises the uncertainty of defining good teaching.  As a society, we are not of one mind about what individual and social ends we want schools to produce.  If we can’t agree on ends, how can we determine if a teacher was effective or not?  Effective at what?  One goal running through the history of American schooling is to create good citizens.  Another is to create productive workers.  A third is to provide individuals with social opportunity.  These goals lead schools in conflicting directions, and teachers can’t accomplish them all with the same methods.

One final form of uncertainty facing teachers is that we can’t even agree on who is the teacher’s client.  In some ways the client is the student, who is the object of education.  But students don’t contract with teachers to carry out their role, school boards do, as representatives of the community as a whole, which would make them the client.  But then there are the parents, a third constituency for teachers to deal with and try to please.  Are teachers the agents of the child, the society that sets up the school system, or the parents who send their children to school?  The answer is yes.

Teaching Looks Easy

So teaching is very hard, which makes it extraordinarily difficult to construct a good measure of effective teaching.  But at the same time, in the eyes of the public, teaching doesn’t look that hard at all.  And this makes us easy targets for anyone selling a simple mechanism for distinguishing the good teacher from the bad.

One reason teaching looks easy is that it seems to be an extension of child-raising.  You don’t need professional training to be a parent, which means that being a teacher doesn’t seem like a big thing.  Students coming into teacher education programs are often already imbued with this spirit.  I care for the kids, so I’ll be a good teacher.

Another reason it looks easy is that teaching is extremely familiar.  Every prospective teacher – every adult – has done a 12-year apprenticeship of observation in the elementary and secondary classroom.  We have watched teachers, up close and personal, during our formative years, and nothing about the practice of teaching seems obscure or complicated.  You keep order, give out and collect assignments, talk, test, and take the summer off.  No big deal.  Missing from this observation, of course, is all the thinking and planning that goes into the process that students experience in the classroom, much less the laborious construction of the teaching persona.

A third thing that makes teaching look easy is that the knowledge and skills teachers convey are the knowledge and skills that all competent adults have.  This isn’t the kind of complex and obscure knowledge you find in medical texts or law books; it’s ordinary knowledge that doesn’t seem to require an advanced degree of skill for the practitioner.  Of course, missing from this kind of understanding of teaching is an acknowledgement of the kind of skill required to teach these subjects and motivate students to learn these subjects, which is not obvious at all.  But the impression of ordinariness is hard for teaching to shake.

A factor that enhances this problem is that, unlike other professionals, teachers give away their expertise.  One test of a successful teacher is that the student no longer needs her.  Good teachers make themselves dispensable.  In contrast, other professions don’t give away their expertise; they rent it by the hour.  You have to keep going back to the doctor, lawyer, accountant, and even pharmacist.  In these arenas, you’re never on your own.  But teachers are supposed to launch you into adult life and then disappear into the background.  As a result, it is easy for adults to forget how hard it was for them to acquire the skills and knowledge they now have and therefore easy to discount the critical role that teachers played in getting them to their current state.

Teachers Are an Easy Target

It’s tough being in a profession that is extraordinarily difficult to practice effectively and that other people consider a walk on the beach.  As a group, teachers are too visible to be inscrutable and too numerous to be elite.  They don’t have the distance, obscurity, and selectivity of the high professions.  As a result, no one is willing to bow to their authority or yield to their expertise.  Teachers, school administrators, and education professors have all had the experience of sitting next to someone on an airplane or at a dinner party who proceeds to tell them what the problem with schools is and also what the cure is.  Everyone is an expert on education except the educator.

One consequence of this is that teachers become an easy target for school reformers.  This follows from the nature of teaching as a practice, as I’ve been describing here, and also from the nature of school reform as a practice.  The history of school reform in the United States is a history of efforts to change the education of Other People’s Children.  The schools that reformers’ own children attend tend to be seen as pretty good; the problem is with the schooling of Others.  It’s those kids who need more structure, higher standards, more incentives, and more coercion in order to bring their learning up to a useful level.  They are the ones who are dragging down our cities and holding back our economic growth.  And public school teachers are the keepers of Other People’s Children.  Since we don’t think those children are getting the kind of schooling they need, then teachers must be a major part of the problem.  As a result, these teachers too are seen as needing more structure, higher standards, more incentives, and more coercion in order to bring their teaching up to a socially useful level.

We tell ourselves that we’re paying more than we can afford for schools that don’t work, so we have to intervene.  The value-added measure of teacher performance is ideally suited to this task.  It’s needed because, in the eyes of reformers, teachers are not sufficiently professional, competent, or reliable to be granted the autonomy of a real profession.  And what will be the consequences?

As in medicine, the first rule of school reform should be: At least, do no harm.  But the value-added intervention violates this rule, driven by the arrogance of reformers who are convinced that teaching is a simple process of delivering content and that learning is just a matter of exerting the effort to acquire this content.  That approach is likely to increase test scores, simply by pressuring teachers to teach to the test.  But my concern is that in the process it’s also likely to interfere with teachers’ ability to lure students into learning.  This requires them to develop and nurture an effective teacher persona, so they can in turn develop and nurture in students the motivation to learn and to continue learning over a lifetime.

As usual, the results of this reform are likely to be skewed by social class.  Schools for the disadvantaged are going to be under great pressure to teach to the test and raise scores on core skills, while schools for the advantaged will be free to pursue a much richer curriculum.  If your children, unlike Others’, are not At Risk, then the schools they attend will not need to be obsessed with drilling to meet minimum standards.  Teachers in these schools will be able to lead their classes in exploring a variety of subjects, experiences, and issues that will be excluded from the classrooms farther down the social scale.  In the effort to raise standards and close the achievement gap, we will creating just another form of educational distinction to divide the top from the bottom.

Posted in Education policy, Educational Research, Uncategorized

The Dysfunctional Pursuit of Relevance in Educational Research

Posted on by David Labaree

In this paper, I explore the issue of relevance in educational research.  I argue that the chronic efforts by researchers to pursue relevance is counterproductive.  Paradoxically, trying to make research more relevant actually makes it less so.  Drawing on an analysis by Mie Augier and Jim March, I show that this is the result of two factors.  One is that there is a profound ambiguity about what education is supposed to accomplish.  What’s relevant for one goal may be irrelevant for another.  Another factor is that the stress on relevance creates a kind of myopia. The researcher is focusing tightly on a particular set of conditions in time and place, which will likely change by the time the study is published.  Better to step back and see the larger picture in order to figure out what this case is a case of.

This paper originally appeared in Educational Researcher in 2008.  Here’s a link to the original text.

The Dysfunctional Pursuit of Relevance in Educational Research

David F. Labaree

In the title of her paper, Jacquelien A. Bulterman-Bos (2008) asks, “Will a Clinical Approach Make Educational Research More Relevant for Practice?” and by the end she comes to the answer, “Yes.”  Overall this is an engaging effort to sort out the nature of educational research and understand its relationship to the practice of teaching.  During the course of this discussion, the author draws on my analysis of the transition that teachers undergo when they enter doctoral programs on the road to becoming researchers – a shift in worldview from the normative to the analytical, from the personal to the intellectual, from the particular to the universal, and from the experiential to the theoretical (Labaree, 2003).  She argues that this depiction of the differences between teaching and doing research is only useful up to a point, because it is grounded in a discredited Cartesian conception of the split between body and mind.  Drawing on Michael Polanyi, she argues that knowledge is not full bodied and useful unless it remains connected to the real world of education, where knowledge necessarily retains within it elements that are normative, personal, particularistic, and experiential.  Therefore, she asserts, educational research needs to be grounded in teaching practice if it is going to be able to represent the context of practice effectively.  This means that educational researchers need to spend years as teachers before becoming researchers, and they need to remain active as classroom teachers during their time doing research.  Such a clinical approach to educational research will produce a form of knowledge about education that is both more valid as a representation of education and more useful to practitioners.

Bulterman-Bos raises the kinds of issues about the aims and meanings of research that scholars in any field should be discussing, and such a discussion is particularly pertinent in a professional field like education, where policymakers, practitioners, funding agencies, and citizens routinely ask how research can help improve the profession.  I welcome her effort to clarify the issues surrounding the relevance of educational research, and I appreciate the opportunity to join in this discussion; but I find that I profoundly disagree with her analysis.  My disagreement operates at three levels of abstraction, starting with a quibble about her interpretation of my own earlier argument about the differences in how teachers and researchers view the educational enterprise and then moving on to much more basic concerns about the nature of scholarly research and the meaning of relevance.

First, although her paper in general represents the argument in my paper accurately, it does introduce a subtle distortion of my account of the teacher-researcher split.  In my paper I acknowledge that, by laying out the polar differences in the orientations of teacher and researcher so starkly, I risk overstating the difference between the two modes of work.  I note that researchers in their own practice also demonstrate elements of the normative, personal, particularistic, and experiential; likewise teachers show elements of the contrasting orientations.  The difference, I argue, is not the function of a mind-body dichotomy but a matter of emphasis, the result of a division of educational labor structured by the institutional settings, occupational constraints, daily work demands, and professional incentives of each realm of practice.  Teachers are primarily engaged in a practice of social improvement, grounded in personal relationships with particular students embedded in time and place, and the professional knowledge they build is largely an accumulation of clinical experiences.  Educational researchers are primarily engaged in a practice of social analysis, grounded in intellectual conceptions of education generalized across contexts, and the professional knowledge they build is largely a web of theories.  The different conditions of work lead to different modes of professional practice and different ways of thinking about education.

This leads to a second broader point, that the way to deal with these differences in orientation is not to combine the two in a single role – clinical researcher – which perfectly balances these elements, but to acknowledge and honor the different zones of expertise and to promote a fruitful dialogue between practitioners in the two zones.  It seems neither practical nor necessary for all researchers to split their time between school teaching and educational research in order to establish the power and credibility of the research knowledge they produce.  A differential allocation of functions between teachers and researchers seems fruitful for both professions, as long as the barrier is relatively low and the conversation across the barrier is ongoing.  Universities are not eager to pay scholars to teach school, and school districts are not eager to pay teachers to do research, so it is hard to see how such a hybrid occupational role can become institutionalized.  And the skills involved in being an expert researcher or teacher are so strikingly different that it would be difficult for individuals to achieve mastery in both at the same time.  Each requires immersion in a particular institutional setting, fluency in a distinctive professional culture, and full engagement in a complex set of professional practices.

According to Bulterman-Bos, these differences in perspective between researchers and teachers are highly dysfunctional, leading to invalid research and ineffective teaching; but I see these differences as carrying great potential value.  To teachers – immersed in a web of pedagogical goals, social contexts, and instructional relationships – research can offer a way to gain perspective on their realm of practice, holding up a theoretical mirror that allows them to see what is unique and what is commonplace in their classroom setting.  To researchers – afloat in the intellectualized and decontextualized realm of educational theory – the classroom can offer a way to gain grounding in the personal and particular world of teaching and learning in schools, providing professional problems to spur theory development and providing clinical settings for testing theory.  Both stand to gain from the interaction, since each side provides what the other is lacking.  The answer, I suggest, is not to resolve this tension by merging the two roles but to take advantage of the tension by using it to enrich both modes of professional practice.

I want to advance this argument another notch by making a third, more fundamental point:  It is counterproductive to press educational research – or, for that matter, any other form of research – to be relevant.  One problem is that relevance is a tricky quality to define, since it is easier to recognize in retrospect than in prospect.  A related problem is that earnest efforts to make research more relevant can paradoxically make it useless or even harmful, by focusing on short term results that are narrowly measured instead of on consequences with a longer horizon and broader scope.

But to argue against the press for relevance is not to say that educational research should be irrelevant.  Research in general draws inspiration from real world problems, and this is particularly true in professional schools, where scholars feel the need to study issues that arise from problems of practice in their professional arena.  In his book exploring the issue of research relevance, Donald Stokes (1997) called this sector of research activity “Pasteur’s quadrant.”  He categorizes research according to the degree that its aim is to pursue fundamental understanding or to respond to pressing problems, and by this metric he locates Niels Bohr in the first category (pure basic research), Thomas Edison in the second (pure applied research), and Louis Pasteur in both (use-inspired basic research).  I argue that scholarly research justifies itself primarily by its contribution to theory, sometimes inspired by immediate social needs (like Pasteur) and sometimes not (like Bohr), and this applies to a professional field like education as much as to a scholarly discipline.  If we as educational researchers fail to contribute to theory with our research, then we are less scholars than engineers or product developers.  Theory development frequently leads to product development, as the relationship between universities and the technology industry has demonstrated so dramatically in the last half century, but the distinctive value of scholarly research dissipates quickly when it segues from being use-inspired to use-driven.  And this is what happens with the press for relevance.

Mie Augier and James G. March (2007) have written a persuasive account of how the drive to make research relevant can render it less so.  Their focus is on the research produced and education provided by business schools, but their arguments apply with equal validity to the field of education.  In both domains, there is a strong professional constituency asking professional schools to prove their usefulness by solving problems and serving the needs of practitioners.  The problem is that this urge to be useful often turns counterproductive.  Augier and March identify two key factors that undermine the relevance of research trying too hard to be relevant: ambiguity and myopia.

Ambiguity comes from the difficulty in trying to define what constitutes relevance for research.  Presumably, educational research is relevant if it has some kind of clear connection to issues and problems in the field of educational practice, particularly if it promises to be useful to practitioners who are trying to deal with these issues and resolve these problems.  But this begs the question, useful to whom and for what?  Think of the wide array of actors involved in education:  teachers, students, principals, and parents; test makers, textbook publishers, and educational technology producers; curriculum developers, superintendents, and school board members; policymakers, politicians, and educational bureaucrats; teacher educators and education school deans.  What might make research useful for some of these might at the same time make it useless for others.  Relevance is in the eye of the beholder.  In addition, research may be useful for helping to accomplish some educational ends but not others.  Studies that seem relevant to people who are trying to raise test scores may not be at all relevant for those who are trying to improve critical thinking, enhance civic commitment, increase skills in mathematical analysis, promote gender equity, reduce the racial achievement gap, increase graduation rates, or enhance human capital production.  These studies may not even seem helpful to people trying to raise student scores on other tests.  In fact, what is helpful in attaining one goal may be harmful in attaining another, which, for example, is the argument made by proponents of critical thinking about the effect of efforts to raise test scores.  Almost any bit of educational research is likely to be seen as more or less useful for some actor concerned about some education-related goal, while simultaneously being seen as useless or harmful by most actors for most purposes.  Under these circumstances, it is trivial to call research relevant or irrelevant, since it all depends on the peculiar mix of actors and goals.

But the problem of the inherent ambiguity of relevant research feeds into the more fundamental problem of its genetic predisposition toward myopia.  Relevance is not only a function of person and purpose but also of place and time.  As Bulterman-Bos and I have both argued, teaching and learning in schools is highly particularized and contextualized.  This makes the relevance of research hard to establish across contexts.  Studies that may be seen as useful in teaching and learning for one student – or subject matter or ethnic group or classroom or grade level or school or school district or state or country – may not be seen as useful in other settings.  Likewise, what makes research useful at one point in time may make it useless or misleading at another.  The knowledge may be so time sensitive that its usefulness expires quickly.  And knowledge that is helpful in meeting an educational goal in the present (say, to improve engagement in a science lesson) may undermine a goal whose accomplishment cannot be measured until decades later (say, to improve science understanding in the workforce).

Augier and March call this tendency myopia in order to call attention to a potentially pathological consequence of the effort to make research relevant:  it may lead to educational knowledge that is short-sighted.  When educational researchers seek to make their work relevant, they feel the need to tailor their work to the demands of educational practice in the present time and the local place (or the location of the intended client-consumer).  The problem is that this work, even if it is helpful in that particular context is not likely to be useful in the conditions of educational practice that exist a few miles away or a few months in the future.  Conversely, research that seems like an abstract exercise in theory building at one time and place may become highly relevant for practical purposes that were unforeseeable at the point when the work was done.  Bohr’s work on quantum mechanics is a good case in point.

In this sense, then, applied research may grow stale quickly while basic research may age well.  Scholarly work that neither arises from a quest for relevance nor demonstrates any particular utility at the time it is carried out may turn out to be highly useful at a later time and in a different place.  Research relevance, therefore, is not only hard to define, but the active pursuit of it may produce educational knowledge that is irrelevant.

References

Augier, Mie & March, James G.  (2007).  The pursuit of relevance in management education. California Management Review, 49:3 (Spring), 129-146.

Bulterman-Bos, Jacquelien A. (2008).  Will a clinical approach make educational research more relevant for practice?  Educational Researcher.

Labaree, David F. (2003). The peculiar problems of preparing and becoming educational researchers. Educational Researcher, 32:4 (May), 13-22.

Stokes, Donald E.  (1997). Pasteur’s quadrant: Basic science and technological innovation. Washington DC: Brookings Institution Press.

Posted in Education policy, Educational Research, History of education

Do No Harm

Posted on by David Labaree

This is a piece I wrote about the harm that educational research has inflicted over the years.  Given a track record of making things worse for school and society, educational researchers would do well to heed the wisdom in the Hippocratic Oath.  If our work often fails to make things better, we should at least strive to do no harm.

The paper first appeared in Teacher Education and Practice in 2011.  Here’s a link to the original.

 

Do No Harm

David F. Labaree

Education is a field of dreams and so is educational research.  As educators, we dream of schools that can improve the lives of students, solve social problems, and enrich the quality of life; and as educational researchers, we dream that our studies will enhance the effectiveness of schools in achieving these worthy goals.  Both fields draw recruits who see the possibilities of education as a force for doing good, and that turns out to be a problem, because the history of both fields shows that the chances for doing real harm are substantial.  Over the years, research on teaching and teacher education – the topic of the discussion in this special issue – has caused a lot of damage to teaching and learning and learning-to-teach in schools.  So I suggest a good principle to adopt when considering the role of research in teacher education is a version of the Hippocratic Oath:  First do no harm.

The history of educational research in the United States in the twentieth century supports a pessimistic assessment of the field’s impact on American school and society.  There was Edward L. Thorndike, whose work emphasized the importance of differentiating the curriculum in order to provide the skills and knowledge that students would later need in playing sharply different roles in a stratified workforce.  There was David Snedden, who labored tirelessly to promote narrowly vocational training for that large group of students who would end up serving in what he called “the rank and file.”  There were the kingpins of educational testing, such as Lewis Terman, who developed instruments that allowed educators to measure student ability and student learning, which in turn helped determine which track students should occupy and what role they should play in later life.  Put together, these kinds of enormously productive educational researchers helped build a system of schooling that emphasized sorting over learning and promoted a vision of teaching that emphasized the delivery of curriculum over the engagement of students.  They laid the foundation for the current machinery of curriculum standards and high-stakes testing that has turned American teaching into a machinery for raising test scores.

Of course, these educational researchers usually did not intend to do harm.  (Snedden is the exception here, a man who was on a mission to dumb down schooling for the lower classes.)  For the most part, they saw making curriculum more scientific and intelligence testing more accurate as ways to allow individuals with merit to escape from the clutches of their social origins.  Like most educational researchers, they were optimists about the possible impact of their work.  But their examples should serve as a cautionary tale for researchers who see their work as an unmitigated exercise in human improvement.

One factor in particular tends to bend the work of researchers toward the dark side of the force, and that is research funding.  Very few government agencies and foundations are eager to support basic research in education.  Instead, funding aligns with the latest educational policy objectives, and to get funded researchers need to demonstrate that their work will in some manner serve these objectives.  That is not to say that the researchers necessarily support these policy missions, but in order to win the grant they do have to harness their work, at least rhetorically, to the aims that motivate the request for proposals.  In the current global policy climate, that means the work needs to address issues around accountability and standards and improving test scores.  If you cannot spin your work in this direction, you will have trouble getting funded.

Another factor that interferes with the educational researcher’s desire to do good for teachers and teacher educators is the need to confront an educational version of Gresham’s Law:  Bad research tends to displace good.  The best research is complex, and this puts the researcher at a competitive disadvantage, since policymakers and teacher educators prefer results that are definitive and easy to understand.  The most sophisticated work we produce tends to show an educational reality that has a complex array of elements interacting within a fiendishly complex organizational structure, which means that research findings have be carefully qualified to the point where it is nearly impossible to say with clarity that a particular form of educational practice is effective or ineffective.  Instead, we have to report that it all depends.  In addition, in order to understand the research findings in any depth, you need to be able to sort through issues of design, methodology, and validity that are only accessible to experts in the field.

Meanwhile, there is a vast array of research available to policymakers and practitioners that supports clear answers to educational problems and does so in a manner that is easy for the layperson to comprehend.  This kind of work comes from two kinds of groups: think tanks, and entrepreneurial organizations for the delivery of education.  Think tanks removes a key element of complexity from the research process by deciding in advance what the politically desirable policy is and then conducting studies that provide clear support for that policy.  In the U.S. there are also a variety of non-governmental organizations that are active in promoting and delivering a particular brand of educational service, such as Teach For America (TFA, with its alternative to traditional teacher preparation) and the Knowledge Is Power Program (KIPP, with its alternative approach to running schools in low income neighborhoods).  These organizations commission research that conveniently demonstrates the effectiveness of what they do.  And both types of research producers are particularly effective at marketing their findings to the relevant actors in the policy and education communities.

University based educational research cannot compete with these other producers in clarity and understandability, but they can undercut the impact of this work a bit by doing what university researchers have always been good at.  We have an advantage in being the only group without a dog in the policy hunt, which allows us to perform credible fundamental research about how schools work, how teaching and learning happens, and how teachers learn to teach.  Work like this can help show how simplistic and politically biased these other research products really are.  And it won’t do much harm.