Posted in Course Syllabus, Schooling, Theory

Course: School — What Is It Good For?

This post is the syllabus of a course I taught for years at the Stanford Graduate School of Education.  It’s called School — What Is It Good For? I’ve copied the syllabus below, to give you an idea of what it’s all about.  The aim is to provide a guided exploration of alternative theories of the social functions that schools serve, especially in American society.  Along the way it tries to lay out a framework for thinking about school theories in general.

The best way to use the syllabus is to download the syllabus here in the form of a Word document.  This document includes embedded links to:

  • most of the readings for the class (including articles and out-of-print books)

  • tips for approaching each week’s assigned readings

  • my notes for shaping the discussion in each class

Please feel free to use this course any way you would like.  You can take it as a self-guided class, either by yourself or as part of a group.  You can draw on it to teach your own course.  Or you can just use it as a prompt to explore some interesting readings in theories of schooling.  Enjoy.

School – What Is It Good For?

 David Labaree

Web: http://www.stanford.edu/~dlabaree/

Twitter: @Dlabaree

Blog: https://davidlabaree.com/

Course Description

This course seeks to answer the question in its title:  School – What Is It Good For?  Unlike the song from the 70s that inspired the course’s title (“War – What Is It Good For?”), the answer to this question is not necessarily “absolutely nothing,” although that will remain a distinct possibility throughout the class.  In practice, the course will focus on a series of books and a few articles in which authors try to establish claims about the particular purposes, functions, impacts, and social roles of schooling – especially in relation to American society.  The class draws in part from the issues that frame my book, Someone Has to Fail: The Zero-Sum Game of Public Schooling.

The course addresses two broad domains of interest to education students:

It explores the big questions that underlie Educational Policy.

It explores a wide range of approaches to Educational Theory.

Americans have a long history of pinning their hopes on education as the way to realize compelling social ideals and solve challenging social problems.  We want schools to promote civic virtue, economic productivity, and social mobility; to alleviate inequalities in race, class, and gender; to improve health, reduce crime, and protect the environment.  So we assign these social missions to schools, and educators gamely accept responsibility for carrying them out.  When the school system inevitably fall far short of these goals, we initiate a wave of school reform to realign the institution with its social goals and ramp up its effectiveness in attaining them.  In this class, we explore the social mixed aims and mixed outcomes of America’s puzzling, estimable, gargantuan, and ineffectual system of public education.

At its heart, this is a story grounded in paradox.  Schooling is perhaps the greatest institutional success in American history.  It grew from a modest and marginal position in the 18th century to the very center of American life in the 21st, where it consumes an enormous share of the time and treasure of both government and citizenry.  Key to its institutional success has been its ability to embrace and embody the social goals that have been imposed upon it.  Yet, in spite of continually recurring efforts, schooling in the U.S. has been remarkably unsuccessful at realizing these goals in the social outcomes of education.  In spite of everything, however, we keep pushing new tasks onto our schools, less as a rational investment in achieving social results than as a matter of faith.  The readings in this course explore the kinds of goals, ideals, problem-solving roles, and visions of the good society that we have imposed on schooling over the years.  They also explore the extent to which schools have been able to realize these aims, and if not, what kinds of effects they have exerted on American life.

Consider the following Policy Visions of what schools should do and Educational Theories about what they can and can’t do, with course readings that will explore each of these issues:

Produce citizens for a democracy:  Gutmann

Create human capital and promote economic growth:  Goldin & Katz, Kristof

Teach core values in American society:  Dreeben

Reproduce an unequal social structure:  Bowles & Gintis

Serve the interests of educational consumers:  Collins

Promote social mobility and social equality:  Boudon, Hertz, Goldin & Katz, Kristof

Promote disciplinary power:  Foucault

Teach core values within a religious community: Peshkin

Promote a mix of social access and social advantage:  Labaree

Readings

            Assigned Books:  We will read the following eight books.  Bowles & Gintis, Collins, and Dreeben are not in print and are through links to a Google drive (marked with an *).

Gutmann, Amy.  (1987).  Democratic education.  Princeton: Princeton University Press.

Goldin, Claudia & Katz, Lawrence F. (2008). The race between education and technology. Cambridge: Belknap Press of Harvard University Press.

*Dreeben, Robert. (1968). On what is learned in school. Reading, MA: Addison-Wesley.  Part 1, part 2, part 3.

*Bowles, Samuel & Gintis, Herbert. (1976). Schooling in capitalist America. New York: Basic Books. Chapters 1-2, 3-4, 5-6, 7-9, 10-11.

*Collins, Randall. (1979). The credential society: An historical sociology of education and stratification. New York: Academic Press.  Chapters 1-2, 3-5, 6, 7.

Foucault, Michel. (1977). Discipline and punish: The birth of the prison (trans. by Alan Sheridan). New York: Pantheon.

Peshkin, Alan.  (1986).  God’s choice, Chicago: University of Chicago Press.

Labaree, David F. (2010). Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press.

* = not in print; available through link to a Google drive

            Assigned Articles:  We will also read a small number of articles and book chapters, which will be available to students through links to a Google drive.

Course Outline

             Below are the topics we will cover, week by week, with the readings for each week.  Just click on the assigned reading to link to the document on Google drive.  For every week you can click on a link to get tips for doing that week’s readings.  In addition, you can link to my notes for that week’s class.

Week 1:  Introduction to Course

Tips for week 1 readings

How to read efficiently: skimming

*Labaree, David F. (2010). What schools can’t do. Zeitschrift für Pädagogische Historiographie, 16:1, 12-18.

*Kristof, Nicholas.  (2009).  Democrats and Schools. New York Times, October 15.

Class notes for week 1

Week 2:  Schools Promote Citizenship

Tips for week 2 readings

Gutmann, Amy.  (1987).  Democratic education.  Princeton: Princeton University Press.

Labaree, David F. (2010). Chapter 2 – Founding the American school system.  In Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press.

Class notes for week 2

Week 3:  Schools Promote Human Capital Production

Tips for week 3 readings

Goldin, Claudia & Katz, Lawrence F. (2008). The race between education and technology. Cambridge: Belknap Press of Harvard University Press.

Labaree, David F. (2010). Chapter 7 – The limits of school learning.  In Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press.

Class notes for week 3

Week 4:  Schools Teach Core Values of Society

Tips for week 4 readings

*Dreeben, Robert. (1968). On what is learned in school. Reading, MA: Addison-Wesley.  Part 1, part 2, part 3.

Labaree, David F. (2010). Chapter 1 – From citizens to consumers: A history of reform goals.  In Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press.

*Labaree, David F. (2013). Schooling in the United States: Historical analysis. In Denis C. Phillips (Ed.), Encyclopedia of Educational Theory and Philosophy. New York: Sage Publications.

Class notes for week 4

Week 5:  Schools Promote the Reproduction of an Unequal Social Structure

Tips for week 5 readings

*Bowles, Samuel & Gintis, Herbert. (1976). Schooling in capitalist America. New York: Basic Books.  Chapters 1-2, 3-4, 5-6, 7-9, 10-11.

Labaree, David F. (2010). Chapter 3 – The progressive effort to reshape the school system.  In Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press.

Class notes for week 5

Week 6:  Schools Promote the Positional Interests of Educational Consumers

Tips for week 6 readings

*Collins, Randall. (1979). The credential society: An historical sociology of education and stratification. New York: Academic Press.  Chapters 1-2, 3-5, 6, 7.

Labaree, David F. (2010). Chapter 8 – Living with the school syndrome.  In Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press.

Class notes for week 6

Week 7:  Schools Promote Social Mobility and Social Equality

Tips for week 7 readings

*Boudon, Raymond. (1986). Education, mobility, and sociological theory. In John G. Richardson (ed.), Handbook of theory and research for the sociology of education (pp. 261-274). New York: Greenwood.

*Hertz, Tom. (2006). Understanding mobility in America. Washington, DC: Center for American Progress.

Labaree, David F. (2010). Chapter 6 – Failing to solve social problems.  In Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press.

Class notes for week 7

Week 8:  Schools Promote Disciplinary Power

Tips for week 8 readings

Foucault, Michel. (1977). Discipline and punish: The birth of the prison (trans. by Alan Sheridan). New York: Pantheon.

Labaree, David F. (2010). Chapter 5 – Classroom resistance to school reform.  In Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press.

Class notes for week 8

Week 9:  Schools Teach Core Values of a Religious Community

Tips for week 9 readings

Peshkin, Alan.  (1986).  God’s choice, Chicago: University of Chicago Press.

Class notes for week 9

Week 10:  Schools Promote Both Social Access and Social Advantage

Tips for week 10 readings

Labaree, David F. (2010).  Someone has to fail: The zero-sum game of public schooling. Cambridge: Harvard University Press, remaining chapters.

Class notes for week 10

Guidelines for Critical Reading

As a critical reader of a particular text (a book, article, speech, proposal), you need to use the following questions as a framework to guide you as you read:

  1. What’s the point? This is the analysis issue: what is the author’s angle?
  2. Who says? This is the validity issue: On what (data, literature) are the claims based?
  3. What’s new? This is the value-added issue: What does the author contribute that we don’t already know?
  4. Who cares? This is the significance issue, the most important issue of all, the one that subsumes all the others: Is this work worth doing?  Is the text worth reading?  Does it contribute something important?

If this is the way critical readers are going to approach a text, then as an analytical writer you need to guide readers toward the desired answers to each of these questions.

Guidelines for Analytical Writing

             In writing papers for this (or any) course, keep in mind the following points.  They apply in particular to the final paper or take-home exam for this class.   Many of the same concerns apply to critical reaction papers as well, but these short papers can be more informal than the final paper.

  1. Pick an important issue: Make sure that your analysis meets the “so what” test.  Why should anyone care about this topic, anyway?  Pick an issue or issues that matters and that you really care about.
  2. Keep focused: Don’t lose track of the point you are trying to make and make sure the reader knows where you are heading and why.
  3. Aim for clarity: Don’t assume that the reader knows what you’re talking about; it’s your job to make your points clearly.  In part this means keeping focused and avoiding distracting clutter.  But in part it means that you need to make more than elliptical references to concepts and sources or to professional experience.  When referring to readings (from the course or elsewhere), explain who said what and why this point is pertinent to the issue at hand.  When drawing on your own experiences or observations, set the context so the reader can understand what you mean.  Proceed as though you were writing for an educated person who is neither a member of this class nor a professional colleague, someone who has not read the material you are referring to.
  4. Provide analysis: A good paper is more than a catalogue of facts, concepts, experiences, or references; it is more than a description of the content of a set of readings; it is more than an expression of your educational values or an announcement of your prescription for what ails education.  A good paper is a logical and coherent analysis of the issues raised within your chosen area of focus.  This means that your paper should aim to explain rather than describe.  If you give examples, be sure to tell the reader what they mean in the context of your analysis.  Make sure the reader understands the connection between the various points in your paper.
  5. Provide depth, insight, and connections: The best papers are ones that go beyond making obvious points, superficial comparisons, and simplistic assertions.  They dig below the surface of the issue at hand, demonstrating a deeper level of understanding and an ability to make interesting connections.
  6. Support your analysis with evidence: You need to do more than simply state your ideas, however informed and useful these may be.  You also need to provide evidence that reassures the reader that you know what you are talking about, thus providing a foundation for your argument.  Evidence comes in part from the academic literature, whether encountered in this course or elsewhere.  Evidence can also come from your own experience.  Remember that you are trying to accomplish two things with the use of evidence.  First, you are saying that it is not just you making this assertion but that authoritative sources and solid evidence back you up.  Second, you are supplying a degree of specificity and detail, which helps to flesh out an otherwise skeletal argument.
  7. Draw on course materials (this applies primarily to reaction papers, not the final paper). Your paper should give evidence that you are taking this course.  You do not need to agree with any of the readings or presentations, but your paper should show you have considered the course materials thoughtfully.
  8. Recognize complexity and acknowledge multiple viewpoints. The issues in the history of American education are not simple, and your paper should not propose simple solutions to complex problems. It should not reduce issues to either/or, black/white, good/bad.  Your paper should give evidence that you understand and appreciate more than one perspective on an issue.  This does not mean you should be wishy-washy.  Instead, you should aim to make a clear point by showing that you have considered alternate views.
  9. Challenge assumptions. The paper should show that you have learned something by doing this paper. There should be evidence that you have been open to changing your mind.
  10. Do not overuse quotation: In a short paper, long quotations (more than a sentence or two in length) are generally not appropriate.  Even in longer papers, quotations should be used sparingly unless they constitute a primary form of data for your analysis.  In general, your paper is more effective if written primarily in your own words, using ideas from the literature but framing them in your own way in order to serve your own analytical purposes.  However, selective use of quotations can be very useful as a way of capturing the author’s tone or conveying a particularly aptly phrased point.
  11. Cite your sources: You need to identify for the reader where particular ideas or examples come from.  This can be done through in-text citation:  Give the author’s last name, publication year, and (in the case of quotations) page number in parentheses at the end of the sentence or paragraph where the idea is presented — e.g., (Ravitch, 2000, p. 22); provide the full citations in a list of references at the end of the paper.  You can also identify sources with footnotes or endnotes:  Give the full citation for the first reference to a text and a short citation for subsequent citations to the same text.  (For critical reaction papers, you only need to give the short cite for items from the course reading; other sources require full citations.)  Note that citing a source is not sufficient to fulfill the requirement to provide evidence for your argument.  As spelled out in #6 above, you need to transmit to the reader some of the substance of what appears in the source cited, so the reader can understand the connection with the point you are making and can have some meat to chew on.  The best analytical writing provides a real feel for the material and not just a list of assertions and citations.  Depth, insight, and connections count for more than a superficial collection of glancing references.  In other words, don’t just mention an array of sources without drawing substantive points and examples from these sources; and don’t draw on ideas from such sources without identifying the ones you used.
  12. Take care in the quality of your prose: A paper that is written in a clear and effective style makes a more convincing argument than one written in a murky manner, even when both writers start with the same basic understanding of the issues.  However, writing that is confusing usually signals confusion in a person’s thinking.  After all, one key purpose of writing is to put down your ideas in a way that permits you and others to reflect on them critically, to see if they stand up to analysis.  So you should take the time to reflect on your own ideas on paper and revise them as needed.  You may want to take advantage of the opportunity in this course to submit a draft of the final paper, revise it in light of comments, and then resubmit the revised version.  This, after all, is the way writers normally proceed.  Outside of the artificial world of the classroom, writers never turn in their first draft as their final statement on a subject.

 

Posted in Capitalism, Global History, Higher Education, History, State Formation, Theory

Escape from Rome: How the Loss of Empire Spurred the Rise of Modernity — and What this Suggests about US Higher Ed

This post is a brief commentary on historian Walter Scheidel’s latest book, Escape from Rome.  It’s a stunningly original analysis of a topic that has long fascinated scholars like me:  How did Europe come to create the modern world?  His answer is this:  Europe became the cauldron of modernity and the dominant power in the world because of the collapse of the Roman empire — coupled with the fact that no other power was able to replace it for the next millennium.  The secret of European success was the absence of central control.  This is what led to the extraordinary inventions that characterized modernity — in technology, energy, war, finance, governance, science, and economy.

Below I lay out central elements of his argument, providing a series of salient quotes from the text to flesh out the story.  In the last few years I’ve come to read books exclusively on Kindle and Epub, which allows me to copy passages that catch my interest into Evernote for future reference. So that’s were these quotes come from and why they don’t include page numbers.

At the end, I connect Scheidel’s analysis with my own take on the peculiar history of US higher education, as spelled out in my book A Perfect Mess.  My argument parallels his, showing how the US system arose in the absence of a strong state and dominant church, which fostered creative competition among colleges for students and money.  Out of this unpromising mess of institutions emerged a system of higher ed that came to dominate the academic world.

Escape from Rome

Here’s how Scheidel describes the consequences for Europe that arose from the fall of Rome and the long-time failure of efforts to impose a new empire there.

I argue that a single condition was essential in making the initial breakthroughs possible: competitive fragmentation of power. The nursery of modernity was riven by numerous fractures, not only by those between the warring states of medieval and early modern Europe but also by others within society: between state and church, rulers and lords, cities and magnates, knights and merchants, and, most recently, Catholics and Protestants. This often violent history of conflict and compromise was long but had a clear beginning: the fall of the Roman empire that had lorded it over most of Europe, much as successive Chinese dynasties lorded it over most of East Asia. Yet in contrast to China, nothing like the Roman empire ever returned to Europe.

Recurrent empire on European soil would have interfered with the creation and flourishing of a stable state system that sustained productive competition and diversity in design and outcome. This made the fall and lasting disappearance of hegemonic empire an indispensable precondition for later European exceptionalism and thus, ultimately, for the making of the modern world we now inhabit.

From this developmental perspective, the death of the Roman empire had a much greater impact than its prior existence and the legacy it bequeathed to later European civilization.

Contrast this with China, where dynasties rose and fell but where empire was a constant until the start of the 20th century.  It’s an extension of an argument that others, such as David Landes, have developed about the creative possibilities unleashed by a competitive state system in comparison to the stability and stasis of an imperial power.  Think about the relative stagnation of the  Ottoman, Austro-Hungarian, and Russian empires in 17th, 18th, and 19th centuries compared with the dynamic emerging nation states of Western Europe.  Think also of the paradox within Western Europe, in which the drivers of modernization came not from the richest and strongest imperial powers — Spain, France, and Austria — but from the marginal kingdom of England and the tiny Dutch republic.

The comparison between Europe in China during the second half of the first millennium is telling:

Two things matter most. One is the unidirectional character of European developments compared to the back and forth in China. The other is the level of state capacity and scale from and to which these shifts occurred. If we look at the notional endpoints of around 500 and 1000 CE, the dominant trends moved toward imperial restoration in China and toward inter-and intrastate fragmentation in Europe.

Scheidel shows how social power fragmented after the fall of Rome in such a way that made it impossible for a new hegemonic power to emerge.

After Rome’s collapse, the four principal sources of social power became increasingly unbundled. Political power was claimed by monarchs who gradually lost their grip on material resources and thence on their subordinates. Military power devolved upon lords and knights. Ideological power resided in the Catholic Church, which fiercely guarded its long-standing autonomy even as its leadership was deeply immersed in secular governance and the management of capital and labor. Economic power was contested between feudal lords and urban merchants and entrepreneurs, with the latter slowly gaining the upper hand. In the heyday of these fractures, in the High Middle Ages, weak kings, powerful lords, belligerent knights, the pope and his bishops and abbots, and autonomous capitalists all controlled different levers of social power. Locked in unceasing struggle, they were compelled to cooperate and compromise to make collective action possible.

He points out that “The Christian church was the most powerful and enduring legacy of the Roman empire,” becoming “Europe’s only functioning international organization.”  But in the realms of politics, war, and economy the local element was critical, which produced a situation where local innovation could emerge without interference from higher authority.

The rise of estates and the communal movement shared one crucial characteristic: they produced bodies such as citizen communes, scholarly establishments, merchant guilds, and councils of nobles and commoners that were, by necessity, relatively democratic in the sense that they involved formalized deliberative and consensus-building interactions. Over the long run, these bodies gave Latin Europe an edge in the development of institutions for impersonal exchange that operated under the rule of law and could be scaled up in response to technological change.

Under these circumstances, the states that started to emerge in Europe in the middle ages built on the base of distributed power and local initiative that developed in the vacuum left by the Roman Empire.

As state power recoalesced in Latin Europe, it did so restrained by the peculiar institutional evolution and attendant entitlements and liberties that this acutely fractured environment had engendered and that—not for want of rulers’ trying—could not be fully undone. These powerful medieval legacies nurtured the growth of a more “organic” version of the state—as opposed to the traditional imperial “capstone” state—in close engagement with organized representatives of civil society.

Two features were thus critical: strong local government and its routinized integration into polity-wide institutions, which constrained both despotic power and aristocratic autonomy, and sustained interstate conflict. Both were direct consequences of the fading of late Roman institutions and the competitive polycentrism born of the failure of hegemonic empire. And both were particularly prominent in medieval England: the least Roman of Western Europe’s former Roman provinces, it experienced what with the benefit of hindsight turned out to be the most propitious initial conditions for future transformative development.

The Pax Romana was replaced by a nearly constant state of war, with the proliferation of castle building and the dispersion of military capacity at the local level.  These wars were devastating for the participants but became primary spur for technological, political, and economic innovation.  Everyone needed to develop an edge to help with the inevitable coming conflict.

After the reformation, the small marginal and Protestant states on the North Sea enjoyed a paradoxical advantage in the early modern period, when Catholic Spain, France, and Austria were developing increasingly strong centralized states.  Their marginality allowed them to build most effectively on the inherited medieval model.

…it made a difference that the North Sea region was alone in preserving medieval decentralized political structures and communitarian legacies and building on them during the Reformation while more authoritarian monarchies rose across much of the continent—what Jan Luiten van Zanden deems “an unbroken democratic tradition” from the communal movement of the High Middle Ages to the Dutch Revolt and England’s Glorious Revolution.

England in particular benefited from the differential process of development in Europe.

Yet even as a comprehensive balance sheet remains beyond our reach, there is a case to be made that the British economy expanded and modernized in part because of rather than in spite of the tremendous burdens of war, taxation, and protectionism. By focusing on trade and manufacture as a means of strengthening the state, Britain’s elites came to pursue developmental policies geared toward the production of “goods with high(er) added value, that were (more) knowledge and capital intensive and that were better than those of foreign competitors so they could be sold abroad for a good price.”

Thanks to a combination of historical legacies and geography, England and then Britain happened to make the most of their pricey membership in the European state system. Economic growth had set in early; medieval integrative institutions and bargaining mechanisms were preserved and adapted to govern a more cohesive state; elite commitments facilitated high levels of taxation and public debt; and the wars that mattered most were won.

Reduced to its essentials, the story of institutional development followed a clear arc. In the Middle Ages, the dispersion of power within polities constrained the intensity of interstate competition by depriving rulers of the means to engage in sustained conflict. In the early modern period, these conditions were reversed. Interstate conflict escalated as diversity within states diminished and state capacity increased. Enduring differences between rival polities shaped and were in turn shaped by the ways in which elements of earlier domestic heterogeneity, bargaining and balancing survived and influenced centralization to varying degrees. The key to success was to capitalize on these medieval legacies in maximizing internal cohesion and state capacity later. This alone made it possible to prevail in interstate conflict without adopting authoritarian governance that stifled innovation. The closest approximations of this “Goldilocks scenario” could be found in the North Sea region, first in the Netherlands and then in England.

As maritime European states (England, Spain, Portugal, and the Dutch Republic) spread out across the globe, the competition increased exponentially — which then provided even stronger incentives for innovation at all levels of state and society.

Polycentrism was key. Interstate conflict did not merely foster technological innovation in areas such as ship design and weaponry that proved vital for global expansion, it also raised the stakes by amplifying both the benefits of overseas conquest and its inverse, the costs of opportunities forgone: successful ventures deprived rivals from rewards they might otherwise have reaped, and vice versa. States played a zero-sum game: their involvements overseas have been aptly described as “a competitive process driven as much by anxiety over loss as by hope of gain.”

In conclusion, Scheidel argues that bloody and costly conflict among competing states was the the source of rapid modernization and the rise of European domination of the globe.

I am advocating a perspective that steers clear of old-fashioned triumphalist narratives of “Western” exceptionalism and opposing denunciations of colonialist victimization. The question is not who did what to whom: precisely because competitive fragmentation proved so persistent, Europeans inflicted horrors on each other just as liberally as they meted them out to others around the globe. Humanity paid a staggering price for modernity. In the end, although this may seem perverse to those of us who would prefer to think that progress can be attained in peace and harmony, it was ceaseless struggle that ushered in the most dramatic and exhilaratingly open-ended transformation in the history of our species: the “Great Escape.” Long may it last.

I strongly recommend that you read this book.  There’s insight and provocation on every page.

The Parallel with the History US Higher Education

As I mentioned at the beginning, my own analysis of the emergence of American higher ed tracks nicely on Scheidel’s analysis of Europe after the fall of Rome.  US colleges arose in the early 19th century under conditions where the state was weak, the church divided, and the market strong.  In the absence of a strong central power and a reliable source of financial support, these colleges came into existence as corporations with state charters but not state funding.  (State colleges came later but followed the model of their private predecessors.)  Their creation had less to do with advancing knowledge than with serving more immediately practical aims.

One was to advance the faith in a highly competitive religious environment.  This provided a strong incentive to plant the denominational flag across the countryside, especially on steadily moving western frontier.  A college was a way for Lutherans and Methodists and Presbyterians and others to announce their presence, educate congregants, attract newcomers, and train clergy.  Thus the huge number of colleges in Ohio, the old Northwest Territory.

Another spur for college formation was the crass pursuit of money.  The early US was a huge, underpopulated territory which had too much land and not enough buyers.  This turned nearly everyone on the frontier into a land speculator (ministers included), feverishly coming up with schemes to make the land in their town more valuable for future residents than the land in other towns in the area.  One way to do this was to set up a school, telegraphing to prospects that this was a place to settle down and raise a family.  When other towns followed suit, you could up the ante by establishing a college, usually bearing the town name, which told the world that yours was not some dusty agricultural village but a vibrant center of culture.

The result was a vast number of tiny and unimpressive colleges scattered across the less populated parts of a growing country.  Without strong funding from church or state, they struggled to survive in a highly competitive setting.  This they managed by creating lean institutions that were adept at attracting and retaining student consumers and eliciting donations from alumni and from the wealthier people in town.  The result was the most overbuilt system of higher education the world has ever seen, with five times as many colleges in 1880 than the entire continent of Europe.

All the system was lacking was academic credibility and a strong incentive for student enrollment.  These conditions were met at the end of the century, with the arrival of the German research university to crown the system and give it legitimacy and with the rise of the corporation and its need for white collar workers.

At this point, the chaotic fragmentation and chronic competition that characterized the American system of higher education turned out to be enormously functional.  Free from the constraints that European nation states and national churches imposed on universities, American institutions could develop programs, lure students, hustle for dollars, and promote innovations in knowledge production and technology.  They knew how to make themselves useful to their communities and their states, developing a broad base of political and financial support and demonstrating their social and economic value.

Competing colleges, like competing states, promoted a bottom-up vitality in the American higher ed system that was generally lacking in the older institutions of Europe that were under control of a strong state or church.  Early institutional chaos led to later institutional strength, a system what was not created by design but emerged from an organic process of evolutionary competition.  In the absence of Rome (read: a hegemonic national university), the US higher education system became Rome.

Posted in Course Syllabus, Education policy, Educational Research, Proseminar Class, Scholarship, Theory

Doctoral Proseminar: An Introduction to Big Issues in the Field of Education

This post contains all of the material for the doctoral proseminar — Introduction to Big Issues in the Field of Education — that I taught at the Stanford Graduate School of Education for the last four years.

The aim of this class is to give first-year doctoral students in education a grounding in some of the big issues surrounding the social role and social practice of schooling, with special emphasis on teaching and learning in classrooms and on school organization.  Each of you will soon be specializing in a particular component of the educational domain, but it will be helpful to you to be able to locate your own special area to broader themes and literatures in the field.  A lot of the readings in the class are nodal pieces in the network of educational citations; these are works you need to become familiar with.  This class should help you answer crucial questions about your own work.  What is your study a case of?  What larger issues does it resonate with?  What does it contribute to the larger discourse about school and society?

I’m posting the full syllabus below.  But it would be more useful to get it as a Word document through this link.  Feel free to share it with anyone you like.

All of the course materials are embedded in the syllabus through hyperlinks to a Google drive.  For each week, the syllabus includes a link to tips for approaching the readings, links to the PDFs of the readings, and a link to the slides for that week’s class.  Slides also include links to additional sources.  So the syllabus is all that is needed to gain access to the full class.

I hope you find this useful.

 

Doctoral Proseminar

An Introduction to Big Issues in the Field of Education

David Labaree

Web: http://www.stanford.edu/~dlabaree/

Twitter: @Dlabaree

Blog: https://davidlabaree.com/

 

Course Description

                The aim of this class is to give first-year doctoral students in education a grounding in some of the big issues surrounding the social role and social practice of schooling, with special emphasis on teaching and learning in classrooms and on school organization.  Each of you will soon be specializing in a particular component of the educational domain, but it will be helpful to you to be able to locate your own special area to broader themes and literatures in the field.  A lot of the readings in the class are nodal pieces in the network of educational citations; these are works you need to become familiar with.  This class should help you answer crucial questions about your own work.  What is your study a case of?  What larger issues does it resonate with?  What does it contribute to the larger discourse about school and society?

In the first week we look at the backstory of schooling in the U.S.  We explore its historical roots, the nature of its original mission, and how that mission evolved over time.  And we also examine the conflicting mix of goals that we have imposed on schools and the various social functions they have accumulated over time.

In week two, we turn to the core practices of teaching and learning in classrooms.  Included as issues such as:  the distinctive characteristics of teaching as a professional practice; the socialization of teachers and the incentives that shape the way teachers play their roles; and the grammar of schooling that both defines it and makes it resistant to change.

In week three, we look at the teacher-student relationship in the classroom and how this relationship is experienced by both parties.  We also examine some of the ways that schools create winners and losers, how they both promote and ameliorate social differences.

In week four, we look as school organization from several perspectives:  alternative ways of school organization and their implications for school outcomes; the loose coupling of the nested components of American schooling (classroom, school, school district, and state system) that make it different from other organizations; and the reasons that structural reforms often have little impact on teaching practice.

In week five, we look at the social and cultural pressures that shape school.  This includes examining how the shared expectations of teachers, students, and parents reinforce our conception of what a “real school” is; the rampant formalism that runs through schooling, favoring process over content; and the organizational features of schooling that allow it to carry social functions that the organization of families does not allow.

In week six, we look at the role that race, class, and culture have in schooling.  Among other things, this means examining the class and race factors that shape the kind of cultural knowledge and skill (cultural capital) schools value and try to teach; and the problem of attempting to teach this culture without at the same time demeaning or repressing other cultures.

In week seven, we look at the issue of cultural capital compared to other forms of capital.  And then we consider two different theoretical perspectives on the social role of the school curriculum – the functionalist view that schools teach the knowledge and values that all adults need in order to function in a modern society; and the social reproduction view that schools teach different knowledge and values to students from different backgrounds, thus preparing them for stratified futures.

In week eight, we read a classic book by the political economist Albert Hirschman – Exit, Voice, and Loyalty – which explores the mechanisms by which different kinds of organizations correct for dysfunctional outcomes.  He shows how market organizations are primarily responsive to exit, in which customers signal their dissatisfaction with a product by buying another one instead.  On the other hand, political organizations are primarily responsive to voice, in which clients signal their dissatisfaction by directly voicing their complaint.  From this view, low functioning schools can be seen as unhappy hybrids – political organizations that respond to voice but provoke dissatisfied customers to exit.

In week nine, we read another classic book by the political scientist and anthropologist James Scott – Seeing Like a State.  Here he addresses the problems inherent in state-initiated efforts at social engineering.  The issue is that such schemes too often involve an attempt by planners in the capital to impose a highly rationalized and universalistic model of social order on remote ecologies that are organic and particularistic.  Which sounds a lot like what happens in school reform.

In week ten, we look at the role that educational researchers play in shaping educational policy, the nature of educational research as a practice, and the trajectory of academic careers in a stratified system of higher education.

Readings

            All of the readings for this class are available as PDFs on the web.   This includes the full text of the two books assigned for the course, Hirschman and Scott.

Course Outline

             Below are the topics we will cover, week by week, with the readings for each week Just click on the assigned reading to link to the document on Google drive.  For every week you can click on a link to get tips for doing that week’s readings.  In addition, you can link to the slides for that week’s class.

1) Introduction:  The Historical Roots and Competing Goals of the U.S. School System

Tips for week 1 readings

Labaree, David F. (1997). Public goods, private goods: The American struggle over educational goals. American Educational Research Journal, 34:1 (Spring), 39-81.

Labaree, David F. (2010). Founding the American school system. In Someone has to fail: The zero-sum game of public schooling (pp. 42-79). Cambridge: Harvard University Press.

Class slides for week 1

2) The Problems of Teaching as a Practice

Tips for week 2 readings

Cohen, David K. (1988), Teaching practice: Plus que ça change.  In Phillip W. Jackson (ed.), Contributing to Educational change (pp. 27-84).  Berkeley: McCutchan.

Lortie, Dan C. (1969). The balance of control and autonomy in elementary teaching. In Amatai Etzioni (Ed.), The semi-professions and their organization. Teachers, nurses, social workers. New York, 1-53.

Tyack, David & Tobin, William. (1994). The “grammar” of schooling: Why has it been so hard to change?  American Educational Research Journal 31: 3 (Autumn), 453-479.

Class slides for week 2

3) The Classroom, the Teacher-Student Relationship, and Tracking

Tips for week 3 readings

Jackson, Philip. (1990). The daily grind.  Life in classrooms (pp. 33-50). New York: Teachers College Press.

Waller, Willard.  (1932/1965). The teacher-pupil relationship. In The sociology of teaching (pp. 189-211). New York: Wiley.

Oakes, Jeannie. (1986). Keeping track, part 1: The policy and practice of curriculum inequality. Phi Delta Kappan, 68, 12-17.

Fine, Michelle. (1986). Why urban adolescents drop into and out of public high school. The Teachers College Record, 87(3), 393-409.

Class slides for week 3

4) The Organization of the School

Tips for week 4 readings

Katz, Michael. (1971). Alternative proposals for American education: The nineteenth century. In Class, Bureaucracy, and Schools (pp. 3-55). New York: Praeger.

Weick, Karl. (1976). Educational organizations as loosely-coupled systems. Administrative Science Quarterly, 21, 1-19.

Cuban, Larry. (2013). Why so many structural changes in schools and so little reform in teaching practice?  Inside the black box of classroom practice: Change without reform in American education (pp. 155-187). Cambridge: Harvard Education Press.

Class slides for week 4

5) Expectations and the Roots of the Stability of the School as an Organization

Tips for week 5 readings

Metz, Mary H. (1990). Real school: A universal drama amid disparate experience. In Douglas E. Mitchell & Margaret E. Goertz (Eds.), Education Politics for the New Century (pp. 75-91). New York: Falmer.

Meyer, John W. & Rowan, Brian. (1983). The structure of educational organizations. In Organizational environments: Ritual and rationality (pp. 71-97), edited by John W. Meyer and William R. Scott. Beverly Hills, CA: Sage.

Parsons, Talcott. (1959). The school as a social system: Some of its functions in American society. In Social structure and personality (pp. 129-154). New York: Free Press.

Labaree, David F. (2014). Schooling in the United States:  Historical analyses. In D.C. Phillips (Ed.), Encyclopedia of educational theory and philosophy (pp. 740-43). Sage Publications: Thousand Oaks, CA.

Class slides for week 5

6) Class, Race, and Culture in the School

Tips for week 6 readings

Bernstein, Basil. (1977). Social class, language and socialization. In Jerome Karabel & A. H. Halsey (eds.), Power and ideology in education (pp. 473-486).  New York: Oxford University Press.

Ladson-Billings, G. (1998). Just what is critical race theory and what’s it doing in a nice field like education? International journal of qualitative studies in education, 11(1), 7-24.

Delpit, Lisa. (1995).  The silenced dialogue.  In Other people’s children (pp. 21-47).  New York: New Press.

Recommended:  The Problem We All Live With, Parts 1 and 2.  (2015). This American Life Podcast.  http://www.thisamericanlife.org/radio-archives/episode/562/the-problem-we-all-live-with.

Class slides for week 6

7) Cultural Difference and the School Curriculum

Tips for week 7 readings

Bourdieu, Pierre. (1986). The forms of capital. In John G. Richardson (ed.), Handbook of theory and research for the sociology of education (pp. 241-258). New York: Greenwood.

McWhorter, John. (2018). There’s nothing wrong with Black English. The Atlantic. https://www.theatlantic.com/politics/archive/2018/08/who-gets-to-use-black-english/566867/?utm_source=twb.

Dreeben, Robert. (1968). The contribution of schooling to the learning of norms: Independence, achievement, universalism, and specificity. In On what is learned in school (pp. 63-90). Reading, MA: Addison-Wesley.

Anyon, Jean. (1981). Social class and school knowledge. Curriculum Inquiry, 11, 3-42.

Class slides for week 7

8) Schools as Political and Market Entities

Tips for week 8 readings

Hirschman, Albert O. (2006). Exit, voice, and loyalty: Responses to decline in firms, organizations, and states. Cambridge: Harvard University Press.

Chubb, John E. & Moe, Terry M. (1988).  Politics, markets, and the organization of schools. American Political Science Review, 82:4 (December), 1065-1087.

Du Bois, W.E.B. (1903). The souls of black folk. New York: Knopf. http://etc.usf.edu/lit2go/203/the-souls-of-black-folk/4457/chapter-13-of-the-coming-of-john/

Class slides for week 8

9) The Problem of School Reform – Imposing a Rationalized Vision on the Ecology of the Classroom; Schooling and the Meritocracy

Tips for week 9 readings

Scott, James. (1999).  Seeing like a state.  New Haven: Yale University Press.  Introduction, chapters 1-2 and 9-10.

McClay, William M. (2016). A distant elite: How meritocracy went wrong. The Hedgehog Review 18:2 (Summer). http://www.iasc-culture.org/THR/THR_article_2016_Summer_McClay.php

Class slides for week 9

10) The Role of Researchers in Educational Policy and the Prospects for New Researchers in the University

Tips for week 10 readings

Cohen, David K. & Garet, Michael S. (1975). Reforming educational policy with applied social research. Harvard Educational Review, 45, 17-43.

Weber, Max. (1919/1958). Science as a vocation. In H. H. Gerth & C. Wright Mills, From Max Weber (pp. 129-156). New York: Oxford University Press.

March, James G. (1975). Education and the pursuit of optimism. Texas Tech Journal of Education, 2:1, 5-17.

Labaree, David F. (2012). Sermon on educational research. Bildungsgeschichte: International Journal for the Historiography of Education, 2:1, 78-87.

Class slides for week 10

Guidelines for Critical Reading

As a critical reader of a particular text (a book, article, speech, proposal), you need to use the following questions as a framework to guide you as you read:

  1. What’s the point? This is the analysis issue: what is the author’s angle?
  2. Who says? This is the validity issue: On what (data, literature) are the claims based?
  3. What’s new? This is the value-added issue: What does the author contribute that we don’t already know?
  4. Who cares? This is the significance issue, the most important issue of all, the one that subsumes all the others: Is this work worth doing?  Is the text worth reading?  Does it contribute something important?

If this is the way critical readers are going to approach a text, then as an analytical writer you need to guide readers toward the desired answers to each of these questions.

 Guidelines for Analytical Writing

             In writing papers for this (or any) course, keep in mind the following points.  They apply in particular to the longer papers, but most of the same concerns apply to short papers as well.

  1. Pick an important issue: Make sure that your analysis meets the “so what” test.  Why should anyone care about this topic, anyway?  Pick an issue or issues that matters and that you really care about.

 

  1. Keep focused: Don’t lose track of the point you are trying to make and make sure the reader knows where you are heading and why.

 

  1. Aim for clarity: Don’t assume that the reader knows what you’re talking about; it’s your job to make your points clearly.  In part this means keeping focused and avoiding distracting clutter.  But in part it means that you need to make more than elliptical references to concepts and sources or to professional experience.  When referring to readings (from the course or elsewhere), explain who said what and why this point is pertinent to the issue at hand.  When drawing on your own experiences or observations, set the context so the reader can understand what you mean.  Proceed as though you were writing for an educated person who is neither a member of this class nor a professional colleague, someone who has not read the material you are referring to.

 

  1. Provide analysis: A good paper is more than a catalogue of facts, concepts, experiences, or references; it is more than a description of the content of a set of readings; it is more than an expression of your educational values or an announcement of your prescription for what ails education.  A good paper is a logical and coherent analysis of the issues raised within your chosen area of focus.  This means that your paper should aim to explain rather than describe.  If you give examples, be sure to tell the reader what they mean in the context of your analysis.  Make sure the reader understands the connection between the various points in your paper.

 

  1. Provide depth, insight, and connections: The best papers are ones that go beyond making obvious points, superficial comparisons, and simplistic assertions.  They dig below the surface of the issue at hand, demonstrating a deeper level of understanding and an ability to make interesting connections.

 

  1. Support your analysis with evidence: You need to do more than simply state your ideas, however informed and useful these may be.  You also need to provide evidence that reassures the reader that you know what you are talking about, thus providing a foundation for your argument.  Evidence comes in part from the academic literature, whether encountered in this course or elsewhere.  Evidence can also come from your own experience.  Remember that you are trying to accomplish two things with the use of evidence.  First, you are saying that it is not just you making this assertion but that authoritative sources and solid evidence back you up.  Second, you are supplying a degree of specificity and detail, which helps to flesh out an otherwise skeletal argument.

 

  1. Draw on course materials (this applies primarily to reaction papers, not the final paper). Your paper should give evidence that you are taking this course.  You do not need to agree with any of the readings or presentations, but your paper should show you have considered the course materials thoughtfully.

 

  1. Recognize complexity and acknowledge multiple viewpoints. The issues in the history of American education are not simple, and your paper should not propose simple solutions to complex problems. It should not reduce issues to either/or, black/white, good/bad.  Your paper should give evidence that you understand and appreciate more than one perspective on an issue.  This does not mean you should be wishy-washy.  Instead, you should aim to make a clear point by showing that you have considered alternate views.

 

  1. Challenge assumptions. The paper should show that you have learned something by doing this paper. There should be evidence that you have been open to changing your mind.

 

  1. Do not overuse quotation: In a short paper, long quotations (more than a sentence or two in length) are generally not appropriate.  Even in longer papers, quotations should be used sparingly unless they constitute a primary form of data for your analysis.  In general, your paper is more effective if written primarily in your own words, using ideas from the literature but framing them in your own way in order to serve your own analytical purposes.  However, selective use of quotations can be very useful as a way of capturing the author’s tone or conveying a particularly aptly phrased point.

 

  1. Cite your sources: You need to identify for the reader where particular ideas or examples come from.  This can be done through in-text citation:  Give the author’s last name, publication year, and (in the case of quotations) page number in parentheses at the end of the sentence or paragraph where the idea is presented — e.g., (Kliebard, 1986, p. 22); provide the full citations in a list of references at the end of the paper.  You can also identify sources with footnotes or endnotes:  Give the full citation for the first reference to a text and a short citation for subsequent citations to the same text.  (For critical reaction papers, you only need to give the short cite for items from the course reading; other sources require full citations.)  Note that citing a source is not sufficient to fulfill the requirement to provide evidence for your argument.  As spelled out in #6 above, you need to transmit to the reader some of the substance of what appears in the source cited, so the reader can understand the connection with the point you are making and can have some meat to chew on.  The best analytical writing provides a real feel for the material and not just a list of assertions and citations.  Depth, insight, and connections count for more than a superficial collection of glancing references.  In other words, don’t just mention an array of sources without drawing substantive points and examples from these sources; and don’t draw on ideas from such sources without identifying the ones you used.

 

  1. Take care in the quality of your prose: A paper that is written in a clear and effective style makes a more convincing argument than one written in a murky manner, even when both writers start with the same basic understanding of the issues.  However, writing that is confusing usually signals confusion in a person’s thinking.  After all, one key purpose of writing is to put down your ideas in a way that permits you and others to reflect on them critically, to see if they stand up to analysis.  So you should take the time to reflect on your own ideas on paper and revise them as needed.  You may want to take advantage of the opportunity in this course to submit a draft of the final paper, revise it in light of comments, and then resubmit the revised version.  This, after all, is the way writers normally proceed.  Outside of the artificial world of the classroom, writers never turn in their first draft as their final statement on a subject.

 

Posted in Education policy, Scholarship, School reform, Social Programs, Sociology, Systems of Schooling, Theory

Peter Rossi: The Iron Law of Evaluation and Other Metallic Rules

This post is a classic paper by Peter Rossi from 1987 (Research in Social Problems and Public Policy, Volume 4, pages 3-20) which addresses a chronic problem in all policy efforts to change complex social systems.  The social organizations of modern life are so large, so complex, so dependent on the cooperation of so many actors and agencies that making measurable changes in these organizations of the kind intended by the policymakers is fiendishly difficult.  These problems become particularly visible through the process of program evaluation.  As a result, Rossi comes up with a set of “laws” that govern the evaluation process.

The Iron Law of Evaluation: The expected value of any net impact
assessment of any large scale social program is zero.

The Stainless Steel Law of Evaluation: The better designed the
impact assessment of social program. the more likely is the resulting estimate of net impact to be zero.

The Brass Law of Evaluation: The more social programs are designed to change individuals, the more likely the net impact of the program will be zero.

The Zinc Law of Evaluation: Only those programs that are likely to
fail are evaluated.

Read this lovely piece and you will get a rich sense of how hard it is to design policies that will effect the kind of change that the policies aims to accomplish.  Social organizations have a life of their own whose momentum is difficult to deflect.

Here’s a link to the original paper.

 

THE IRON LAW OF EVALUATION
AND OTHER METALLIC RULES
Peter H. Rossi

INTRODUCTION

Evaluations of social programs have a long history, as history goes in the
social sciences, but it has been only in the last two decades that evaluation
has come close to becoming a routine activity that is a functioning part of
the policy formation process. Evaluation research has become an activity
that no agency administering social programs can do without and still
retain a reputation as modern and up to date. In academia, evaluation
research has infiltrated into most social science departments as an integral
constituent of curricula. In short, evaluation has become institutionalized.
There are many benefits to social programs and to the social sciences
from the institutionalization of evaluation research. Among the more
important benefits has been a considerable increase in knowledge concerning
social problems and about how social programs work (and do not
work). Along with these benefits. however, there have also been attached
some losses. For those concerned with the improvement of the lot of
disadvantaged persons, families and social groups, the resulting knowledge
has provided the bases for both pessimism and optimism. On the
pessimistic side, we ha\e learned that designing successful programs is a
difficult task that is not easily or often accomplished. On the optimistic
side, we have learned more and more about the kinds of programs that can
be successfully designed and implemented. Knowledge derived from evaluations
is beginning to guide our judgments concerning what is feasible
and how to reach those feasible goals.

To draw some important implications from this knowledge about the
workings of social programs is the objective of this paper. The first step is
to formulate a set of “laws” that summarize the major trends in evaluation
findings. Next. a set of explanations arc provided for those overall findings.
Finally, we explore the consequences for applied social science activities
that flow from our new knowledge of social programs.

SOME “LAWS” OF EVALUATION

A dramatic but slightly overdrawn view of two decades of evaluation
efforts can be stated as a set of “laws,” each summarizing some strong
tendency that can be discerned in that body of materials. Following a 19th
Century practice that has fallen into disuse in social science. these laws
are named after substances of varying durability. roughly indexing each
law’s robustness.

The Iron Law of Evaluation: The expected value of any net impact
assessment of any large scale social program is zero.

The Iron Law arises from the experience that few impact assessments
of large scale social programs have found that the programs in question
had any net impact. The law also means that. based on the evaluation
efforts of the las twenty years. the best a priori estimate of the net impact
assessment of any program is zero, i.e., that the the program will have no
effect.

The Stainless Steel Law of Evaluation: The better designed the
impact assessment of social program. the more likely is the resulting
estimate of net impact to be zero.

This law means that the more technically rigorous the net impact
assessment. the more likely arc its results to be zero–ur no effect.
Specifically, this law implies that estimating net impacts through randomized
controlled experiments, the avowedly best approach to estimating
nd impacts. is more likely to show zero effects than other less
rigorous approaches.

The Brass Law of Evaluation: The more social programs are designed
to change individuals, the more likely the net impact of the program will
be zero.

This law means that social programs designed to rehabilitate individuals
by changing them in some way or another are more likely to fail. The
Brass Law may appear to be redundant since all programs, including those
designed to deal with individuals, are covered by the Iron Law. This
redundancy is intended to emphasize the especially difficult task faced in
designing and implementing effective programs that are designed to rehabilitate
individuals.

The Zinc Law of Evaluation: Only those programs that are likely to
fail are evaluated.

Of the several metallic laws of evaluation, the zinc law has the most
optimistic slant since it implies that there are effective programs but that
such effective programs are never evaluated. It also implies that if a social
program is effective, that characteristic is obvious enough and hence
policy makers and others who sponsor and fund evaluations decide
against evaluation.

It is possible to formulate a number of additional laws of evaluation,
each attached to one or another of a variety of substances varying in
strength ranging from strong, robust metals to flimsy materials. The substances
involved are only limited by one’s imagination. But, if such laws
are to mirror the major findings of the last two decades of evaluation
research they would all carry the same message: The laws would claim
that a review of the history of the last two decades of efforts to evaluate
major social programs in the United States sustain the proposition that
over this period the American establishment of policy makers, agency
officials, professionals and social scientists did not know how to design
and implement social programs that were minimally effective, let alone
spectacularly so.

HOW FIRM ARE THE METALLIC LAWS OF EVALUATION?

How seriously should we take the metallic laws? Are they simply the
social science analogue of poetic license, intended to provide dramatic
emphasis? Or, do the laws accurately summarize the last two decades’
evaluation experiences?

First of all, viewed against the evidence, the iron law is not entirely
rigid. True, most impact assessments conform to the iron law’s dictates in
showing at best marginal effects and all too often no effects at all. There
are even a few evaluations that have shown effects in the wrong directions,
opposite to the desired effects. Some of the failures of large scale programs
have been particularly disappointing because of the large investments
of time and resources involved: Manpower retraining programs
have not been shown to improve earnings or employment prospects of
participants (Westat, 1976-1980). Most of the attempts to rehabilitate pris-
oners have failed to reduce recidivism (Lipton, Martinson, and Wilks, 1975).
Most educational innovations have not been shown to improve student
learning appreciably over traditional methods (Raizen and Rossi, 1981 ).

But, there are also many exceptions to the iron rule! The “iron” in the
Iron Law has shown itself to be somewhat spongy and therefore easily,
although not frequently, broken. Some social programs have shown
positive effects in the desired directions, and there are even some quite
spectacular successes: the American old age pension system plus Medicare
has dramatically improved the lives of our older citizens. Medicaid
has managed to deliver medical services to the poor to the extent that the
negative correlation between income and consumption of medical services
has declined dramatically since enactment. The family planning
clinics subsidized by the federal government were effective in reducing the
number of births in areas where they were implemented (Cutright and
Jaffe, 1977). There are also human services programs that have been shown
to be effective, although mainly on small scale, pilot runs: for example, the
Minneapolis Police Foundation experiment on the police handling of
family violence showed that if the police placed the offending abuser in
custody over night that the offender was less likely to show up as an
accused offender over the succeeding six months ( Sherman and Berk, 1984 ).
A meta-evaluation of psychotherapy showed that on the average, persons
in psychotherapy-no matter what brand-were a third of a standard
deviation improved over control groups that did not have any therapy
(Smith, Glass, and Miller, 1980). In most of the evaluations of manpower
training programs, women returning to the labor force benefitted
positively compared to women who did not take the courses, even though
in general such programs have not been successful. Even Head Start is
now beginning to show some positive benefits after many years of equivocal
findings. And so it goes on, through a relatively long list of successful
programs.

But even in the case of successful social programs, the sizes of the net
effects have not been spectacular. In the social program field, nothing has
yet been invented which is as effective in its way as the small pox vaccine
was for the field of public health. In short, as is well known (and widely
deplored) we arc not on the verge of wiping out the social scourges of our
time: ignorance, poverty, crime, dependency, or mental illness show great
promise to be with us for some time to come.

The Stainless Steel Law appears to be more likely to hold up over a
The Iron Law of Evaluation and Other Metallic Rules 7
large series of cases than the more general Iron Law. This is because the
fiercest competition as an explanation for the seeming success of any
program-especially human services programs-ordinarily is either selfor
administrator-selection of clients. In other words, if one finds that a
program appears to be effective, the most likely alternative explanation to
judging the program as the cause of that success is that the persons
attracted to that program were likely to get better on their own or that the
administrators of that program chose those who were already on the road
to recovery as clients. As the better research designs-particularly randomized
experiments-eliminate that competition, the less likely is a
program to show any positive net effect. So the better the research design,
the more likely the net impact assessment is likely to be zero.

How about the Zinc Law of Evaluation? First, it should be pointed out
that this law is impossible to verify in any literal sense. The only way that
one can be relatively certain that a program is effective is to evaluate it,
and hence the proposition that only ineffective programs are evaluated can
never be proven.

However, there is a sense in which the Zinc law is correct. If the a
priori, beyond-any-doubt expectations of decision makers and agency
heads is that a program will be effective, there is little chance that the
program will be evaluated at all. Our most successful social program,
social security payments to the aged has never been evaluated in a rigorous
sense. It is “well known” that the program manages to raise the incomes
of retired persons and their families, and “it stands to reason” that this
increase in income is greater than what would have happened, absent the
social security system.

Evaluation research is the legitimate child of skepticism, and where
there is faith, research is not called upon to make a judgment. Indeed, the
history of the income maintenance experiments bears this point out.
Those experiments were not undertaken to find out whether the main
purpose of the proposed program could be achieved: that is, no one
doubted that payments would provide income to poor people-indeed,
payments by definition are income, and even social scientists are not
inclined to waste resources investigating tautologies. Furthermore, no one
doubted that payments could be calculated and checks could be delivered
to households. The main purpose of the experiment was to estimate the
sizes of certain anticipated side effects of the payments, about which
economists and policy makers were uncertain-how much of a work
disincentive effect would be generated by the payments and whether the
payments would affect other aspects of the households in undesirable
ways-for instance, increasing the divorce rate among participants.

In short, when we look at the evidence for the metallic laws, the
evidence appears not to sustain their seemingly rigid character, but the
evidence does sustain the “laws” as statistical regularities. Why this
should be the case, is the topic to be explored in the remainder of this
paper.

IS THERE SOMETHING WRONG WITH EVALUATION RESEARCH?

A possibility that deserves very serious consideration is that there is
something radically wrong with the ways in which we go about conducting
evaluations. Indeed, this argument is the foundation of a revisionist school
of evaluation, composed of evaluators who are intent on calling into
question the main body of methodological procedures used in evaluation
research, especially those that emphasize quantitative and particularly
experimental approaches to the estimation of net impacts. The revisionists
include such persons as Michael Patton ( 1980) and Egon Guba (1981 ).
Some of the revisionists are reformed number crunchers who have seen
the errors of their ways and have been reborn as qualitative researchers.
Others have come from social science disciplines in which qualitative
ethnographic field methods have been dominant.

Although the issue of the appropriateness of social science methodology
is an important one, so far the revisionist arguments fall far short
of being fully convincing. At the root of the revisionist argument appears
to be that the revisionists find it difficult to accept the findings that most
social programs, when evaluated for impact assessment by rigorous quantitative
evaluation procedures, fail to register main effects: hence the
defects must be in the method of making the estimates. This argument per
se is an interesting one, and deserves attention: all procedures need to be
continually re-evaluated. There are some obvious deficiencies in most
evaluations, some of which are inherent in the procedures employed. For
example, a program that is constantly changing and evolving cannot
ordinarily be rigorously evaluated since the treatment to be evaluated
cannot be clearly defined. Such programs either require new evaluation
procedures or should not be evaluated at all.

The weakness of the revisionist approaches lies in their proposed
solutions to these deficiencies. Criticizing quantitative approaches for
their woodenness and inflexibility, they propose to replace current methods
with procedures that have even greater and more obvious deficiencies.
The qualitative approaches they propose are not exempt from issues of
internal and external validity and ordinarily do not attempt to address
these thorny problems. Indeed, the procedures which they advance as
substitutes for the mainstream methodology are usually vaguely des-
scribed, constituting an almost mystical advocacy of the virtues of qualitative
approaches, without clear discussion of the specific ways in which
such procedures meet validity criteria. In addition, many appear to adopt
program operator perspectives on effectiveness, reasoning that any effort
to improve social conditions must have some effect, with the burden of
proof placed on the evaluation researcher to find out what those effects
might be.

Although many of their arguments concerning the woodenness of many
quantitative researches are cogent and well taken, the main revisionist
arguments for an alternative methodology are unconvincing: hence one
must look elsewhere than to evaluation methodology for the reasons for
the failure of social programs to pass muster before the bar of impact
assessments.

SOURCES OF PROGRAM FAILURES

Starting with the conviction that the many findings of zero impact are real,
we are led inexorably to the conclusion that the faults must lie in the
programs. Three kinds of failure can be identified, each a major source of
the observed lack of impact:
The first two types of faults that lead a program to fail stem from
problems in social science theory and the third is a problem in the
organization of social programs:

I. Faults in Problem Theory: The program is built upon a faulty understanding
of the social processes that give rise to the problem to
which the social program is ostensibly addressed;

2. Faults in Program Theory: The program is built upon a faulty
understanding of how to translate problem theory into specific
programs.

3. Faults in Program Implementation: There are faults in the organizations,
resources levels and/or activities that are used to deliver
the program to its intended beneficiaries.

Note that the term theory is used above in a fairly loose way to cover all
sorts of empirically grounded generalized knowledge about a topic, and is
not limited to formal propositions.

Every social program, implicitly or explicitly is based on some understanding
of the social problem involved and some understanding of the
program. If one fails to arrive at an appropriate understanding of either,
the program in question will undoubtedly fail. In addition, every program
is given to some organization to implement. Failures to provide enough
resources, or to insure that the program is delivered with sufficient fidelity
can also lead to findings of ineffectiveness.

Problem Theory

Problem theory consists of the body of empirically tested understanding
of the social problem that underlies the design of the program in
question. For example, the problem theory that was the underpinning for
the many attempts at prisoner rehabilitation tried in the last two decades
was that criminality was a personality disorder. Even though there was a
lot of evidence for this viewpoint, it also turned out that the theory is not
relevant either to understanding crime rates or to the design of crime
policy. The changes in crime rates do not reflect massive shifts in personality
characteristics of the American population, nor does the personality
disorder theory of crime lead to clear implications for crime reduction
policies. Indeed, it is likely that large scale personality changes are beyond
the reach of social policy institutions in a democratic society.
The adoption of this theory is quite understandable. For example, how
else do we account for the fact that persons seemingly exposed to the
same influences do not show the same criminal (or noncriminal) tendencies?
But the theory is not useful for understanding the social distribution
of crime rates by gender, socio-economic level, or by age.

Program Theory

Program theory links together the activities that constitute a social
program and desired program outcomes. Obviously, program theory is
also linked to problem theory, but is partially independent. For example,
given the problem theory that diagnosed criminality is a personality disorder,
a matching program theory would have as its aims personality
change oriented therapy. But there are many specific ways in which
therapy can be defined and at many different points in the life history of
individuals. At the one extreme of the lifeline, one might attempt preventive
mental health work directed toward young children: at the other
extreme, one might provide psychiatric treatment for prisoners or set up
therapeutic groups in prison for convicted offenders.

Implementation

The third major source of failure is organizational in character and has
to do with the failure to implement properly programs. Human services
programs are notoriously difficult to deliver appropriately to the appropriate
clients. A well designed program that is based on correct problem and
program theories may simply be implemented improperly, including not
implementing any program at all. Indeed, in the early days of the War on
Proverty, many examples were found of non-programs-the failure to
implement anything at all.

Note that these three sources of failure are nested to some degree:

1. An incorrect understanding of the social problem being addressed
is clearly a major failure that invalidates a correct program theory
and an excellent implementation.

2. No matter how good the problem theory may be, an inappropriate
program theory will lead to failure.

3. And, no matter how good the problem and program theories, a
poor implementation will also lead to failure.

Sources of Theory Failure

A major reason for failures produced through incorrect problem and
program theories lies in the serious under-development of policy related
social science theories in many of the basic disciplines. The major problem
with much basic social science is that social scientists have tended to
ignore policy related variables in building theories because policy related
variables account for so little of the variance in the behavior in question.It
does not help the construction of social policy any to know that a major
determinant of criminality is age, because there is little, if anything, that
policy can do about the age distribution of a population, given a committment
to our current democratic, liberal values. There are notable exceptions
to this generalization about social science: economics and political
science have always been closely attentive to policy considerations; this
indictment concerns mainly such fields as sociology, anthropology and
psychology.

Incidentally, this generalization about social science and social scientists
should warn us not to expect too much from changes in social policy.
This implication is quite important and will be taken up later on in this
paper.

But the major reason why programs fail through failures in problem and
program theories is that the designers of programs are ordinarily amateurs
who know even less than the social scientists! There are numerous examples
of social programs that were concocted by well meaning amateurs
(but amateurs nevertheless). A prime example are Community Mental
Health Centers, an invention of the Kennedy administration, apparently
undertaken without any input from the National Institute of Mental
Health, the agency that was given the mandate to administer the program.
Similarly with Comprehensive Employment and Training Act (CETA) and
its successor, the current Job Partnership Training Act (JPTA) program,
both of which were designed by rank amateurs and then given over to the
Department of Labor to run and administer. Of course, some of the
amateurs were advised by social scientists about the programs in question,
so the social scientists are not completely blameless.

The amateurs in question are the legislators, judicial officials, and other
policy makers who initiate policy and program changes. The main problem
with amateurs lies not so much in their amateur status but in the fact
that they may know little or nothing about the problem in question or
about the programs they design. Social science may not be an extraordinarily
well developed set of disciplines, but social scientists do know
something about our society and how it works, knowledge that can prove
useful in the design of policy and programs that may have a chance to be
successfully effective.

Our social programs seemingly are designed by procedures that lie
somewhere in between setting monkeys to typing mindlessly on typewriters
in the hope that additional Shakespearean plays will eventually be
produced, and Edisonian trial-and-error procedures in which one tactic
after another is tried in the hope of finding out some method that works.
Although the Edisonian paradigm is not highly regarded as a scientific
strategy by the philosophers of science, there is much to recommend it in
a historical period in which good theory is yet to develop. It is also a
strategy that allows one to learn from errors. Indeed, evaluation is very
much a part of an Edisonian strategy of starting new programs, and
attempting to learn from each trial.

PROBLEM THEORY FAILURES

One of the more persistent failures in problem theory is to under-estimate
the complexity of the social world. Most of the social problems with which
we deal are generated by very complex causal processes involving interactions
of a very complex sort among societal level, community level, and
individual level processes. In all likelihood there are biological level processes
involved as well, however much our liberal ideology is repelled by
the idea. The consequence of under-estimating the complexity of the
problem is often to over-estimate our abilities to affect the amount and
course of the problem. This means that we are overly optimistic about how
much of an effect even the best of social programs can expect to achieve. It
also means that we under-design our evaluations, running the risk of
committing Type II errors: that is, not having enough statistical power in
our evaluation research designs to be able to detect reliably those small
effects that we are likely to encounter.

It is instructive to consider the example of the problem of crime in our
society. In the last two decades, we have learned a great deal about the
crime problem through our attempts by initiating one social program aft~r
another to halt the rising crime rate in our society. The end result of this
series of trials has largely failed to have significant impacts on the crime
rates. The research effort has yielded a great deal of empirical knowledge
about crime and criminals. For example, we now know a great deal about
the demographic characteristics of criminals and their victims. But, we
still have only the vaguest ideas about why the crime rates rose so steeply
in the period between 1970 and 1980 and, in the last few years, have started
what appears to be a gradual decline. We have also learned that the
criminal justice system has been given an impossible task to perform and,
indeed, practices a wholesale form of deception in which everyone acquiesces.

It has been found that most perpetrators of most criminal acts go
undetected, when detected go unprosecuted, and when prosecuted go
unpunished, Furthermore, most prosecuted and sentenced criminals are
dealt with by plea bargaining procedures that are just in the last decade
getting formal recognition as occurring at all. After decades of sub-rosa
existence, plea bargaining is beginning to get official recognition in the
criminal code and judicial interpretations of that code.

But most of what we have learned in the past two decades amounts to a
better description of the crime problem and the criminal justice system as
it presently functions. There is simply no doubt about the importance of
this detailed information: it is going to be the foundation of our understanding
of crime; but, it is not yet the basis upon which to build policies
and programs that can lessen the burden of crime in our society.
Perhaps the most important lesson learned from the descriptive and
evaluative researches of the past two decades is that crime and criminals
appear to be relatively insensitive to the range of policy and program
changes that have been evaluated in this period. This means that the
prospects for substantial improvements in the crime problem appear to be
slight, unless we gain better theoretical understanding of crime and criminals.
That is why the Iron Law of Evaluation appears to be an excellent
generalization for the field of social programs aimed at reducing crime and
leading criminals to the straight and narrow way of life. The knowledge
base for developing effective crime policies and programs simply does not
exist; and hence in this field, we are condemned-hopefully temporarilyto
Edisonian trial and error.

PROGRAM THEORY AND IMPLEMENTATION FAILURES

As defined earlier, program theory failures are translations of a proper
understanding of a problem into inappropriate programs, and program
implementation failures arise out of defects in the delivery system used.
Although in principle it is possible to distinguish program theory failures
from program implementation failures, in practice it is difficult to do so.
For example, a correct program may be incorrectly delivered, and hence
would constitute a “pure” example of implementation failure, but it would
be difficult to identify this case as such, unless there were some instances
of correct delivery. Hence both program theory and program implementation
failures will be discussed together in this section.

These kinds of failure are likely the most common causes of ineffective
programs in many fields. There are many ways in which program theory
and program implementation failures can occur. Some of the more common
ways are listed below.

Wrong Treatment

This occurs when the treatment is simply a seriously flawed translation
of the problem theory into a program. One of the best examples is the
housing allowance experiment in which the experimenters attempted to
motivate poor households to move into higher quality housing by offering
them a rent subsidy, contingent on their moving into housing that met
certain quality standards (Struyk and Bendick, 1981). The experimenters
found that only a small portion of the poor households to whom this offer
was made actually moved to better housing and thereby qualified for and
received housing subsidy payments. After much econometric calculation,
this unexpected outcome was found to have been apparently generated by
the fact that the experimenters unfortunately did not take into account
that the costs of moving were far from zero. When the anticipated dollar
benefits from the subsidy were compared to the net benefits, after taking
into account the costs of moving, the net benefits were in a very large
proportion of the cases uncomfortably close to zero and in some instances
negative. Furthermore, the housing standards applied almost totally
missed the point. They were technical standards that often characterized
housing as sub-standard that was quite acceptable to the households
involved. In other words, these were standards that were regarded as
irrelevant by the clients. It was unreasonable to assume that households
would undertake to move when there was no push of dissatisfaction from
the housing occupied and no substantial net positive benefit in dollar
terms for doing so. Incidentally, the fact that poor families with little
formal education were able to make decisions that were consistent with
the outcomes of highly technical econometric calculations improves one’s
appreciation of the innate intellectual abilities of that population.

Right Treatment But Insufficient Dosage

A very recent set of trial policing programs in Houston, Texas and
Newark, New Jersey exemplifies how programs may fail not so much
because they were administering the wrong treatment but because the
treatment was frail and puny (Police Foundation, 1985). Part of the goals of
the program was to produce a more positive evaluation of local police
departments in the views of local residents. Several different treatments
were attempted. In Houston, the police attempted to meet the presumed
needs of victims of crime by having a police officer call them up a week of
so after a crime complaint was received to ask “how they were doing” and
to offer help in “any way.” Over a period of a year, the police managed to
contact about 230 victims, but the help they could offer consisted mainly
of referrals to other agencies. Furthermore, the crimes in question were
mainly property thefts without personal contact between victims and
offenders, with the main request for aid being requests to speed up the
return of their stolen property. Anyone who knows even a little bit about
property crime in the United States would know that the police do little or
nothing to recover stolen property mainly because there is no way they can
do so. Since the callers from the police department could not offer any
substantial aid to remedy the problems caused by the crimes in question,
the treatment delivered by the program was essentially zero. It goes
without saying that those contacted by the police officers did not differ
from randomly selected controls-who had also been victimized but who
had not been called by the police-in their evaluation of the Houston
Police Department.

It seems likely that the treatment administered, namely expressions of
concern for the victims of crime, administered in a personal face-to-face
way, would have been effective if the police could have offered substantial
help to the victims.

Counter-acting Delivery System

It is obvious that any program consists not only of the treatment
intended to be delivered, but it also consists of the delivery system and
whatever is done to clients in the delivery of services. Thus the income
maintenance experiments’ treatments consist not only of the payments,
but the entire system of monthly income reports required of the clients,
the quarterly interviews and the annual income reviews, as well as the
payment system and its rules. In that particular case, it is likely that the
payments dominated the payment system, but in other cases that might
not be so, with the delivery system profoundly altering the impact of the
treatment.

Perhaps the most egregious example was the group counselling program
run in California prisons during the 1960s (Kassebaum, Ward, and
Wilner, 1972). Guards and other prison employees were used as counseling
group leaders, in sessions in which all participants-prisoners and
guards-were asked to be frank and candid with each other! There are
many reasons for the abysmal failure3 of this program to affect either
criminals’ behavior within prison or during their subsequent period of
parole, but among the leading contenders for the role of villain was the
prison system’s use of guards as therapists.

Another example is the failure of transitional aid payments to released
prisoners when the payment system was run by the state employment
security agency, in contrast to the strong positive effect found when run by
researchers (Rossi, Berk, and Lenihan, 1980). In a randomized experiment
run by social researchers in Baltimore, the provision of 3 months of
minimal support payments lowered the re-arrest rate by 8 percent, a small
decrement, but a significant one that was calculated to have very high cost
to benefit ratios. When, the Department of Labor wisely decided that
another randomized experiment should be run to see whether YOAA”
Your Ordinary American Agency”-could achieve the same results,
large scale experiments in Texas and Georgia showed that putting the
treatment in the hands of the employment security agencies in those two
states cancelled the positive effects of the treatment. The procedure which
produced the failure was a simple one: the payments were made contingent
on being unemployed, as the employment security agencies usually
administered unemployment benefits, creating a strong work disincentive
effect with the unfortunate consequence of a longer period of unemployment
for experimentals as compared to their randomized controls and
hence a higher than expected re-arrest rate.

Pilot and Production Runs

The last example can be subsumed under a more general point — namely,
given that a treatment is effective in a pilot test does not mean that
when turned over to YOAA, effectiveness can be maintained. This is the
lesson to be derived from the transitional aid experiments in Texas and
Georgia and from programs such as The Planned Variation teaching demonstration.
In the latter program leading teaching specialists were asked to
develop versions of their teaching methods to be implemented in actual
school systems. Despite generous support and willing cooperation from
their schools, the researchers were unable to get workable versions of
their teaching strategies into place until at least a year into the running of
the program. There is a big difference between running a program on a
small scale with highly skilled and very devoted personnel and running a
program with the lesser skilled and less devoted personnel that YOAA
ordinarily has at its disposal. Programs that appear to be very promising
when run by the persons who developed them, often turn out to be
disappointments when turned over to line agencies.

Inadequate Reward System

The internally defined reward system of an organization has a strong
effect on what activities are assiduously pursued and those that are
characterized by “benign neglect.” The fact that an agency is directed to
engage in some activity does not mean that it will do so unless the reward
system within that organization actively fosters compliance. Indeed, there
are numerous examples of reward systems that do not foster compliance.
Perhaps one of the best examples was the experience of several police
departments with the decriminalization of public intoxification. Both the
District of Columbia and Minneapolis-among other jurisdictions-rescinded
their ordinances that defined public drunkenness as misdemeanors,
setting up detoxification centers to which police were asked to
bring persons who were found to be drunk on the streets. Under the old
system, police patrols would arrest drunks and bring them into the local
jail for an overnight stay. The arrests so made would “count” towards the
department measures of policing activity. Patrolmen were motivated
thereby to pick up drunks and book them into the local jail, especially in
periods when other arrest opportunities were slight. In contrast, under the
new system, the handling of drunks did not count towards an officer’s
arrest record. The consequence: Police did not bring drunks into the new
detoxification centers and the municipalities eventually had to set up
separate service systems to rustle up clients for the dextoxification
systems.

The illustrations given above should be sufficient to make the general
point that the apropriate implementation of social programs is a problematic
matter. This is especially the case for programs that rely on persons to
deliver the service in question. There is no doubt that federal, state, and
local agencies can calculate and deliver checks with precision and efficiency.
There also can be little doubt that such agencies can maintain a
physical infra-structure that delivers public services efficiently, even
though there are a few examples of the failure of water and sewer systems
on scales that threaten public health. But there is a lot of doubt that human
services that are tailored to differences among individual clients can be
done well at all on a large scale basis.
We know that public education is not doing equally well in facilitating
the learning of all children. We know that our mental health system does
not often succeed in treating the chronically mentally ill in a consistent
and effective fashion. This does not mean that some children cannot be
educated or that the chronically mentally ill cannot be treated-it does
mean that our ability to do these activities on a mass scale is somewhat in
doubt.

CONCLUSIONS

This paper started out with a recital of the several metallic laws stating
that evaluations of social programs have rarely found them to be effective
in achieving their desired goals. The discussion modified the metallic laws
to express them as statistical tendencies rather than rigid and inflexible
laws to which all evaluations must strictly adhere. In this latter sense, the
laws simply do not hold. However, when stripped of their rigidity, the laws
can be seen to be valid as statistical generalizations, fairly accurately
representing what have been the end results of evaluations “on-the-average.”
In short, few large-scale social programs have been found to be even
minimally effective. There have been even fewer programs found to be spectacularly
effective. There are no social science equivalents of the Salk vaccine.

Were this conclusion the only message of this paper, then it would tell a
dismal tale indeed. But there is a more important message in the examination
of the reasons why social programs fail so often. In this connection,
the paper pointed out two deficiencies:

First, policy relevant social science theory that should be the intellectual
underpinning of our social policies and programs is either deficient or
simply missing. Effective social policies and programs cannot be designed
consistently until it is thoroughly understood how changes in policies and
programs can affect the social problems in question. The social policies
and programs that we have tested have been designed, at best, on the basis
of common sense and perhaps intelligent guesses, a weak foundation for
the construction of effective policies and programs.

In order to make progress, we need to deepen our understanding of the
long range and proximate causation of our social problems and our understanding
about how active interventions might alleviate the burdens of
those problems. This is not simply a call for more funds for social science
research but also a call for a redirection of social science research toward
understanding how public policy can affect those problems.

Second, in pointing to the frequent failures in the implementation of
social programs, especially those that involve labor intensive delivery of
services, we may also note an important missing professional activity in
those fields. The physical sciences have their engineering counterparts;
the biological sciences have their health care professionals; but social
science has neither an engineering nor a strong clinical component. To be
sure, we have clinical psychology, education, social work, public administration,
and law as our counterparts to engineering, but these are only
weakly connected with basic social science. What is apparently needed is
a new profession of social and organizational engineering devoted to the
design of human services delivery systems that can deliver treatments
with fidelity and effectiveness.

In short, the double message of this paper is an argument for
further development of policy relevant basic social science and the establishment
of the new profession of social engineer.

NOTES

I. Note that the law emphasizes that it applied primarily to “large scale” social
programs, primarily those that are implemented by an established governmental agency
covering a region or the nation as a whole. It does not apply to small scale demonstrations or to programs run by their designers.
2. Unfortunately, it has proven difficult to stop large scale programs even when evaluations prove them to be ineffective. The federal job training programs seem remarkably resistant to the almost consistent verdicts of ineffectiveness. This limitation on the Edisonian paradigm arises out of the tendency for large scale programs to accumulate staff and clients that have extensive stakes in the program’s continuation.
3. This is a complex example in which there are many competing explanations for the
failure of the program. In the first place, the program may be a good example of the failure of problem theory since the program was ultimately based on a theory of criminal behavior as psychopathology. In the second place, the program theory may have been at fault for employing counselling as a treatment. This example illustrates how difficult it is to separate out the three sources of program failures in specific instances.

REFERENCES

Cutright, P. and F. S. Jaffe
1977 Impact of Family Planning Programs on Fertility: The U.S. Experience. New
York: Praeger.
Guba, E. G. and Y. S. Lincoln
1981 Effective Evaluation: Improving the Usefulness of Evaluation Results Through
Responsive and Naturalistic Approaches. San Francisco: Jossey-Bass.
Kassebaum, G., D. Ward, and D. Wilner
1971 Prison Treatment and Parole Survival. New York: John Wiley.
Lipton, D., R. Martinson, and L. Wilks
1975 The Effectiveness of Correctional Treatment. New York: Praeger.
Patton, M.
1980 Qualitative Evaluation Methods. Beverly Hills, CA: Sage Publications.
Police Foundation
1985 Evaluation of Newark and Houston Policing Experiments. Washington, DC.
Raizen, S. A. and P. H. Rossi (eds.)
1980 Program Evaluation in Education: When? How? To What Ends? Washington,
DC: National Academy Press.
Rossi, P. H., R. A. Berk and K. J. Lenihan
1980 Money, Work and Crime. New York: Academic.
Sherman, L. W. and R. A. Berk.
1984. “Deterrent effects of arrest for domestic assault.” American Sociological Review
49: 261-271.
Smith, M. L., G. V. Glass, and T. I. Miller
1980 The Benefits of Psychotherapy: An Evaluation. Baltimore: The Johns Hopkins
University Press.
Struyk, R. J. and M. Bendick
1981 Housing Vouchers for the Poor. Washington, DC: The Urban Institute.
Westat, Inc.
1976- Continuous Longitudinal Manpower Survey, Reports 1-10. Rockville, MD:
1980 Westat, Inc.