ic/research community that is largely outThe sin-been advocacy—fo - PDF document

ic/research community that is largely outThe sin-been advocacy—followed up by mas-Reality 1. Large-scale reform requireshighly specific, systematic, and structur-as "technologies.") curric-plary curricula to be found. The only ex-658lan-auage arts or social studies curricula. WhileMyth 2. Theory is a useful guide for thedesign of programs and reforms. portant to the design of sophisticated pro-grams than research and theory. curriculum — a counterintuitive decisionThe REsearch/Academic/Reform commu-Kaes-to use researchcisions as separate from research knowl-edgmethodology, structure, dosages, and ma-No other field expects its practitioners to but to write theplicable to most local conditions—if practice, i.e., new paradigms. scientificknowledge. In education, most reformses are incremental ones. new pieces of technology and 2,000-yearteachers and students conversingMyth 5. The best way to achieve reformIn someday. relates ofdetrackingall of these are important and are learning is to develop more powerful pro-grams to enhance learning. duced as by-products of increased learn-fining systematic learning environments. Myth 7. You can understand large-scaleon a very small scale. meta-Reality 7. It’s the scale, stupid! Large-scale change reflects properties that arefect in small-scale research. While small-implement such programs rob teachers oftheir individuality. new generation of far more powerful pro-nents — e.g., scripts and choreography —such as Reading Recovery and Junior GreatFor the last 100 years REAR has beeneveryone the same — instead of by devel-Repeated failure of reform initiatives.advocacy of child-centeredness is associ-the century, the goal ofchild-centered-ness“junior high school” — as an alternativeMassive waste of resources on staff de-velopment and dissemination. opment with religious fervor as for considering and implementing reformlarge-Sarasonalso adopted a stance that essentially sai:::T]he medical community has madecancersphasized the complexity and scope of thepresent conceptions and practices...13lemming-that it is either workable or effective. 14gressive and traditional reforms. For example, in a recent article, Diane Rav-skills." 15better.16worse than a broken clock; at least the clockwere incorporated into the legislation. This Meta-analysislargeMisleading conclusions and mislead-ing uses of research. make sense19derstanding of the nature of student/teach-deed, as I was writing this article, the then-Educational Research Journal (AERJ) higher-ones20cluded that the teacher was teaching dif-ferently from the way they would havethinking than we would. " 21 Unfortunately,So what else is new? The AERJ researchresearch results from meta-analyses. Meta-scale. In meta-analysis, results from a seriesas gifted programs. 23Unfortunately, meta-the programs in the included studies, meta-fective programsfor pscience, and efforts for precise interventions. ing “All children can learn” says nothingcan how ferent youngsters can learn, best to use a computer — “Use it as a tool!”dents should nologies should sig- nificantly is by the use of more powerfulist. Any other type of reform is a sham —of the REsearch/Academic/Reform compas-sionately, even desperately, want to helpschools.ing as the primary courses shot through with hypeer — a creature that understands Englishword problems on tests as simpleinnovative ideas but few innovations —es. Education can no longer afford a well-10--dent, that we do we do tivation to learn, and that we do tech- idem,idem,At-idem,Hansot,Calif.:#79-A5,Tobin,idem,McQuaide1993/JanuaryMirel,1991-93," AmericanFall 1994, pp. 481-518; and Dianne Taylor302-19.Exemplary Materials,” Educational Leadership, 8. Thomas Kuhn, The Structure of Scientific Revolu-a discussion of this 35-minute principle, see Pogrow,idem,Fogarty, eds., If Minds Matter: A Foreword to the10.researchengagingSarason,1990)findYork Times, Har-1990),307-36.ter 1/Title I, see Stanley Pogrow, “The Forgottenidem,624-30.Pogrow, "Challenging At-Risk Students"; and idem,Rowan,and Yuk-Fai555-585-610.Walberg,1994/Januarymeta-anal-SlavinAllan,pp. 60–65; and Stanley Pogrow, “Good StatisticsBracey, “The Fourth Bracey Report114-27.520/621-9373. WannabePogroweducation reform overIllustrationJimHummel BY STANLEY POGROWOR MORE THAN a decade, wehave been buried in proposed re-orms. Those responsible for thisavalanche of reforms have takenthe perspective that there are prob-lems with the education establishment,problems with society, problems with thepolitical structure, problems with currentpractice - in short, problems witheverything except reformers and theirproposed reforms.Reformers typically feel that their so-lutions would work if only people wouldget on board. When people do not jumppractitioners are at fault or that the soci-ety is at fault for socializing individualsin ways that prevent them from appreci-ating the wisdom of the reforms. Whenthe proposed reforms, lacking popular sup-port, inevitably end up not working, theed in style but not in substance. Once again,the practitioners are at fault.I will argue that the biggest problemin education is with the reformers them-selves and with the academicians and re-searchers who develop the ideas and ra-veloper of the HOTS (Higher Order Think-1 and from the historyof education reform over the past 100 years.This appears to be a time when reformis blossoming. The Kappan and other ma-jor education publications have highlight-ed dozens of reforms. Examples of current-ing, teacher empowerment, authentic as-sessment, and team teaching. Lovers ofto be on the run.But this isn’t the first instance of hyper-reform in the history of education. An-other such period ran from the mid-1960s-ly advocated reforms of that period -open space, individualization, communi-ty-based education — survived.Unfortunately, the fate of these earlierreforms is typical. The history of educa-tion reform is one of consistent failure ofmajor reforms to survive and become in-stitutionalized. David Tyack and his col-leagues have found that education reforms,becoming instead routinized incrementally structural in nature, that are easily moni-tored (e.g., the Carnegie unit), or that cre-ate new constituencies. Cuban refers to thehistorical success of attempted curriculum2 Indeed, reports of re-search on the innovations of the late 1980sand early 1990s are starting to appear andare generally disclosing failure.3Does the consistent failure of all butthe simplest reforms suggest that educa-tors are stupid, lazy, unimaginative, andin education is the same as that in other Innovation and Entrepreneur-ship, Peter Drucker draws on a wide rangeof human experience to determine the fun-damental conditions under which new ideasbecome successful and enduring innova-tions in any field. He finds that, historical-ly, the vast majority of innovative ideashave failed to take root. Most remain justinteresting ideas. Drucker arrives at thefollowing conclusions regarding the fateof new ideas:l ideas that become successful inno-vations represent a solution that is clear-ly definable, is simple, and includes a com-plete system for implementation and dis-l successful innovations start small andtry to do one specific thing; andl knowledge-based innovations are leastlikely to succeed and can succeed only ifall the needed knowledge is available. 4Drucker’s conditions for success are achronicle of human tolerance for uncer-conditions are violated by virtually everynew idea for change that is currently sweep-example, school restructuring violates allof Drucker’s principles. Process learningprinciple. Indeed, students in my educa-tional administration classes decided thatonly one current reform met all three ofDrucker's conditions — integrated socialservice centers in schools.Educational practitioners are no lessskilled in implementing innovations thanpractitioners in other professions. The faultlies in the types of reforms they are se-duced into pursuing by a reform/academ-STANLEY POGROW is an associate professor of educational administration at the Uni-versity of Arizona, Tucson, where he specializes in instructional and administrative uses of com-puters. He is the developer of the HOTS and SUPERMATH programs.JUNE 1996 657 people must be willing to invest a decadely, we need to learn that mass advocacyshould follow, not precede, the careful de-be a researcher, a reformer, or a practi-tioner. A few academicians and reform-1. For information about the HOTS program, seeStanley Pogrow, “Making Reform Work for the Ed-ucationally Fisadvantaged,”February 1995, pp. 20-24;(New York: Scholastic, 1990); “ASocratic Approach to Using Computers with Risk Students,” 1990, pp. 61-67; and “Challenging At-RiskStudents: Findings from the HOTS Program,” January 1990, pp. 389-97.For discussions of which types of innovations sur-vive, see Favid Tyack, Michael Kirst, and Elisabeth(Palo Alto, Institute for Research on Ed-ucational Finance and Governance, Stanford Uni-versity, Project Report September 1979);and Favid Tyack and William “The Gram-mar of Schooling: Why Has It Been So Hard toFall 1994, pp. 453-79. For a discussion of what hap-pens to innovations that survive, see Larry Cuban,“What Happens to Reforms That Last? The Case ofthe Junior High School,” American Summer 1992, pp. 227-51; and“The Lure of Curriculum Reform and Its Pitiful His-October 1993, pp. 182-85.3. For the failure of state-level reform, see RobertA. Frahm, “The Failure of Connecticut’s ReformPlan: Lessons for the Nation,” October 1994, pp. 156-59; and Judith and Ann-Maureen Pliska, “The Challenge to Penn-sylvania’s Education Reform,” 1994, pp. 16-21.For the failure of school restructuring and site-basedmanagement, see Jeffrey “School Reform Un-plugged: The Bensenville New American Schooland Ira E. Bogotch, “School-Level Effects of Teach-ers’ Participation in Fecision Making,” Fall 1994, pp.4. Peter F. Frucker, York HarperRow, 1985).5. Stanley Pogrow, “Where’s the Beef? Looking for1993, pp. 39-45.6. Carl F. Kaestle, “The Awful Reputation of Edu-cation Research,”ary/February 1993, pp. 23-31.7. Claude Goldenberg and Ronald Gallimore, “Lo-cal Knowledge, Research Knowledge, and Educa-ing Improvement,”vember 1991, pp. 2-14.(Chicago: University of Chicago Press, 1962).9. My research has shown that you can stimulate thedevelopment of thinking skills in educationally dis-advantaged students and so spark a wide variety ofimprovements in learning in just 35 minutes a dayof exemplary activities over a two-year period. For“Challenging At-Risk Students”; and “Con-verting At-Risk Students into Reflective Learners,”in Arthur L. Costa, James A. Bellanca, and Robin(Palatine, Ill.: Skylight Publishing, 1992).In some content fields I could not find expertacademicians who were willing to look at curricu-la and make judgments about the relative quality ofthe materials; in others, the judges panicked whenthe moment came to apply the criteria that they haddeveloped. Academicians seemed more interestedin making relativistic arguments as to why ratingscould not be done.11. It makes no practical sense to use comparisongroups to determine the large-scale effectiveness ofan intervention. A more appropriate statistical pro-cedure would be to determine the consistency of ef-fects across 50 to 100 treatment sites and simply toassume that, if consistent effects are occurring, it isa result of the intervention. However, such a studywould probably not be published in the top research12. Here’s a personal example of how researchersmisapply small-scale findings. Researchers tell me has found that students indiscussions on the use of computers reduces theirenjoyment of the technology and so reduces its po-tential for learning. Yet Socratic dialogue aboutcomputer experiences is the key element in the suc-cess of the HOTS program. While the finding citedby researchers is true over the short term, my large-scale experience has consistently found that, afterseveral months of discussing computer use, studentsexhibit far higher levels of cognitive developmentand enjoyment than the use of technology alonecould generate.13. Seymour B. (San Francisco: Jossey-Bass,pp. 37-38.14. The fall 1994 issue offeatured a special section onsite-based management/shared decision making.Two of the four articles were devoted to aspects ofconceptualizing what site-based management is. Thethird article studied the process of implementingshared decision making in six schools and foundheightened conflict with little effect on school re-form. The fourth article went beyond process andstudied the effects in one district of shared decisionmaking on student achievement and teachertice. (It was good to a researcher actually in-terested in student learning.) The fundamental con-clusion was that student achievement had not im-proved and that teachers had not changed their in-structional practices.15. Fiane Ravitch, “First, Save the Schools,”16. For examples of those recommending local cur-ricula, see Larry Cuban, “The Lure of CurricularReform and Its Pitiful History”; and Fecker Wal-court Brace Jovanovich,Question in the Chapter 1 Febate: Why Are the Stu-dents Having So Much Trouble Learning?,”26 May 1993, pp. 36, 26; and “What to Fo About Chapter 1: An Alternative Viewfrom the Street,” April 1992, pp.18. See the theory of cognitive underpinnings in“Converting At-Risk Students.”19. When it became clear that HOTS was produc-ing significant gains, I went back to the research lit-erature to see where the conclusion that thinkingshould first be developed in content had originated.Almost all the research had been conducted withhighly educated adults as subjects; it had nothing todle school.20. Steven Raudenbush, Brian Cheong, “Higher Order Instructional Goals in Sec-ondary Schools: Class, Teacher, and School Influ-Fall 1993, pp. 523-54.21. Elizabeth Fennema et al., “Using Children’sMathematical Knowledge in Instruction,” Fall 1993, pp. 22. Okhee Lee and Charles W. Anderson, “Task En-gagement and Conceptual Change in Middle SchoolScience Classrooms,”Fall 1993, pp. 23. Edward Baker, Margaret Wang, and Herbert “The Effects of Inclusion on Learning,”24. For a criticism of the tendency of the yses conducted by Robert to overlook the ef-fects of high-quality programs and thereby to gross-ly underestimate the effects of good programs, seeSusan Femirsky “Ability Grouping ResearchReviews: What Fo They Say About Grouping andthe Gifted?,” March 1991,pp. 60–65; and Stanley Pogrow, “Good StatisticsAbout Bad Programs Tell Little,”letter to the editor, October 1991, p. 93.25. Gerald W. “The Fourth Bracey Reporton the Condition of Public Education,” October 1994, pp. 26. The software is called Word Problem Proces-sors. For more information on this software, contactthe author by fax at JUNE 1996 forms of curricula in the hands of verymunity provide hope. Practitionersyoung people. The hardest thing for ateacher or principal to accept is failurethat will help them be more successful.you remove yourself from the fray for amoment to sit back and think about it,as school restructuring, using the com-students, we do not sit back.REAR also provides excuses for fail-ure. It spews out ever more esoteric, jar-plain why apparent failures aren’t reallyfailures, why it’s impossible to be moreshield the profession from outside criti-cism, while also providing rallying criesSome of the current defense mecha-nisms being deployed by REAR includeblaming society, blaming tests, or claim-who do poorly do so because of demo-beyond the control ofThis islike arguing that we shouldn’t expect tobe able to fly because gravity is beyondbillions trying to get schools to performsubstantive improvement. The membersing education as a playing field for theirturing about knowing what works whenwork on even a small scale. This rhetorichas outlived its usefulness and has be-come a self-delusional detriment to edu-cational progress. The dissemination ofresearch knowledge and inservice train-veloped. For example, my staff and I re-teaching and learning word problems inwork of courses shot through with hypestudents create their own sense of howlanguage and math go together. We de-implementation, and rethinking the roletechnologies. Such organizations woulddata on their effectiveness. This would forcefaculty members to confront the limita-One problem might arise in that col-skilled inventors, craftspeople, and tin-kerers who can put the pieces together.There are few individuals with expertisein such areas as biology, artificial intelli-which are likely to be critical for gener-ating new technologies. Moreover, veryneed to increase the diversity of skillswithin education faculties, much as wehave increased racial and sexual diversi-cists and philosophers on the faculty ofan engineering school. I wonder what aA vibrant research and reform com-munity is critical to the future of educa-tury, these communities have been in aEducation can no longer afford a re-search and academic community that isment and a too-limited traditional move-ment, each waiting for its to 20-yearturn in the limelight. It can no longer af-ford the piling on of ill-conceived move-of current ideas. The result has been in-saying that we do know what worksstudents thinking on a higher level, thathow to systematically blend the best ofments are no substitutes for better PHI FELTA KAPPAN they seek to apply their findings to, thelectual potential of educationally disad-required first developing their sense ofers have no instincts by which to judgeIt is impossible to develop a true un-er interaction from reading research. In-latest issue of the prestigious tering higher-order thinking skills. I im-mediately put aside my writing to seeers were much more likely to ask order thinking questions of higher-per-Surprise! In their tortured at-blamed tests, institutional norms, teach-ers, and so on but they ignored the obdents might not in fact know how to re-In the second article, researchers fol-ferently from the way they would haveexpected her to. As a result, the authorswas observed for the equivalent of fouramong cognitive qualities of academicEven more misleading have been theanalysis is a relatively new technique toof individual small-scale studies are ag-schoolwide models of reform has comefrom a series of meta-analyses that findabout the nature or quality of the inter-ventions that generated the results. In-for the effects of a program that he wasunaware had not been implemented andmarily weak programs and thereby se-verely underestimates the impact of ef-the repeated failure of reforms, we haveessentially shifted from a focus on out-up sides based on the process of a pro-outs” or “I am philosophically opposedall that’s left is to look good philosophi-cally while you fail.cognition in the design of reforms. Say-sider the reform breakthrough on howThat advice says nothing about what tolum, and carrying out the process all theOpposition to the mindless use of stan-Yet, despite all these problems, thethe only way to improve education JUNE 1996 The performing arts have survived andsystematize the delivery of highly cre-and individual interpretation. The sameusing the myths described above to de-For example, full inclusion is implement-ed by eliminating pullouts and treatingdents. The nature of the reforms producedThe failure of reforms produces muchtimes. For example, consider the child-centered school. In one incarnation, thethe century, the goal ofwas a major rationale for the creationof a new type of school to be called theate child-centered schools that increasesonably consistent basis. But fear not!PHI FELTA KAPPANform on a large scale, inservice trainingand dissemination strategies are largelyrecord, we keep on pushing staff devel-plementation. This lack became glaring-ly evident to me some time ago as I washim that AIFS could be cured by a par-to say that he needed to conduct scale tests over the next five to 10 yearsof the latest reform proposals for schoolwe looked silly in comparison.expresses the sameable to cure next year or 20years from now. On the contrary, it em-A current example of thisleast five years old, but there still is nowhy is everyone doing site-based man-agement and so many other current re-of adequate validation norms formary reason to adopt reforms is the fail-Unfortunately, her rationale was a litanyof the failures of the status quo: “Fewerthan half the city’s ninth-graders gradu-ate within four years. Of those who do,nearly 40% enter the City University ofall three of its tests of minimal reading,writing, and math Similarly, cur-riculum theorists point to the repeatedfailure of nationally generated curriculato argue that curricula should be devel-oped locally even though there is noevidence offered that locally developedcurricula workriodically tries progressive ideas. Whengoing on for more than a century and isthird grade. In the next-to-last reauthor-ization, the traditionalists prevailed, and succeed if it lacks an underlying technol-ogy. The individualized education movement of the 1970s is a classic example ofa reform with absolutely no technologytake today. Current REAR reforms thatclude restructuring, site-based manage-ment, full inclusion, constructivism, andideas is “paradigm shift.” The conceptof a paradigm shift can be traced to thework of Thomas Kuhn, who has docu-lations are to the evolution ofshifts are important in the evolution ofyears of tinkering with combinations ofient rallying cry that often provides asmokescreen to cover the failure to dealgenerally need to be changed. My ownexemplary activities for just a small partof the If we cannot figure out howtainly cannot figure out how to consis-We keep creating re-ing. For example, in the 1980s we becameempowering, eliminat-er lead to substantial improvements inauthentic testing are similar diversions.)school reform is a large-scale issue, andREAR is virtually ignorant about large-periods of time. Their knowledge comesNewer research techniques, such aslarge scale. REAR then convinces itselfand the profession that the knowledgegained from these small-scale investiga-While small-are not necessarily workable on a largefew classrooms, in a few schools, with afew teachers, at a few grade levels, for awill actually work on a large scale. Con-inaccurate in the case of a few individu-curate overall picture of large-scale re-over an extended period of time. Indeed,cacy of ideas. For example, instead of askscale point of view would ask whethernology to allow 80% of the sites that ex-ert a reasonable effort actually to makeJUNE 1996 659 Education reforms almost always failbinations of a number of mythsgoes like this: a sense of urgency is cre-national fellowship develops among thebelievers; stories of success appear in ajournal such as this; and a massive na-with only a smidgen of technique or re-ing, editorials, and articles cannot makeup for the absence of a powerful, yet sim-is a systematic way of doing somethinging is a waste of time.opment of child-centered curricula. I re-cently completed a three-year study toidentify exemplary middle school ula. There were few examples of exem-and the only exemplary comprehensivePHI FELTA KAPPANwere no exemplary comprehensive singular example. The reality for teach-ers and principals is that exemplary pro-grams usually do not exist for the goalsic and reform community.to developing successful programs is tohave the right metaphor. The key meta-phor in the HOTS program is the “dinnermal content, it was decided that HOTSwould not link to the regular classroomfor a Title I program. Once you have thesome of the gaps and help with perhapswhat works, if only others would applythat knowledge. For example, Carl tle found that key researchers blamed “theawful reputation of education research”to use researcheral theory. The feeling is widespread inthe REAR community that its responsi-to apply that theory. For example, in de-acy to develop the reading skills of mi-nority students, Claude Goldenberg andRonald Gallimore view the local knowl-Essentially, they argue that localedge of details will invariably be differ-erate it. I am reasonably intelligent, andtechniques for implementing a complexing and too much a hit-or-miss proposi-tion. This is not a criticism of educators.Indeed, in medicine, if individual practi-tioners invent their own procedures, wecall it “malpractice.”The equivalent of expecting teachersperform Shakespeare but to write thethose lines. The primary role of teachersis to teach, not to develop their own in-terventions because the REAR commu-nity prefers to philosophize and preach.The simple fact of the matter is thatwhat Goldenberg and Gallimore calledlocal knowledge is as important in edu-cific students need to improve their per-formance; we don’t need general philo-can learn.” My own experience is that itmine implementation details that are ap-plicable to most local conditions—ifBut the bottom line is that no amountof advocacy will cause an innovation to a decade, webeen buried in proposed orms. Those responsible for thisavalanche of reforms have takenthe perspective that there are prob-lems with the education establishment, in short, problems witheverything except reformers and theirproposed reformsReformers typically feel that their so-ety is at fault for socializing individualsin ways that prevent them from appreci-ating the wisdom of the reforms. Whenport, inevitably end up not working, theI will argue that the biggest problemin education is with the reformers them-sessment, and team teaching. Lovers ofreform in the history of education. An-ly advocated reforms of that period reforms is typical. The history of educa-Foes the consistent failure of all butthe simplest reforms suggest that educa-tors are stupid, lazy, unimaginative, anduncaring? No! The record of innovationin education is the same as that in otherareas. Inly, the vast majority of innovative ideasinteresting ideas. Frucker arrives at theideas that become successful inno-chronicle of human tolerance for uncer-of Frucker’s principles. Process learningonly one current reform met all threeEducational practitioners are no lesslies in the types of reforms they are se-JUNE 1996 Reforming the ReformersPHI FELTA KAPPAN people must be willing to invest a decadely, we need to learn that mass advocacyshould follow, not precede, the careful de-be a researcher, a reformer, or a practi-tioner. A few academicians and reform-1. For information about the HOTS program, seeStanley Pogrow, “Making Reform Work for the Ed-ucationally Fisadvantaged,”February 1995, pp. 20-24;(New York: Scholastic, 1990); “ASocratic Approach to Using Computers with Risk Students,” 1990, pp. 61-67; and “Challenging At-RiskStudents: Findings from the HOTS Program,” January 1990, pp. 389-97.For discussions of which types of innovations sur-vive, see Favid Tyack, Michael Kirst, and Elisabeth(Palo Alto, Institute for Research on Ed-ucational Finance and Governance, Stanford Uni-versity, Project Report September 1979);and Favid Tyack and William “The Gram-mar of Schooling: Why Has It Been So Hard toFall 1994, pp. 453-79. For a discussion of what hap-pens to innovations that survive, see Larry Cuban,“What Happens to Reforms That Last? The Case ofthe Junior High School,” American Summer 1992, pp. 227-51; and“The Lure of Curriculum Reform and Its Pitiful His-October 1993, pp. 182-85.3. For the failure of state-level reform, see RobertA. Frahm, “The Failure of Connecticut’s ReformPlan: Lessons for the Nation,” October 1994, pp. 156-59; and Judith and Ann-Maureen Pliska, “The Challenge to Penn-sylvania’s Education Reform,” 1994, pp. 16-21.For the failure of school restructuring and site-basedmanagement, see Jeffrey “School Reform Un-plugged: The Bensenville New American Schooland Ira E. Bogotch, “School-Level Effects of Teach-ers’ Participation in Fecision Making,” Fall 1994, pp.4. Peter F. Frucker, York HarperRow, 1985).5. Stanley Pogrow, “Where’s the Beef? Looking for1993, pp. 39-45.6. Carl F. Kaestle, “The Awful Reputation of Edu-cation Research,”ary/February 1993, pp. 23-31.7. Claude Goldenberg and Ronald Gallimore, “Lo-cal Knowledge, Research Knowledge, and Educa-ing Improvement,”vember 1991, pp. 2-14.(Chicago: University of Chicago Press, 1962).9. My research has shown that you can stimulate thedevelopment of thinking skills in educationally dis-advantaged students and so spark a wide variety ofimprovements in learning in just 35 minutes a dayof exemplary activities over a two-year period. For“Challenging At-Risk Students”; and “Con-verting At-Risk Students into Reflective Learners,”in Arthur L. Costa, James A. Bellanca, and Robin(Palatine, Ill.: Skylight Publishing, 1992).In some content fields I could not find expertacademicians who were willing to look at curricu-la and make judgments about the relative quality ofthe materials; in others, the judges panicked whenthe moment came to apply the criteria that they haddeveloped. Academicians seemed more interestedin making relativistic arguments as to why ratingscould not be done.11. It makes no practical sense to use comparisongroups to determine the large-scale effectiveness ofan intervention. A more appropriate statistical pro-cedure would be to determine the consistency of ef-fects across 50 to 100 treatment sites and simply toassume that, if consistent effects are occurring, it isa result of the intervention. However, such a studywould probably not be published in the top research12. Here’s a personal example of how researchersmisapply small-scale findings. Researchers tell me has found that students indiscussions on the use of computers reduces theirenjoyment of the technology and so reduces its po-tential for learning. Yet Socratic dialogue aboutcomputer experiences is the key element in the suc-cess of the HOTS program. While the finding citedby researchers is true over the short term, my large-scale experience has consistently found that, afterseveral months of discussing computer use, studentsexhibit far higher levels of cognitive developmentand enjoyment than the use of technology alonecould generate.13. Seymour B. (San Francisco: Jossey-Bass,pp. 37-38.14. The fall 1994 issue offeatured a special section onsite-based management/shared decision making.Two of the four articles were devoted to aspects ofconceptualizing what site-based management is. Thethird article studied the process of implementingshared decision making in six schools and foundheightened conflict with little effect on school re-form. The fourth article went beyond process andstudied the effects in one district of shared decisionmaking on student achievement and teachertice. (It was good to a researcher actually in-terested in student learning.) The fundamental con-clusion was that student achievement had not im-proved and that teachers had not changed their in-structional practices.15. Fiane Ravitch, “First, Save the Schools,”16. For examples of those recommending local cur-ricula, see Larry Cuban, “The Lure of CurricularReform and Its Pitiful History”; and Fecker Wal-court Brace Jovanovich,Question in the Chapter 1 Febate: Why Are the Stu-dents Having So Much Trouble Learning?,”26 May 1993, pp. 36, 26; and “What to Fo About Chapter 1: An Alternative Viewfrom the Street,” April 1992, pp.18. See the theory of cognitive underpinnings in“Converting At-Risk Students.”19. When it became clear that HOTS was produc-ing significant gains, I went back to the research lit-erature to see where the conclusion that thinkingshould first be developed in content had originated.Almost all the research had been conducted withhighly educated adults as subjects; it had nothing todle school.20. Steven Raudenbush, Brian Cheong, “Higher Order Instructional Goals in Sec-ondary Schools: Class, Teacher, and School Influ-Fall 1993, pp. 523-54.21. Elizabeth Fennema et al., “Using Children’sMathematical Knowledge in Instruction,” Fall 1993, pp. 22. Okhee Lee and Charles W. Anderson, “Task En-gagement and Conceptual Change in Middle SchoolScience Classrooms,”Fall 1993, pp. 23. Edward Baker, Margaret Wang, and Herbert “The Effects of Inclusion on Learning,”24. For a criticism of the tendency of the yses conducted by Robert to overlook the ef-fects of high-quality programs and thereby to gross-ly underestimate the effects of good programs, seeSusan Femirsky “Ability Grouping ResearchReviews: What Fo They Say About Grouping andthe Gifted?,” March 1991,pp. 60–65; and Stanley Pogrow, “Good StatisticsAbout Bad Programs Tell Little,”letter to the editor, October 1991, p. 93.25. Gerald W. “The Fourth Bracey Reporton the Condition of Public Education,” October 1994, pp. 26. The software is called Word Problem Proces-sors. For more information on this software, contactthe author by fax at JUNE 1996 forms of curricula in the hands of verymunity provide hope. Practitionersyoung people. The hardest thing for ateacher or principal to accept is failurethat will help them be more successful.you remove yourself from the fray for amoment to sit back and think about it,as school restructuring, using the com-students, we do not sit back.REAR also provides excuses for fail-ure. It spews out ever more esoteric, jar-plain why apparent failures aren’t reallyfailures, why it’s impossible to be moreshield the profession from outside criti-cism, while also providing rallying criesSome of the current defense mecha-nisms being deployed by REAR includeblaming society, blaming tests, or claim-who do poorly do so because of demo-beyond the control ofThis islike arguing that we shouldn’t expect tobe able to fly because gravity is beyondbillions trying to get schools to performsubstantive improvement. The membersing education as a playing field for theirturing about knowing what works whenwork on even a small scale. This rhetorichas outlived its usefulness and has be-come a self-delusional detriment to edu-cational progress. The dissemination ofresearch knowledge and inservice train-veloped. For example, my staff and I re-teaching and learning word problems inwork of courses shot through with hypestudents create their own sense of howlanguage and math go together. We de-implementation, and rethinking the roletechnologies. Such organizations woulddata on their effectiveness. This would forcefaculty members to confront the limita-One problem might arise in that col-skilled inventors, craftspeople, and tin-kerers who can put the pieces together.There are few individuals with expertisein such areas as biology, artificial intelli-which are likely to be critical for gener-ating new technologies. Moreover, veryneed to increase the diversity of skillswithin education faculties, much as wehave increased racial and sexual diversi-cists and philosophers on the faculty ofan engineering school. I wonder what aA vibrant research and reform com-munity is critical to the future of educa-tury, these communities have been in aEducation can no longer afford a re-search and academic community that isment and a too-limited traditional move-ment, each waiting for its to 20-yearturn in the limelight. It can no longer af-ford the piling on of ill-conceived move-of current ideas. The result has been in-saying that we do know what worksstudents thinking on a higher level, thathow to systematically blend the best ofments are no substitutes for better PHI FELTA KAPPAN they seek to apply their findings to, thelectual potential of educationally disad-required first developing their sense ofers have no instincts by which to judgeIt is impossible to develop a true un-er interaction from reading research. In-latest issue of the prestigious tering higher-order thinking skills. I im-mediately put aside my writing to seeers were much more likely to ask order thinking questions of higher-per-Surprise! In their tortured at-blamed tests, institutional norms, teach-ers, and so on but they ignored the obdents might not in fact know how to re-In the second article, researchers fol-ferently from the way they would haveexpected her to. As a result, the authorswas observed for the equivalent of fouramong cognitive qualities of academicEven more misleading have been theanalysis is a relatively new technique toof individual small-scale studies are ag-schoolwide models of reform has comefrom a series of meta-analyses that findabout the nature or quality of the inter-ventions that generated the results. In-for the effects of a program that he wasunaware had not been implemented andmarily weak programs and thereby se-verely underestimates the impact of ef-the repeated failure of reforms, we haveessentially shifted from a focus on out-up sides based on the process of a pro-outs” or “I am philosophically opposedall that’s left is to look good philosophi-cally while you fail.cognition in the design of reforms. Say-sider the reform breakthrough on howThat advice says nothing about what tolum, and carrying out the process all theOpposition to the mindless use of stan-Yet, despite all these problems, thethe only way to improve education JUNE 1996 The performing arts have survived andsystematize the delivery of highly cre-and individual interpretation. The sameusing the myths described above to de-For example, full inclusion is implement-ed by eliminating pullouts and treatingdents. The nature of the reforms producedThe failure of reforms produces muchtimes. For example, consider the child-centered school. In one incarnation, thethe century, the goal ofwas a major rationale for the creationof a new type of school to be called theate child-centered schools that increasesonably consistent basis. But fear not!PHI FELTA KAPPANform on a large scale, inservice trainingand dissemination strategies are largelyrecord, we keep on pushing staff devel-plementation. This lack became glaring-ly evident to me some time ago as I washim that AIFS could be cured by a par-to say that he needed to conduct scale tests over the next five to 10 yearsof the latest reform proposals for schoolwe looked silly in comparison.expresses the sameable to cure next year or 20years from now. On the contrary, it em-A current example of thisleast five years old, but there still is nowhy is everyone doing site-based man-agement and so many other current re-of adequate validation norms formary reason to adopt reforms is the fail-Unfortunately, her rationale was a litanyof the failures of the status quo: “Fewerthan half the city’s ninth-graders gradu-ate within four years. Of those who do,nearly 40% enter the City University ofall three of its tests of minimal reading,writing, and math Similarly, cur-riculum theorists point to the repeatedfailure of nationally generated curriculato argue that curricula should be devel-oped locally even though there is noevidence offered that locally developedcurricula workriodically tries progressive ideas. Whengoing on for more than a century and isthird grade. In the next-to-last reauthor-ization, the traditionalists prevailed, and succeed if it lacks an underlying technol-ogy. The individualized education movement of the 1970s is a classic example ofa reform with absolutely no technologytake today. Current REAR reforms thatclude restructuring, site-based manage-ment, full inclusion, constructivism, andideas is “paradigm shift.” The conceptof a paradigm shift can be traced to thework of Thomas Kuhn, who has docu-lations are to the evolution ofshifts are important in the evolution ofyears of tinkering with combinations ofient rallying cry that often provides asmokescreen to cover the failure to dealgenerally need to be changed. My ownexemplary activities for just a small partof the If we cannot figure out howtainly cannot figure out how to consis-We keep creating re-ing. For example, in the 1980s we becameempowering, eliminat-er lead to substantial improvements inauthentic testing are similar diversions.)school reform is a large-scale issue, andREAR is virtually ignorant about large-periods of time. Their knowledge comesNewer research techniques, such aslarge scale. REAR then convinces itselfand the profession that the knowledgegained from these small-scale investiga-While small-are not necessarily workable on a largefew classrooms, in a few schools, with afew teachers, at a few grade levels, for awill actually work on a large scale. Con-inaccurate in the case of a few individu-curate overall picture of large-scale re-over an extended period of time. Indeed,cacy of ideas. For example, instead of askscale point of view would ask whethernology to allow 80% of the sites that ex-ert a reasonable effort actually to makeJUNE 1996 659 Education reforms almost always failbinations of a number of mythsgoes like this: a sense of urgency is cre-national fellowship develops among thebelievers; stories of success appear in ajournal such as this; and a massive na-with only a smidgen of technique or re-ing, editorials, and articles cannot makeup for the absence of a powerful, yet sim-is a systematic way of doing somethinging is a waste of time.opment of child-centered curricula. I re-cently completed a three-year study toidentify exemplary middle school ula. There were few examples of exem-and the only exemplary comprehensivePHI FELTA KAPPANwere no exemplary comprehensive singular example. The reality for teach-ers and principals is that exemplary pro-grams usually do not exist for the goalsic and reform community.to developing successful programs is tohave the right metaphor. The key meta-phor in the HOTS program is the “dinnermal content, it was decided that HOTSwould not link to the regular classroomfor a Title I program. Once you have thesome of the gaps and help with perhapswhat works, if only others would applythat knowledge. For example, Carl tle found that key researchers blamed “theawful reputation of education research”to use researcheral theory. The feeling is widespread inthe REAR community that its responsi-to apply that theory. For example, in de-acy to develop the reading skills of mi-nority students, Claude Goldenberg andRonald Gallimore view the local knowl-Essentially, they argue that localedge of details will invariably be differ-erate it. I am reasonably intelligent, andtechniques for implementing a complexing and too much a hit-or-miss proposi-tion. This is not a criticism of educators.Indeed, in medicine, if individual practi-tioners invent their own procedures, wecall it “malpractice.”The equivalent of expecting teachersperform Shakespeare but to write thethose lines. The primary role of teachersis to teach, not to develop their own in-terventions because the REAR commu-nity prefers to philosophize and preach.The simple fact of the matter is thatwhat Goldenberg and Gallimore calledlocal knowledge is as important in edu-cific students need to improve their per-formance; we don’t need general philo-can learn.” My own experience is that itmine implementation details that are ap-plicable to most local conditions—ifBut the bottom line is that no amountof advocacy will cause an innovation to a decade, webeen buried in proposed orms. Those responsible for thisavalanche of reforms have takenthe perspective that there are prob-lems with the education establishment, in short, problems witheverything except reformers and theirproposed reformsReformers typically feel that their so-ety is at fault for socializing individualsin ways that prevent them from appreci-ating the wisdom of the reforms. Whenport, inevitably end up not working, theI will argue that the biggest problemin education is with the reformers them-sessment, and team teaching. Lovers ofreform in the history of education. An-ly advocated reforms of that period reforms is typical. The history of educa-Foes the consistent failure of all butthe simplest reforms suggest that educa-tors are stupid, lazy, unimaginative, anduncaring? No! The record of innovationin education is the same as that in otherareas. Inly, the vast majority of innovative ideasinteresting ideas. Frucker arrives at theideas that become successful inno-chronicle of human tolerance for uncer-of Frucker’s principles. Process learningonly one current reform met all threeEducational practitioners are no lesslies in the types of reforms they are se-JUNE 1996 Reforming the ReformersPHI FELTA KAPPAN