Always Crashing in the Same Car – The Myth of Innovation by Procurement

Two years ago, the Los Angeles Unified School District (LAUSD) began implementing a $1.3 billion plan to put an Apple iPad, preloaded with curriculum from Pearson, into the hands of every student Lapowsky, I. (2015, May 8). What Schools Must Learn From LA’s iPad Debacle.. “This is going to level the playing field as far as what schools are doing throughout the district,” claimed Cynthia Williams, principal of a district school Blume, H. (2013b, August 27). LAUSD launches its drive to equip every student with iPads.. Others were critical of the plan from the start. Issues included the price per iPad ($678 USD, more than in stores), the fact that Pearson’s curriculum software would not be completed before delivery and the district’s repeated preference for investment in technology rather than teachers Blume, H. (2013a, June 18). L.A. school board OKs $30 million for Apple iPads. Los Angeles Times..

Problems began to materialize before a single iPad was delivered. Shortly after the original contract was signed, Apple unveiled a new iPad model Blume, H. (2014, August 25). L.A. Unified halts contract for iPads.. Superintent John Deasy, the project’s champion, diffused this situation by negotiating a change to the contract. Technical issues also began to emerge, including student-reported difficulties when trying to input test answers. The ethics of the procurement process were then challenged as emails revealed Deasy and his deputy, James Aquino, had been in contact with executives at Apple and Pearson at least two years before the contract was signed Blume, H. (2014, August 25). L.A. Unified halts contract for iPads..

The problems kept coming. By the end of 2014, Aquino and Deasy had resigned and the FBI had opened an investigation into the program Geuss, M. (2015, September 28). Los Angeles schools reach $6.4 million settlement with Apple, Lenovo.. This past spring, the LAUSD, citing “crippling technical issues with the Pearson platform”, threatened Apple with legal action if a refund wasn’t forthcoming Lapowsky, I. (2015, May 8). What Schools Must Learn From LA’s iPad Debacle.. An internal memo suggested that at this point, Pearson curriculum had largely been abandoned, as it could only be reliably accessed by 5% of students Lapowsky, I. (2015, May 8). What Schools Must Learn From LA’s iPad Debacle.. The contract has since been cancelled and to date, $6.4 million has been promised to the school district Geuss, M. (2015, September 28). Los Angeles schools reach $6.4 million settlement with Apple, Lenovo..

According to a spokesperson, the LAUSD is “still very much moving forward in technology and continuing to deliver devices to schools.” The school board has approved another $40 million for technology purchases, including more iPads Lapowsky, I. (2015, May 8). What Schools Must Learn From LA’s iPad Debacle..

The Troubling History of Educational Technology

In 1987 the economist and soon-to-be Nobel-laureate Robert Solow quipped “we see computers everywhere except in the productivity statistics” Brynjolfsson, E. (1993). The Productivity Paradox of Information Technology. Communications of the ACM, 36(12), 67–77.. Ever since, people have referred to the counter-intuitive fact that information technology does not produce productivity improvements as the Solow Paradox. Education is not immune to this paradox Fahy, P. (1998). Reflections on the Productivity Paradox and Distance Education Technology. International Journal of E-Learning & Distance Education, 13(2), 66–73.. Yet the LAUSD is far from unique in its enthusiasm for technology. Larry Cuban, Professor Emeritus of Education at Stanford University, has described how the advent of film, radio and television media all led to claims of impending revolution to the field of education; in each case, expectations were dramatically overstated Cuban, L. (2003). Oversold and Underused. Cambridge, Mass.: Harvard University Press.. For example, speaking in 1922, Thomas Edison suggested “the motion picture is destined to revolutionize our educational system and that in a few years it will supplant largely, if not entirely, the use of textbooks” Watters, A. (2015, February 19). The History of the Future of Education..

Emerging technologies and ideas tend to follow the Hype Cycle model developed by Gartner Inc. As technologies mature they move through the following stages: Technology Trigger, Peak of Inflated Expectations, Trough of Disillusionment, Slope of Enlightenment, and Plateau of Productivity Fenn, J., & Raskino, M. (2008). Mastering the Hype Cycle: How to Choose the Right Innovation at the Right Time. Boston, Mass: Harvard Business Review Press..

Those familiar with the history of educational technology would not be surprised to hear that the LAUSD iPad program has fallen from the peak of inflated expectations into the trough of disillusionment. Nor would they be surprised to learn that the new superintendent intends to double down. Shortly after the original deal, Cuban pointed out that there is no body of evidence suggesting the use of iPads (or any computer) increases math or reading scores on state standardized tests. Furthermore, there is no evidence suggesting the use of iPads (or computers) in class increases the likelihood of attaining decent paying jobs upon graduation Cuban, L. (2013, June 21). iPads in Los Angeles and TCO..

For most institutions … new technologies represent a black hole of additional expense. Most campuses have simply bolted new technologies onto a fixed plant, a fixed faculty, and a fixed notion of classroom instruction – Carol Twigg Twigg, C. A. (2003). Improving Learning and Reducing Costs: New Models for Online Learning. EDUCAUSE Review, (5).

Deasy and Aquino may well have had students’ interests at heart when they championed the iPad project. With one of the most diverse student bodies in the nation, many district teachers relished the opportunity to provide students with access to cutting edge technology and a more personalized learning experience. The teacher could focus on the students most in need of support. Students more comfortable with the subject could move on to more challenging material. But the history of educational technology demonstrates personalized learning has been a mirage for almost 100 years. To rush into such an ambitious program, providing teachers with only three days of training, using curriculum software that was still in development reveals a serious lapse in judgment Blume, H. (2013b, August 27). LAUSD launches its drive to equip every student with iPads..

Deasy & Aquino are not alone in their uncritical approach to technology. Clayton Christensen, a Harvard business professor and the author of the theory of disruptive innovation, claimed in 2013 that half of all US universities would be bankrupt in 15 years Suster, M. (2013, March 3). In 15 Years From Now Half of US Universities May Be in Bankruptcy. My Surprise Discussion with @ClayChristensen.. Sebastian Thrun, Stanford professor, AI researcher, and massive open online course (MOOC) innovator, suggested that in 50 years there would be only 10 universities and that his company, Udacity, had a shot at being one of them Leckart, S. (2012, March 20). The Stanford Education Experiment Could Change Higher Learning Forever.. Educational technology writer Audrey Watters has noted these types of predictions border on “end of the world” millennialism Watters, A. (2013, May 24). The Myth and the Millennialism of “Disruptive Innovation.”.

Claims of impending revolution are not the only messages that are repeated and accepted uncritically. Jose Ferreira, the founder and CEO of ed tech company Knewton, recently described his product as “a robot tutor in the sky that can semi-read your mind and figure out what your strengths and weaknesses are, down to the percentile”; a claim educational technology consultant and writer Michael Feldstein has likened to “selling snake oil” Westervelt, E. (2015, October 13). Meet The Mind-Reading Robo Tutor In The Sky; Feldstein, M. (2015, October 13). Yes, I did say that Knewton is “selling snake oil.”. Given such claims, it’s not hard to see why expectations often become inflated.

Despite the work of Watters, Cuban and others (see Ellen Lagemann, Francis Toyama), the revolutionary claims of disruptive innovation theorists and ed tech entrepreneurs have been uncritically parroted by the media. The temporary ousting of Teresa Sullivan, president of the University of Virginia, is an example of how keenly the disruptive potential of technology is felt. Members of the Board of Visitors that staged the coup felt she was not doing enough to embrace technology (i.e., MOOCs) as a form of cost savings and a hedge against disruptive innovation Rice, A. (2012, September 11). What the Failed Removal of UVA President Teresa Sullivan Means for Higher Education. The New York Times. Retrieved from http://www.nytimes.com/2012/09/16/magazine/teresa-sullivan-uva-ouster.html
.

Our faith in technology is steadfast. The LAUSD is again moving forward with technology procurements including iPads. But are we learning from our mistakes? Is there a reason why technology-driven projects in education fail to live up to their hype? Are there successful projects from which we can learn?

Redesign versus Procurement

Carol Twigg, head of the National Center for Academic Transformation (NCAT), has been exploring the potential of technology to make education more effective and efficient for over two decades. Twigg recently implemented a developmental math redesign using commercially available instructional software. While soliciting institutional partners for the project, Twigg was forthright about her expectations.

At the start, we said that if institutions followed our advice—derived from the successes achieved in past course-redesign programs in developmental and college-level mathematics—we could guarantee that they would improve student learning, increase completion of the developmental-math sequence, prepare students to succeed in college-level math, and reduce instructional costs.

 

And that is exactly what happened. – Carol Twigg Twigg, C. A. (2013). Improving learning and reducing costs: outcomes from changing the equations. Change, (4), 6.

Care was taken to not inflate expectations. Twigg’s redesign model is positioned squarely at the plateau of productivity. Working with 38 institutions and over 100,000 students, Twigg and her team redesigned developmental math programs to improve effectiveness and efficiency. 83% of programs showed significant improvements to learning as measured by comparing common final exam scores, common exam items, gains on pre / post tests between traditional and redesigned formats. 97% of programs showed significant savings of on average 20%. Twigg’s redesign involved moving classes to computer labs, where commercially available instructional software (ALEKS, Hawkes Learning System, MyMathLab) guided students at their own pace through a modularized curriculum. Students had to master each unit (75-90%) in order to move on. The cost savings were realized by increased section sizes, increased number of sections per faculty member, and in some cases, the sharing of computer lab space across courses Twigg, C. A. (2013). Improving learning and reducing costs: outcomes from changing the equations. Change, (4), 6.. Far from being a technology procurement exercise, Twigg’s redesign altered the technology, geography, pedagogy, organization and responsibilities of both teacher and student.

Twigg and her team provided detailed guidance on every aspect of the course’s design. Institutions that could not commit to this level of change had to withdraw.

Despite repeated advice from both NCAT staff and the Redesign Scholars, a number of projects failed to do such things as require lab participation, award participation points as an incentive for student engagement, establish deadlines and clear expectations, monitor students’ progress and intervene when they were not meeting deadlines, and so on. Six of the original 38 institutions withdrew due to an inability to meet the program’s requirements. – Carol Twigg Twigg, C. A. (2013). Improving learning and reducing costs: outcomes from changing the equations. Change, (4), 6.

Although the technology used to deliver the instruction is new, the pedagogy guiding its use is nearly 50 years old. In 1968, Benjamin Bloom, professor of education at the University of Chicago, published a paper called Learning for Mastery in which he argued that in most cases students’ aptitude predicts the rate at which a student can learn a subject, rather than their ability to learn it. Bloom advocated frequent testing designed to reveal areas requiring remediation. Upon completion of a test not meeting the mastery threshold, further instruction and time to review was recommended before the test was run again. Students only proceed when they have mastered the content according to a decided upon threshold Bloom, B. S. (1968). Learning for Mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints, Number 1..

We are convinced that the grade of A as an index of mastery of a subject can, under appropriate conditions, be achieved by up to 95 percent of students in a class. – Benjamin Bloom Bloom, B. S. (1968). Learning for Mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints, Number 1..

That same year, Fred S. Keller, professor of psychology at Columbia University, published a paper called Good-bye Teacher, in which he outlined a novel method for the administration of courses. He described modularized, self-paced courses wherein students must master one concept before being allowed to proceed to the next. The instructor’s responsibilities are delegated in part to undergraduate and graduate assistants. Students are tested and receive feedback after each unit. If their performance falls below the mastery threshold, they repeat the test when ready with no penalty Keller, F. S. (1968). Good-bye, teacher… Journal of Applied Behavior Analysis, 1(1), 79.. Despite demonstrating promise as an improvement on the traditional lecture, PSI failed to move from the fringes to mainstream practice Grant, L. K., & Spencer, R. E. (2003). The Personalized System of Instruction: Review and Applications to Distance Education. The International Review of Research in Open and Distributed Learning, 4(2)..

Our Bias Towards Technological Innovation

Twigg’s “Changing the Equation” redesign relied on both new instructional technology and an older unorthodox pedagogy. Rather than an exercise in procurement, it was a root and branch transformation of the existing course. Put simply, “CTE scaled up a proven innovation” Twigg, C. A. (2013). Improving learning and reducing costs: outcomes from changing the equations. Change, (4), 6.. Twigg is referring to the whole course redesign model she has previously implemented at a smaller scale. But a good deal of that model depends on pedagogical techniques developed nearly 50 years ago and shelved soon thereafter.

Is it possible that the theory of disruptive innovation itself has now entered the trough of disillusionment of Gartner’s hype cycle? George Siemens, professor, writer and an influential theorist on education and technology, recently published a popular blog post called Adios Ed-Tech, Hello Something Else. In his post, Siemens writes, “I no longer want to be affiliated with the tool-fetish of edtech” Siemens, G. (2015, September 9). Adios Ed Tech. Hola something else..

Siemens was an organizing committee member of the recent DLRN2015 conference. Tweets from the conference reflect a shared frustration with the disruption narrative. Matt Crosslin, a co-organizer and research at University of Texas Arlington and Laura Gogia of Virginia Commonwealth University had this exchange:

DLRN Co-organizer, Candace Thille, is a professor at Stanford’s Graduate School of Education, a redesign scholar at NCAT and a longtime researcher of online learning. She too has expressed frustration with the hype surrounding educational technology:

Phil Hill, a blogger and ed tech analyst at e-Literate and MindWires consulting, has recently described how the theory of disruptive innovation is being inappropriately generalized. He also summarized recent research suggesting flaws in Christensen’s original case study analysis Hill, P. (2015, October 6). Cracks In The Foundation Of Disruptive Innovation..

Researchers and analysts are increasingly frustrated by the mainstream narrative of innovation by procurement. To those familiar with history of ed tech, good intentions don’t make up for the continued waste of money and students’ time that accompanies the boom and bust as technology expectations are right-sized. Who fact-checks the claims of private profit-seeking companies providing the “innovative” products? President Sullivan’s mercifully temporary ousting at the University of Virginia starkly illustrates how the media contributes uncritically to the pressure on institutions to adopt untested technology. Why are educational institutions loath to alter practice unless it accompanies new hardware or software? How many process innovations, like those developed by Keller and Bloom, are ignored, overshadowed by flashier tools that promise results without the accompanying pain of root and branch redesign? Though the cases presented above raise many questions, one point is clear – by now, we should know better.

References

    About Jeff Pinto

    Educational Technology consultant, developer and researcher.

    Leave a Reply