We are working on educational technology. That is, technology to assist in education. More specifically, we are developing software that helps people learn. There are many types of such software. We are most immediately focused on two such types.
- adaptive educational technology for personalized learning
- cognitive tutors
The term “adaptive” with regard to educational technology has various interpretations. Educational technology that adapts to individuals in any of various ways is the most common interpretation of adaptive educational technology. This interpretation is a form of personalized learning. Personalized learning is often considered a more general term which includes human tutors who adapt how they engage with and educate learners. In the context of educational technology, these senses of adaptive and personalized learning are synonymous.
Another sense of “adaptive” with regard to educational technology refers to the internal workings of the educational technology itself. This sense refers to the use of machine learning, data mining, and so forth within the educational technology. This sense is a technical detail concerning how personalized learning is enabled by technology. It is used primarily by technologists who work in the space of adaptive educational technology. In the broader market of education, adaptive education refers to technology-enabled personalized learning.
Adaptive education is an important objective that is transforming education. It is the most significant disruptive force impacting education today. And that is no small statement. Behind healthcare, education is the largest market, by far. Estimates vary from $3T to $5T for North America versus globally. That’s ‘$T’ for trillions of dollars per year! Small wonder that the front-runner in adaptive educational technology, Knewton, has raised over $100M.
Given its economic weight it may seem disingenuous to also note the altruism in advancing education. Nonetheless, it is fair and reasonable to draw some sense of purpose and satisfaction from such efforts despite any extent to which greed is not good!
Any such altruistic pride, however, may be diminished by the state of the art. The fact is that adaptive education is in a most rudimentary state. The benefits of adaptive education are “enjoyed” by the largest traditional textbook publishers more than the global community of educators. And the benefits of adaptive education in terms of advancing education are quite limited when viewed in terms of how much they accelerate or increase learning or the extent to which they are doing so globally.
The limited impact of technology on learning arises from multiple factors. For example, the publisher of a textbook with significant market share is naturally disinclined to embrace the disruptive innovation of adaptive technology for personalized learning. Such a publisher is more likely to advance incrementally so as to minimize loss of market share as educational resources become increasingly commoditized, especially in the form of open textbooks.
It is not entirely fair, however, to impugn traditional publishers. Harnessing the state of the art in adaptive educational technology is expensive. Given the high cost of developing adaptive educational solutions, publishers are in the best position to realize a return on their investment and are prudent to do so incrementally.
As is common with disruptive technology and innovation, the promise of electronically-enhanced learning is limited by its present cost and will accelerate as those costs fall. This begs the question, of course, as to what those costs are and how they might fall over time. There are other questions, too, such as whether the state of the art is sufficient for immediate and disruptive transformation of education and how the art may advance over the near future so as to more fully realize transformative benefits.
With regard to present obstacles to realizing the potential for electronically-enhanced learning to benefit humanity, there are two immediate issues: access to sufficient technology and the cost of engineering educational software that adaptively improves individuals’ learning.
Apparently, Knewton has the best technology for incrementally improving learning outcomes across the extremely broad front of education. Knewton’s technology is based on various well-understood approaches within the literature. Such approaches include various psychometric models, such as item response theory (IRT), models of learning styles and modes of engagement, and various approaches to machine learning, such as Bayesian networks, Markov random fields, clustering and so on.
Aside from Knewton, whose blog and whitepapers describe their approaches in commendable detail, other adaptive technology is either inaccessible, captive, or immature. Knewton provides its technology as a service that can be leveraged by almost any delivery platform. Knewton is not in the authoring or delivery platform business. They are focused on delivering the essential ingredient of electronically personalized learning.
McGraw-Hill has good technology, such the recently acquired “Knowledge Spaces” of ALEKS and other technology acquired with Area 9. McGraw-Hill’s Smart Books are particularly impressive and more polished than (although not as deep or forward-looking as) the Inquire prototype developed in Vulcan’s Project Halo. It seems reasonable to expect that this technology will be somewhat limited to publications and affiliates of McGraw-Hill, however.
There are other players, such as SnapWiz and CCKF who have adaptive technology. Those offerings incorporate authoring and delivery, unlike Knewton. CCKF, branded as RealizeIt, also touts the availability of its technology as a web service, in direct competition with Knewton. How their technology stacks up against Knewton is not clear from their literature, however.
Carnegie-Mellon is a hot-bed of activity in the development of the necessary technology, including “Knowledge Tracing” and Learning or Performance Factor Analysis, which are more advanced than the commercial offerings and more promising for improving learning outcomes in more complex domains and higher education. There are too many notable academics and too much research to review here, though. Suffice it to say that the technology for modeling learners and learning is sufficiently mature that various approaches perform in the same ballpark and none is far ahead of Knewton in practice.
So even if Knewton isn’t perfect, it’s certainly good enough for now. There is nothing in Knewton’s limitations that are holding us back from more fully realizing the benefits of electronically-enhanced learning. Present limitations on realizing such benefits are more pedagogical than technological.
The pedagogical problem is that electronically enhancing learning requires the electronics to know something about how to assess and teach its subject matter. All of the technologies available commercially and in the literature require a model of the skills to be learned and applied in solving problems or answering questions. This is the problem that you typically don’t see addressed on vendors’ web sites but it’s the clear focus of much of the research literature.
The problem boils down to this: electronically enhanced learning technology needs to know what it’s trying to achieve and how to measure and advance progress towards desired outcomes. This involves formulating learning objectives and how to measure the degree to which they are mastered and how to improve learners’ mastery of them.
To boil it down further: the real problem in realizing the benefits of electronically enhancing learning is formulating and relating learning objectives and educational resources, including instructional content and assessment items. Knewton calls this “graphing” and the result is a so-called “knowledge graph”. Such graphs are not unique to Knewton, however. They are core and explicitly documented in the literature and by ALEKS. They are less evident but nonetheless clear in the cases of most other vendors.
Building these graphs is our present focus, although we have a longer term strategy. As you might expect, we are applying automation and curation to understand educational resources and assist in formulating and relating learning objectives with educational resources that assess and/or teach them.
Consider the simple question:
- Is it true that the head of the phospholipid is polar?
What does an electronic tutor need to understand about this in order to help educate people?
This is an example of the problem of formulating learning objectives. There are multiple challenges, such as granularity versus complexity of the undertaking. Do we formulate a learning objective concerning phospholipids? Do we formulate a learning objective concerning the sense of “polar” used here? If so, we get into lots of other knowledge concerning phospholipids and what it means for a molecule to be polar. Our decisions about this will be guided by available educational resources which may change over time. In this case, the explanation for this question is given as follows:
- The heads of the phospholipids are hydrophilic, which means “water loving”. Background knowledge from chemistry is that water is a polar molecule, and that like solutes dissolve in like solvents. Therefore if the heads are “water loving”, they must also be polar.
There is a bias in this explanation away from simply “knowing” the answer. This explanation assumes the learner thinks about the implications of phospholipids being hydrophilic. This suggests that the learning objective is more about understanding what it means to be hydrophilic and the consequences or implications of being hydrophilic with regard to polarity.
So we have a choice, we can formulate lots of learning objectives to cover much if not most of the knowledge within our content. Alternatively, we can formulate learning objectives with respect to our assessment items (e.g., questions), focusing on what must be learned in order to correctly answer. In practice, this is what is done, although not in great detail. For example, in this case, someone reduced all this content to possessing cognitive skill with regard to:
- structure and property of part of molecule
In my opinion, this is a gross learning objective, in several senses of the word. We might be able to produce a graphic dashboard showing the distribution of mastery concerning this learning objective based on scores on assessment items linked to it, but that graphic will not be very meaningful and the electronics is not going to directly advance mastery of this concept without more detailed learning objectives. A better learning objective would be:
- Understand the role of lipid structure and polarity in cell membranes.
This is just the tip of the iceberg on the cognitive modeling problem that is the real bottleneck in realizing the promise of electronically enhanced education.