The Death of Testing and the Rise of Learning Analytics

I know that it is sad news for some, but more than a few of us have assessed the situation, and the prognosis is not good for our friend (or perhaps the arch enemy to others of us), the test. We might be witnessing the death of testing. Tests are not going away tomorrow or even next year, but their value will fade over the upcoming years until, finally, tests are, once and for all, a thing of the past. At least that is one possible future.

Tests are largely a 20th century educational technology that had no small impact on learning organizations around the world, not to mention teachers and students. They’ve increased anxiety, kept people up all night (often with the assistance of caffeine), and consumed large chunks of people’s formative years.

They’ve also made people lots of money. There are the companies that help create and administer high-stakes tests. There are the-the companies that created those bubble tests and the machines that grade them. There are the test proctoring companies along with the many others that have created high-tech ways to prevent and/or detect cheating on tests. There are the test preparation companies. There are even researchers who’ve done well as consultants, helping people to design robust, valid and reliable tests. Testing is a multi-billion dollar industry.

death of testingGiven this fact, why am I pointing to the death of the test? It is because of the explosion of big data, learning analytics, adaptive learning technology, developments around integrated assessments in games and simulations and much more. These technologies are making and will continue to make it possible to constantly monitor learner progress. Assessment will be embedded in the learning experiences. When you know how a student is making progress and exactly where that student is in terms of reaching a given goal, why do you need a test at the end? The student doesn’t even need to know that it is happening, and the data can be incredibly rich, giving insights and details often not afforded by traditional tests.

Such embedded assessment is the exception today, but not for long. That is why many testing companies and services are moving quickly into the broader assessment space. They realize that their survival depends upon their capacity to integrate in seamless ways with content, learning activities and experiences, simulations and learning environments. This is also why I have been urging educational publishing companies to start investing in feedback and assessment technologies. This is going to critical for their long-term success.

At the same time, I’m not convinced that all testing will die. Some learning communities will continue to use them even if they are technically unnecessary. Tests still play a cultural role in some learning contexts. My son is in martial arts and the “testing day” is an important and valued benchmark in community. Yes, there are plenty of other ways to assess, but the test is part of the experience in this community. The same is true in other learning contexts. Testing is not always used because it is the best way to measure learning. In these situations, testing will likely remain a valued part of the community. In some ways, however, this helps to make my point. Traditional testing is most certainly not the best or most effective means of measuring learning today. As the alternatives expand and the tools and resources for these alternatives become more readily available, tests will start the slow but certain journey to the educational technology cemetery, finding a lot alongside the slide rule and the overhead projector.

Tagged : / / /

Educational Publishers & Content Providers: The Future is About Analytics, Feedback & Assessment

What is the future of educational publishers and content providers? As more content becomes freely distributed online and there are more creative (and sometimes free) products and services that help aggregate, curate, chunk, edit and beautify this content; there are questions about the role of educational publishers and content providers. While there is something to be said for a one-stop-shop for content, that might not be enough to secure a solid spot in the marketplace of the future, especially given that content is not the only thing for which people are shopping.

Some fear or simply predict the demise of such groups, but I expect a long and vibrant future. In fact, over the past decade or two, we’ve already witnessed publishing companies rebrand themselves as education companies with a broader portfolio of offerings than ever before. They’ve done so by adding experts in everything from educational psychology and brain research to instructional design, software development to game design, educational assessment to statistics, analytics, and testing. These are exactly the types of moves that will help them establish, maintain, and extend their role in the field of education. This is a shift from a time when many educational publishers and content providers would suggest that it is best to leave the “teaching” up to the professional educators. Now, more realize that there is not (nor has there really ever been) a clear distinction between the design of educational products and services and the use of them for teaching. Each influences the other, and understanding of educational research is critical for those who design and develop the products and services that inform what and how educators teach students.

According to this article, the preK-12 testing and assessment market is almost a 2.5 billion dollar market, “making them the single largest category of education sales” in 2012-2013! A good amount of this is the result of efforts to nationalize and standardize curriculum across geographic regions (like with the Common Core), allowing education companies to design a single product that aligns with the needs of a larger client base. However, even apart from such moves for standardization, more people are becoming aware of the possibilities and impact of using feedback loops and rich data to inform educational decisions.

This is just the beginning. If you are in educational publishing or a startup in the education sector, this is not only a trend to watch, but one to embrace. Start thinking about the next version of your products and services and how learning analytics and feedback loops fit with them. If you look at the K-12 Horizon Report’s 5-year predictions, you see learning analytics, the Internet of everything, and wearable technology. What do all three of these have in common? They are an extension of the Internet’s revolution of increased access to information, but this time it is increasing a new type of information and making it possible to analyze and make important decisions based on the data. Now we have a full circle. Data is experienced by learners. The actions and changes of the learner become new data points, which give feedback directly to the learner, to a teacher, or the product that provided the initial data. There is a new action taken by the learner, teacher and/or interactive product and the cycle continues (see the following image for three sample scenarios).

Screen Shot 2015-02-16 at 2.36.14 PM

Some (although an increasingly small number) still think of the Internet and digital revolution in terms of widespread access to rich content. Those are people who think that digitizing content is adequate. Since the 2000s, we’ve experience the social web, one that is read and write. Now we live in a time where those two are merged, and each action individually and collectively becomes a new data point that can be mined and analyzed for important insights.

While there are hundreds of analytics, data warehousing and mining, adaptive learning, and analytic dashboard providers; there is a powerful opportunity for educational content providers who find ways to animate their content with feedback, reporting features, assessment tools, dashboards, early alert features, and adaptive learning pathways. Education’s future is largely one of blended learning, and a growing number of education providers (from K-12 schools to corporate trainers) are learning to design experiences that are constantly adjusting and adapting.

The concept that we are just making products for the true experts, teachers, is noble and respectable, but the 21st century teacher will be looking for new content and learning experiences that interact with them (and their students), tools that give them rich and important data (often real-time or nearly-now) about what is working, what is not, who is learning, who is not, and why. They will be looking for ways to track and monitor learning progress. If a content provider does not do such things, it will be in jeopardy, with the exception of extremely scarce or high-demand content that can’t be easily accessed elsewhere.

As such, content still matters. It always will. However, the thriving educational content providers and publishers of the 21st century understand that the most high-demand features will involve analytics, feedback (to the learner, teacher, or back to the content for real-time or nearly now adjustments), assessment, and tracking.

Tagged : / / / / /

5 Predictions About Educational Credentialing in 2024

I am doing a bit of consulting later in the week, and one of my tasks is to make a few predictions about education in 2024. My part of the day is focused upon alternate and micro-credentialing. With that in mind, here are five predictions. I don’t necessarily like all these outcomes, but based upon the trends, I see many of them as highly likely, especially as they relate to adult and continuing education; and education for trades and regulated professions. What do you think? As you read this short list, you may be surprised about how much does not seem to be directly tied to credentialing. That is because, at least in much of American higher education, credentials and assessments tend to shape and direct much education practice.

I’ve always seen assessment as a bit boring until I started to recognize how it has become the most powerful aspect of many education environments. Change or add a given assessment or evaluation practice and you can quickly see a transformation in an entire system. Look at the conversations about Common Core in K-12 education. It was when the use of assessments started to take root that the debates become most intense.

Do you have any predictions of your own?

1. Unbundled Education – Education will become increasingly unbundled and aggregated across networks and contexts. This will give way to increased grass-roots educational initiatives, the capacity for learners to self-blend learning experiences from multiple sources and organizations, and cross-organizational credentials. Highly regulated sectors and those with strong centralized professional organizations and standards will be most insulated from some of this. It will lead to significant turmoil and disruption in many higher education institutions.

2. Networked Learning will become a fundamental life and work skill. While the most regulated industries will be more insulated, there will be significant conflict between democratizing and authoritarian models of education and training. Regardless, a fundamental aspect of lifelong learning will be the development, maintenance and ongoing expansion of a personal learning network. Related to this, we will see massive formal learning networks within geographic areas, specific fields and professions, and other distinct physical or virtual communities.

3. For many professions and trades, competency-based education and assessment will largely replace assessment of readiness through traditional letter grade systems, GPAs and similar measures. Systems like traditional letter grades will be phased out with the emergence of more accurate and granular measures of learner progress and competence. This will impact both initial training and continuing education.

4. Depending upon the context, alternate and micro-credentialing systems will replace or supplement letter grades, course, credits, and degrees (but the most regulated industries will be more insulated from this disruption). These emerging credentialing systems will have features like expiration dates and detailed information about the criteria met to earn the credential.

5. Educational experiences will provide significant learner control and/or learner-specific adjustments of time, place, pace and learning pathway. As part of this, adaptive learning and robust learning progression designs will replace many industrial or one-size-fits all models of education and training. For better or worse, with the maturity of adaptive learning tools, there will be a renewed and invigorated battle between the  “science of teaching and learning” and the “art of teaching and learning.” Learning analytics and big data will drive the design of high-impact, competency-based individualized learning experiences.