Announcing the First Fully-Adaptive Automated Speaking Assessment

Read the press release here.

Emmersion is thrilled to announce the release of the question-by-question fully-adaptive version of our English speaking assessment! Emmersion’s automated speaking assessment has been providing immense value to organizations all over the world by reducing the time and cost of effective and accurate language screening. Adding question-by-question adaptivity greatly increases its value to organizations in several ways, and even unlocks exciting possibilities that extend beyond assessment.

What Is An Adaptive Assessment?

Adaptive assessment, often referred to as Computerized Adaptive Test or CAT, is a process in which each question in the assessment is determined dynamically based on the test-taker’s performance on the previous question. Instead of the question and sequence of questions being predetermined and fixed, the questions are pulled from a large calibrated question bank. 

The basic goal is to choose an easier question if the test-taker got the previous question wrong, and a more difficult question if the test-taker got the question right. This helps ensure that the questions being asked are the most relevant and informative.  

The Benefits of Adaptive Assessment

First and most important to organizations’ bottom line, adding adaptivity reduces the administration time for the assessment while increasing its quality. You read that right! With adaptivity, a shorter testing experience can coincide with an increase in accuracy.

Each question is scored immediately, and, in the context of the millions of data points we’ve already collected, the system determines which question will be the most appropriate for the test-taker’s level. Each question is designed to provide the most evaluative information possible, making the overall process much more efficient and saving precious time. 

Next, the user experience for test-takers is dramatically improved through adaptivity. In fixed-form assessments, test-takers are confronted with questions of a wide range of difficulty in order to collect enough data to pinpoint test-takers’ ability level. For advanced learners, many of the questions are too simple, making the experience boring and potentially causing over-confidence. For novice learners, many of the questions are too difficult, making the experience unmotivating and causing doubt in their ability.
Adaptivity provides the user experience similar to the Goldilocks fairy tale—not too difficult or too easy, but is just right for their level. This experience leaves them feeling accomplished and confident that they represented their ability well.

Finally, utilizing an adaptive assessment engine significantly reduces the possibility of cheating. Since the question selection is completely dynamic, there is no way for the test-taker to anticipate and prepare for the questions in advance. On a more macro level, adding adaptivity allows for a streamlined automated process of new question calibration.

As more assessments are taken, more new questions are calibrated and added to the question bank and older questions are retired. Keeping the question bank fresh creates another important layer that prevents test-takers from being able to anticipate or prepare for the questions. This also makes retests significantly more effective for organizations that need to test a single candidate multiple times.

The Challenges of Creating an Adaptive Speaking Assessment

The technology for computer adaptive assessments has been around for decades. Emmersion’s WebCAPE adaptive reading assessment was originally created over 20 years ago. Though technology and psychometric practices have improved adaptive assessment significantly over the years, adding adaptivity to an automated speaking assessment until now has been uncharted territory. 

Most adaptive assessments are multiple choice, meaning there is one right answer and one or more wrong answers. The difficulty with automated speaking assessments is that there is no single right or wrong answer. Instead, each answer contains hundreds of data points and the system provides an accuracy score on a spectrum.
An adaptive system does not just decide if an answer is right or wrong, but how right or how wrong in order to determine the next question. This process is immensely more complicated, requiring a lot more calibration and fine tuning than a traditional multiple-choice adaptive assessment. At Emmersion, we like doing hard things!

Adaptivity Unlocks the Future of Personalization

Adding adaptivity as a foundational technology to the English speaking assessment unlocks multiple revolutionary opportunities for personalization both in the assessment and the learning process.

Emmersion AI Adaptive Assessment Engine

Let’s be honest—even though assessments are immensely important, they are a means to an end. No one gets excited about the prospect of taking or reviewing an assessment. What everyone really cares about are the evaluation scores that are produced from the assessment. Organizations use these scores to make vital decisions such as whether to hire a candidate or how to correctly place a candidate into a remediation or learning program.

Test-takers find value in these scores by having a clear picture of their abilities and how they can improve. To that end, we are constantly looking for ways to improve the accuracy, reliability, and speed at which we deliver scoring data to our users.

Instead of building a suite of individual assessments, we are building an adaptive assessment engine to provide our clients with the precise data they need to make critical decisions. Above, we described the benefit of introducing adaptivity into the speaking assessment itself. Having multiple adaptive assessment modules enables the possibility of an adaptive assessment engine.

Clients start by defining the scoring data they need in terms of skills and level of detail. The system then pieces together the most efficient assessment experience based on these needs. The questions are tailored to specifically elicit the data needed to provide the relevant scores, keeping the assessment experience targeted and efficient.

In cases where multiple assessment modules are needed, each module builds off the previous module adaptively to leverage all the information possible to keep the experience short and powerful. Eventually, the system can even make on-the-spot determinations to include or exclude an assessment module based on the performance of previous modules.

The Role of Adaptive Assessment in Personalized Learning

This is especially useful in cases where clients want to assess one skill for all candidates but then assess a secondary skill only for those who meet an ability level cut-point for the first skill. Such an adaptive assessment engine leverages AI and machine learning technology to personalize the assessment experience and deliver accurate and immediate scoring data to decision-makers.

It’s no secret that one-to-one tutoring is far superior to group instruction in producing better learning outcomes. Effective tutors are able to personalize the learning experience by engaging in continuous evaluation, providing an immediate feedback loop and quickly adapting the instruction to meet the learner’s needs.

Online learning providers have been unable to create fully-personalized learning experiences because they lack the necessary automated assessment tools. Combining the Emmersion AI Adaptive Assessment Engine with modular online learning content can result in an online learning system that can start to achieve the benefits of human tutors.

Adaptivity provides room for the transition into diagnostic questions to determine individual learning gaps because it enables the assessment system to determine a more accurate score immediately. As the assessment system produces evaluation data that uncovers specific learning gaps, an adaptive learning system can leverage these data to build modular, personalized learning paths.

Learners will experience improved outcomes because the instruction is tailored to their individual skill level and targeted toward their specific learning gaps. In this way, introducing an adaptive assessment engine to the online language learning market can approximate the advantage of one-to-one tutoring in an affordable and scalable model. 

Closing the Global Communication Gap

I founded Emmersion together with my cofounders with a very specific, lofty mission in mind: to close the global communication gap. I believe that language is the key to connecting the world and breaking down communication barriers. Even before founding Emmersion, I have dedicated the majority of my education, research, and career to accomplishing this mission. I take great pride in guiding Emmersion to tackle the vital obstacles that stand in the way of creating better, more effective language assessment and language learning systems.

Today, we are making huge strides toward realizing our vision of closing the global communication gap by introducing our fully-adaptive speaking assessment. Tomorrow, we will continue innovating and building toward a better, more affordable, and more scalable personalized language assessment and learning engine.

Read the full Technical Report of the TrueNorth English Form D / Adaptive Version here.

The TrueNorth English Speaking Assessment was developed to modernize English language testing with patented artificial intelligence and machine learning. This technology allows for immediate results and scoring as opposed to all other language assessments that take 24 to 48 hours to grade. TrueNorth provides a convenient and immediate English testing solution that has been validated and calibrated to global testing standards and is also available in several other languages.

Some of the largest international companies rely on TrueNorth testing every day. With this technology, BPOs and international companies utilize real-time data and reporting to assist in attracting and hiring the best talent. Additionally, more than 650 universities, colleges, and training institutions around the globe use the TrueNorth platform for course placement and progress tracking. TrueNorth is delivered online with only the need for headphones and microphones.

Leave a Reply

Your email address will not be published. Required fields are marked *