There are several hypotheses on how stress affects learning results, exam outcomes and learner performance, with studies indicating that stress greatly hinders knowledge acquisition and memory recall (e.g. Theobald Et al., Vogel Et al.).

While such studies help generate hypotheses on improvements for e-learning and EdTech apps, it can be challenging to effectively test those without unbiased and real-time insights into user experience.

Kouo, using its emotion analytics platform, helped Memrise understand the effects of stress on learners and provided them with new insights into what emotions drive motivation to build effective language practice habits.

The Memrise challenge

Memrise wanted to provide learners with the practice they needed without putting them under too much stress. To this end, they designed MemBot.

Built using GPT-3 to create human-like conversations on any subject, MemBot aims to make language practice low-stress, low-cost and accessible to anyone, anywhere.

But to understand if MemBot can deliver on that promise, they needed to know:

1. If it was lower stress than a tutor.
2. If low stress made for better practice.

The Kouo x Memrise study on learner stress.

To help Memrise understand how their product was performing with real users, Kouo ran an independent study on 60 language learners, comparing emotional responses to two different practice experiences.

Experience 1: A practice session with a Tutor.
The most effective practice method apart from immersion.

Experience 2: A practice session with MemBot.
An adaptive practice module developed using GPT-3.

All learners were Intermediate level in Spanish (Esp), and they all participated in both Tutor and MemBot sessions.

During the sessions, real-time user emotions were recorded using data from wearables (like the Apple Watch) and analysed with Kouo’s AI-based models. This method allowed the assessment of emotional reactions to MemBot sessions in the participants’ natural environments for study.

In addition, Kouo also measured emotions uninterruptedly throughout the whole experience, mapping each participant’s emotional reactions to their user journeys.

Following practice sessions, subjects got assessed on perceived stress, relaxation, self-judgement, focus, skill acquisition and the likelihood of repeat practice using survey questionnaires. The answers then were matched to the learners’ emotional models and reactions and used to cohort user emotions by outcomes, actions and behaviour.

What were the results?

The study was an enormous success, not just for MemBot but in the field of language practice.

Learners experienced an impressive 45% lower stress when practising using MemBot than when speaking with a tutor. Importantly, Kouo also found that the learners’ perception of overall stress was highly correlated with the exit stress level experienced, with a correlation coefficient of over 0.8 and a 99% statistical confidence as a predictor.

This meant that even when an experience is objectively high stress if towards the end stress is decreased, learners will more likely consider the whole experience to be low stress helping them retain a habit of practice with MemBot.

To test and understand the implications of this insight, a new set of sessions have been run, adjusted to be less stressful at the end.

When compared, the session adjusted to be less stressful at the end saw learner stress reduced by 10% and retention increased by 9%.

Kouo identified where stress mattered the most.

The additional finding on where stress had the biggest impact was crucial for Memrise to deliver the better learning experiences they wanted with MemBot.

Even with overall stress levels 45% lower in MemBot sessions, addressing habit formation and learner retention was only possible with Kouo’s insight into where stress mattered the most.

Using Kouo’s emotion analytics, Memrise were able to test its product, improve its base hypotheses and find solutions for learner retention leading to improved user experiences.

Conclusion.

MemBot proved to be a superior language practice companion. It is 45% less stressful than a tutor, and because it is a digital companion, far lower in cost and more accessible than any other practice method.

But importantly, the insights gained using Kouo’s emotion analytics platform into stress have many far-reaching implications for Edtech and education.

Understanding how to encourage positive habit formation is vitally important across a variety of other applications and products, from Femtech to Mental Health, Wellness and Sport etc.

Kouo’s analytics platform provides a viable solution for product managers and companies to understand how their products affect their user base and how to fine-tune them to support user needs.

Working with emotion analytics has been proven to help reduce development costs by as much as 80%. It allows product teams to eliminate hypotheses and reduce the number of experiments needed to find the right solutions or the root cause of product issues.

Get in touch with us to learn more about the study or get a demo of Kouo’s emotion analytics platform.

‍