An evaluation of an adaptive learning system based on multimodal affect recognition for learners with intellectual disabilities

Article


Standen, P. J., Brown, D. J., Taheri, M., Galvez Trigo, M. J., Boulton, H., Burton, A., Hallewell, M. J., Lathe, J. G., Shopland, N., Blanco Gonzalez, M. A., Kwiatkowska, G., Milli, E., Cobello, S., Mazzucato, A., Traversi, M. and Hortal, E. 2020. An evaluation of an adaptive learning system based on multimodal affect recognition for learners with intellectual disabilities. British Journal of Educational Technology. 51 (5), pp. 1748-1765. https://doi.org/10.1111/bjet.13010
AuthorsStanden, P. J., Brown, D. J., Taheri, M., Galvez Trigo, M. J., Boulton, H., Burton, A., Hallewell, M. J., Lathe, J. G., Shopland, N., Blanco Gonzalez, M. A., Kwiatkowska, G., Milli, E., Cobello, S., Mazzucato, A., Traversi, M. and Hortal, E.
Abstract

Artificial intelligence tools for education (AIEd) have been used to automate the provision of learning support to mainstream learners. One of the most innovative approaches in this field is the use of data and machine learning for the detection of a student’s affective state, to move them out of negative states that inhibit learning, into positive states such as engagement. In spite of their obvious potential to provide the personalisation that would give extra support for learners with intellectual disabilities, little work on AIEd systems that utilise affect recognition currently addresses this group. Our system used multimodal sensor data and machine learning to first identify three affective states linked to learning (engagement, frustration, boredom) and second determine the presentation of learning content so that the learner is maintained in an optimal affective state and rate of learning is maximised. To evaluate this adaptive learning system, 67 participants aged between 6 and 18 years acting as their own control took part in a series of sessions using the system. Sessions alternated between using the system with both affect detection and learning achievement to drive the selection of learning content (intervention) and using learning achievement alone (control) to drive the selection of learning content. Lack of boredom was the state with the strongest link to achievement, with both frustration and engagement positively related to achievement. There was significantly more engagement and less boredom in intervention than control sessions, but no significant difference in achievement. These results suggest that engagement does increase when activities are tailored to the personal needs and emotional state of the learner and that the system was promoting affective states that in turn promote learning. However, longer exposure is necessary to determine the effect on learning.

JournalBritish Journal of Educational Technology
Journal citation51 (5), pp. 1748-1765
ISSN0007-1013
Year2020
PublisherWiley for British Educational Research Association
Publisher's version
License
File Access Level
Anyone
Digital Object Identifier (DOI)https://doi.org/10.1111/bjet.13010
Publication dates
Online29 Jul 2020
Publication process dates
Accepted30 Jun 2020
Deposited05 Aug 2020
FunderHorizon 2020
Copyright holder© 2020 The Authors
Permalink -

https://repository.uel.ac.uk/item/8845x

Download files


Publisher's version
bjet.13010-1.pdf
License: CC BY-NC 4.0
File access level: Anyone

  • 261
    total views
  • 267
    total downloads
  • 7
    views this month
  • 3
    downloads this month

Export as

Related outputs

Let’s just do it: Accessibility and inclusion, why does it matter? Involving people with learning disabilities in research
Choksi, A., Kaur, B., Weinberg, R., Dhillon, S., Kerai, K. and Kwiatkowska, G. 2023. Let’s just do it: Accessibility and inclusion, why does it matter? Involving people with learning disabilities in research. in: Tilly, L and Walmsley, J. (ed.) Rights in Practice for People with a Learning Disability: Stories of Citizenship Palgrave Macmillan.
In response to ‘Use of technology by older adults with an intellectual disability in Ireland to support health, well-being and social inclusion during the COVID-19 pandemic’, by Darren McCausland, Mary McCarron and Philip McCallion
Kwiatkowska, G., Dhillon, S. K. and Kerai, K. 2023. In response to ‘Use of technology by older adults with an intellectual disability in Ireland to support health, well-being and social inclusion during the COVID-19 pandemic’, by Darren McCausland, Mary McCarron and Philip McCallion. British Journal of Learning Disabilities. 51 (2), pp. 191-194. https://doi.org/10.1111/bld.12529
Digital Financial Inclusion in a Cashless Society
Kwiatkowska, G., Choksi, A., Kaur, B., Weinberg, R., Dhillon, S. and Kerai, K. 2023. Digital Financial Inclusion in a Cashless Society. RIX Centre, University of East London RIX Centre, University of East London.
Using multimedia technology to enhance self-advocacy of people with intellectual disabilities: Introducing a theoretical framework for ‘Multimedia Advocacy’
Watts, P., Kwiatkowska, G. and Minnion, A. 2023. Using multimedia technology to enhance self-advocacy of people with intellectual disabilities: Introducing a theoretical framework for ‘Multimedia Advocacy’. Journal of Applied Research in Intellectual Disabilities. 36 (4), pp. 739-749. https://doi.org/10.1111/jar.13107
Preparing social work students for practice by involving young people with profound and multiple learning disabilities in teaching and learning
Kwiatkowska, G. and Stowell, K. 2022. Preparing social work students for practice by involving young people with profound and multiple learning disabilities in teaching and learning. Research in Teacher Education. 12 (1), pp. 35-40. https://doi.org/10.15123/uel.8xvyz
Research in Teacher Education: Volume 12, No. 1, May 2022
Czerniawski, G., Byrne, F., Doyle, M., Trimmer-Platman, T., Beigi, R., Humphreys, H., Kwiatkowska, G., Mcwhinney-Tripp, J., Boyd, P., Thomas, M. and Stowell, K. 2022. Research in Teacher Education: Volume 12, No. 1, May 2022. The School of Education and Communities, University of East London. https://doi.org/10.15123/uel.8xvy9
Digital Lifeline Fund Report: Easy Read
Mackey, J., Howe, L., Appleby, M., Stone, E., Kwiatkowska, G., Choksi, A. and RIX Social Researchers, University of East London 2022. Digital Lifeline Fund Report: Easy Read. Good Things Foundation.
Digital Lifeline: Infographic
Mackey, J., Howe, L., Appleby, M., Stone, E., Kwiatkowska, G., Choksi, A. and RIX Social Researchers, University of East London 2022. Digital Lifeline: Infographic. Good Things Foundation.
Digital Lifeline: A Qualitative Evaluation
Mackey, J., Howe, L., Appleby, M., Stone, E., Kwiatkowska, G., Choksi, A. and RIX Social Researchers, University of East London 2022. Digital Lifeline: A Qualitative Evaluation. Good Things Foundation.
Digital Lifeline Fund: Evaluation Summary
Mackey, J., Howe, L., Appleby, M., Stone, E., Kwiatkowska, G., Choksi, A. and RIX Social Researchers, University of East London 2022. Digital Lifeline Fund: Evaluation Summary. Good Things Foundation.
The RIX Multi Me Toolkit: Rix Research & Media Feasibility Report
Minnion, A., Kwiatkowska, G., Watts, P., Finch, J. and Wilkie, C. 2020. The RIX Multi Me Toolkit: Rix Research & Media Feasibility Report. RIX Research & Media, University of East London.
RIX Wiki Evaluation Toolkit
Minnion, A., Kwiatkowska, G. and Sharpe, D. 2017. RIX Wiki Evaluation Toolkit. Institute for Health and Human Development, University of East London.
Interactive Sensory Objects for and by People with Learning Disabilities
Hollinworth, N., Allen, K., Hwang, F., Minnion, A. and Kwiatkowska, G. 2016. Interactive Sensory Objects for and by People with Learning Disabilities. The International Journal of the Inclusive Museum. 9 (1), pp. 21-38. https://doi.org/10.18848/1835-2014/CGP/v09i01/21-38
One Page Profiles and EHC plans the Wiki way: Using RIX Wikis to implement EHC plans
Kwiatkowska, G. 2016. One Page Profiles and EHC plans the Wiki way: Using RIX Wikis to implement EHC plans. Special Needs Jungle.
The RIX Wiki Software
Minnion, A. and Kwiatkowska, G. 2014. The RIX Wiki Software. UK and EU RIX Research & Media.
Volunteering and learning in HE: exploring and acknowledging student experience
Weston, C., Guardini, Elena, Minnion, A. and Kwiatkowska, G. 2013. Volunteering and learning in HE: exploring and acknowledging student experience. Research in Teacher Education. 3 (2), pp. 7-14. https://doi.org/10.15123/uel.85w25
Research in Teacher Education: Volume 3, No.2, October 2013
Czerniawski, G., Weston, C., Kidd, W., Atwal, Kulvarn, Bonilla Medina, S X, Etherington, Margaret, Weller, Alan, Fielding, Michael, Paffard, F., Wells, D., Guardini, Elena, Minnion, A., Kwiatkowska, G., Cruz Arcila, Ferney, Gariazzo, Julia and Viswarajan, S. 2013. Research in Teacher Education: Volume 3, No.2, October 2013. The School of Education and Communities, University of East London. https://doi.org/10.15123/uel.89178
W2ID – Web 2.0 for People with Intellectual Disabilities
Saward, C., Minnion, A. and Kwiatkowska, G. 2013. W2ID – Web 2.0 for People with Intellectual Disabilities. Education, Audiovisual & Culture Executive Agency.
Multimedia Advocacy for Person Centred Planning: Small Research Pilot Report
Kwiatkowska, G. and Minnion, A. 2013. Multimedia Advocacy for Person Centred Planning: Small Research Pilot Report. RIX Research & Media, University of East London.
Interactive Sensory Objects Developed for and by People with Learning Disabilities
Hollingworth, N., Hwang, F., Minnion, A., Kwiatkowska, G., Lowe, T., Allan, K. and Weldin, N. 2012. Interactive Sensory Objects Developed for and by People with Learning Disabilities. HCI2012 - People & Computers The 26th BCS Conference on Human Computer Interaction. Birmingham, UK 12 - 14 Sep 2012 BCS Learning and Development. https://doi.org/10.14236/ewic/HCI2012.91