Emotion recognition for mental health prediction using AI techniques: An overview
Main Article Content
Abstract
Abstract
Human facial expressions are a mirror of human thoughts, feelings and human mental states. Facial Emotion Recognition (FER) can provide a social advantage. It's like a form of silent communication. Emotion recognition technology will help to automatically detect the patient's emotions during illness and avoid external acts such as suicide, mental disorders or mental health problems. If we understand all the signs of emotions, we can solve many problems for human beings. Emotion recognition and detection is also useful for healthcare. Through emotional state recognition, we can get information about patients. Recognizing a patient's emotions for a specific disease using artificial intelligence techniques is a challenging task. This article presents recognition, detection and methods for mental health patients. Using artificial intelligence techniques with an emotion detection library and matching emotions to mental health. This article uses an emotional scale to show that there is a link between negative emotions and mental health problems. In this paper, she provided a comprehensive review of AI-based FER methodology, including datasets, feature extraction techniques, algorithms, and recent breakthroughs with their applications in facial expression recognition. In the future, all aspects of FER for different ages would significantly influence the health research community.
Downloads
Article Details
COPYRIGHT
Submission of a manuscript implies: that the work described has not been published before, that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication, the authors agree to automatic transfer of the copyright to the publisher.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
- The journal allows the author(s) to retain publishing rights without restrictions.
- The journal allows the author(s) to hold the copyright without restrictions.
References
Reference
Chirag Dalvi et al., A Survey of AI-based Facial Emotion Recognition: Features, ML & DL Techniques, Age-wise Datasets and Future Directions. DOI 10.1109/ACCESS.2021.3131733, IEEE Access.
A. MEHRABIAN and S. R. FERRIS, “Inference of Attitudes from Nonverbal Communication in Two Channels,†J. Consult. Psychol., vol. 31, no. 3, pp. 248–252, 1967, DOI: 10.1037/h0024648.
M. A. Lumley et al., “Pain and Emotion: A Biopsychosocial Review of Recent Research,†J. Clin. Psychol., vol. 67, no. 9, p. 942, Sep. 2011, DOI: 10.1002/JCLP.20816.
A. Dzedzickis, A. Kaklauskas, and V. Bucinskas, “Human Emotion Recognition: Review of Sensors and Methods,†Sensors 2020, Vol. 20, Page 592, vol. 20, no. 3, p. 592, Jan. 2020, doi: 10.3390/S20030592.
J. A. R. Eliot and A. Hirumi, “Emotion theory in education research practice: an interdisciplinary critical literature review,†Educ. Technol. Res. Dev. 2019 675, vol. 67, no. 5, pp. 1065–1084, Feb. 2019, doi: 10.1007/S11423-018-09642-3.
SeungJun Oh 1,2 and Dong-Keun Kim 3,4., Comparative Analysis of Emotion Classification Based on Facial Expression and Physiological Signals Using Deep Learning. Appl. Sci. 2022, 12, 1286. https://doi.org/10.3390/ app12031286.
Kanagaraj, G.; Ponnambalam, S.G.; Jawahar, N. Supplier Selection: Reliability Based Total Cost of Ownership Approach Using Cuckoo Search. In Trends in Intelligent Robotics, Automation, and Manufacturing, Proceedings of the First International Conference, IRAM 2012, Kuala Lumpur, Malaysia, 28–30 November 2012; Ponnambalam, S.G., Parkkinen, J., Ramanathan, K.C., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; pp. 491–501.
Maaoui, C.; Pruski, A. Emotion Recognition through Physiological Signals for Human-Machine Communication. In Cutting Edge Robotics 2010; Kordic, V., Ed.; IntechOpen: London, UK, 2010; pp. 317–332.
Ali, M.; Mosa, A.H.; Machot, F.A.; Kyamakya, K. Emotion Recognition Involving Physiological and Speech Signals: A Comprehensive Review. In Recent Advances in Nonlinear Dynamics and Synchronization. Studies in Systems, Decision and Control; Kyamakya, K., Mathis, W., Stoop, R., Chedjou, J., Li, Z., Eds.; Springer: Berlin/Heidelberg, Germany, 2018; Volume 109, pp. 287–302.
Nazish Azam et al., Automatic emotion recognition in healthcare data using supervised machine learning. DOI 10.7717/peerj-cs.751.
Saganowski S., Bringing Emotion Recognition Out of the Lab into Real Life: Recent Advances in Sensors and Machine Learning. Electronics 2022, 11, 496. https:// doi.org/10.3390/electronics11030496.
Randolph R. Cornelius, Theoretical Approaches to Emotion, ISCA Archive, 14 August 2015.
Rakesh Panday and Anil K. Choubey, Emotion and Health: An Overview, Psy. & Ment.Health,135-152, 2010.
Jannis T. Kraiss et al., The relationship between emotion regulation and well-being in patients with mental disorders: A meta-analysis, https://doi.org/10.1016/j.comppsych.2020.152189.
Kring AM, Sloan DM. Emotion regulation and psychopathology: A transdiagnostic approach to etiology and treatment: Guilford press; 2009.
Krueger RF, Eaton NR. Transdiagnostic factors of mental disorders. World Psychiatry. 2015; 14:27–9.
Nolen-Hoeksema S, Wisco BE, Lyubomirsky S. Rethinking rumination. Perspectives on psychological science. 2008; 3:400–24.
Rottenberg J, Gross JJ, Gotlib IH. Emotion context insensitivity in major depressive disorder. J Abnorm Psychol. 2005; 114:627.
Johnson SL. Mania and dysregulation in goal pursuit: a review. Clin Psychol Rev. 2005; 25:241–62.
Linehan M. Cognitive-behavioral treatment of borderline personality disorder: Guilford press; 1993.
Lynch TR, Trost WT, Salsman N, Linehan MM. Dialectical behavior therapy for borderline personality disorder. Annu Rev Clin Psychol. 2007; 3:181–205.
McLaughlin KA, Mennin DS, Farach FJ. The contributory role of worry in emotion generation and dysregulation in generalized anxiety disorder. Behav Res Ther. 2007; 45:1735–52.
Clyne C, Blampied NM. Training in emotion regulation as a treatment for binge eating: a preliminary study. Behaviour Change. 2004; 21:269–81.
Bydlowski S, Corcos M, Jeammet P, Paterniti S, Berthoz S, Laurier C, et al. Emotionprocessing deficits in eating disorders. International journal of eating disorders. 2005; 37:321–9.
Bohlmeijer ET, Westerhof GJ. A new model for sustainable mental health, integrating well-being into psychological treatment (in press). In: Kirby J, Gilbert P, editors. Making an impact on mental health and illness. London: Routlege; 2018.
Franken K, Lamers SM, ten Klooster PM, Bohlmeijer ET, Westerhof GJ. Validation of the mental health continuum-short form and the dual continua model of wellbeing and psychopathology in an adult mental health setting. J Clin Psychol. 2018; 74:2187–202.
Aldao A, Nolen-Hoeksema S, Schweizer S. Emotion-regulation strategies across psychopathology: a meta-analytic review. Clin Psychol Rev. 2010; 30:217–37.
Daniel Nixon et al., A novel AI therapy for depression counseling using face emotion techniques, http://doi.org/10.1016/j.gltp.2022.03.008.
Dr. Trayambak Tiwar et al., Mental health in relation to emotional intelligence among university students, 2017 Indian Association of Health, Research and Welfare ISSN-p-2229-5356, e-2321-3698.
Matthias Berkinga and Peggilee Wuppermanb, Emotion regulation and mental health: recent findings, current challenges, and future directions, Curr Opin Psychiatry 2012, 25:128–134 DOI:10.1097/YCO.0b013e3283503669.
Sreeja P.S. and G.S. Mahalakshmi, Emotion Models: A Review, I J C T A, 10(8), 2017, pp. 651-657, https://www.researchgate.net/publication/319173333
Aneja, Deepali, et al. "Learning to generate 3D stylized character expressions from humans." 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018.
Livingstone & Russo (2018). The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. doi:10.1371/journal.pone.0196391
P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar and I. Matthews, "The Extended Cohn-Kanade Dataset (CK+): A complete facial expression dataset for action unit and emotion-specified expression," in 3rd IEEE Workshop on CVPR for Human Communicative Behavior Analysis, 2010
Lyons, Michael; Kamachi, Miyuki; Gyoba, Jiro (1998). The Japanese Female Facial Expression (JAFFE) Database. doi:10.5281/zenodo.3451524.
M. Valstar and M. Pantic, "Induced disgust, happiness and surprise: an addition to the MMI facial expression database," in Proc. Int. Conf. Language Resources and Evaluation, 2010
I. Sneddon, M. McRorie, G. McKeown and J. Hanratty, "The Belfast induced natural emotion database," IEEE Trans. Affective Computing, vol. 3, no. 1, pp. 32-41, 2012
Singh, Shivendra; Benedict, Shajulin (2020). Thampi, Sabu M.; Hegde, Rajesh M.; Krishnan, Sri; Mukhopadhyay, Jayanta; Chaudhary, Vipin; Marques, Oge; Piramuthu, Selwyn; Corchado, Juan M. (eds.). "Indian Semi-Acted Facial Expression (iSAFE) Dataset for Human Emotions Recognition". Advances in Signal Processing and Intelligent Recognition Systems. Communications in Computer and Information Science. Singapore: Springer. 1209: 150–162. doi:10.1007/978-981-15-4828-4_13. ISBN 978-981-15-4828-4.
S. M. Mavadati, M. H. Mahoor, K. Bartlett, P. Trinh and J. Cohn., "DISFA: A Spontaneous Facial Action Intensity Database," IEEE Trans. Affective Computing, vol. 4, no. 2, pp. 151–160, 2013
N. Aifanti, C. Papachristou and A. Delopoulos, The MUG Facial Expression Database, in Proc. 11th Int. Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Desenzano, Italy, April 12–14, 2010.
S L Happy, P. Patnaik, A. Routray, and R. Guha, "The Indian Spontaneous Expression Database for Emotion Recognition," in IEEE Transactions on Affective Computing, 2016, doi:10.1109/TAFFC.2015.2498174.
Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D.H.J., Hawk, S.T., & van Knippenberg, A. (2010). Presentation and validation of the Radboud Faces Database. Cognition & Emotion, 24(8), 1377—1388. doi:10.1080/02699930903485076
"Facial Expression Research Group Database (FERG-DB)". grail.cs.washington.edu. Retrieved 2016-12-06.
Mollahosseini, A.; Hasani, B.; Mahoor, M. H. (2017). "AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild". IEEE Transactions on Affective Computing. PP (99): 18–31. arXiv:1708.03985. doi:10.1109/TAFFC.2017.2740923. ISSN 1949-3045. S2CID 37515850.
"IMPA-FACE3D Technical Reports". visgraf.impa.br. Retrieved 2018-s03-08.
Zafeiriou, S.; Kollias, D.; Nicolaou, M.A.; Papaioannou, A.; Zhao, G.; Kotsia, I. (2017). "Aff-Wild: Valence and Arousal in-the-wild Challenge" (PDF). Computer Vision and Pattern Recognition Workshops (CVPRW), 2017: 1980–1987. doi:10.1109/CVPRW.2017.248. ISBN 978-1-5386-0733-6. S2CID 3107614.
Kollias, D.; Tzirakis, P.; Nicolaou, M.A.; Papaioannou, A.; Zhao, G.; Schuller, B.; Kotsia, I.; Zafeiriou, S. (2019). "Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond". International Journal of Computer Vision (IJCV), 2019. 127 (6–7): 907–929. doi:10.1007/s11263-019-01158-4. S2CID 13679040.
Kollias, D.; Zafeiriou, S. (2019). "Expression, affect, action unit recognition: Aff-wild2, multi-task learning and arcface" (PDF). British Machine Vision Conference (BMVC), 2019. arXiv:1910.04855.
Kollias, D.; Schulc, A.; Hajiyev, E.; Zafeiriou, S. (2020). "Analysing affective behavior in the first abaw 2020 competition". IEEE International Conference on Automatic Face and Gesture Recognition (FG), 2020: 637 643 arXiv:2001.11409. doi:10.1109/FG47880.2020.00126. ISBN 978-1-7281-3079-8. S2CID 210966051.
Li., S. "RAF-DB". Real-world Affective Faces Database.
Li, S.; Deng, W.; Du, J. (2017). "Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild". 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR): 2584–2593. doi:10.1109/CVPR.2017.277. ISBN 978-1-5386-0457-1. S2CID 11413183.
Z. Fei et al., Deep convolution network-based emotion analysis towards mental health care, Neurocomputing, https://doi.org/10.1016/j.neucom.2020.01.034.
Shvartzshnaider et al., Learning Privacy Expectations by Crowdsourcing Contextual Informational Norms. This work was supported in part by NSF awards number CNS1355398, CNS-1409415, CNS-1423139, CNS-1553437, and CNS1617286. Copyright c 2016, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
Quanzeng You and Jiebo Luo, Building a Large-Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark. arXiv:1605.02677v1 [cs.AI] 9 May 2016.
Alessandro Ortis et al., An Overview on Image Sentiment Analysis: Methods, Datasets and Current Challenges. DOI: 10.5220/0007909602900300 In Proceedings of the 16th International Joint Conference on e-Business and Telecommunications (ICETE 2019), pages 290-300 ISBN: 978-989-758-378-0.
Shiliange sun etal, 2019, A survey of Optimization Methods from a Machine Learning Perspective.
Yogish Naik, May 2014, Detailed Survey of Different Face Recognition Approaches, ISSN, Volume 3, Issue 5.
Sushma Jaiswal etal, 2011, Comparison Between Face Recognition Algorithm – Eigenface, Fisherfaces and Elastic Bunch Graph Matching, ISSN, Volume2, No 7.
Priyanka Dharani and Dr. AS Vibhute, May 2017, Face recognition using wavelet Neural Network, ISSN, Volume 7, Issue 5.
Nivedita Chitra and Geeta Nijhawan, Jun 2016, Facial Expression Recognition Using Local Binary Pattern and Support Vector Machine, ISSN, Volume 3, Issue 6.
Ankit Jain etal, May 2019, An Emotion Recognition Framework Through Local Binary Patterns, ISSN, Volume 6, Issue 5.
https://iq.opengenus.org/lbph-algorithm-for-face-recognition/
https://medium.com/analytics-vidhya/optimization-algorithms-for-deep-learning-1f1a2bd4c46b
Prof. PhD. Hilal H. Saleh and Assistant Prof. PhD. Rana F. Ghani, Geometric-based Feature Extraction and Classification for Emotion Expressions of 3D Video Film, https://www.researchgate.net/publication/320569287
H. F. Huang, and S. C. Tai, “Facial Expression Recognition Using New Feature Extraction Algorithmâ€, Electronic Letters on Computer Vision and Image Analysis, Published by Computer Vision Center, vol. 11, no. 1, pp. 41 – 54, 2012.
K. Roy, P. Bhattacharya, and C. Y. Suen, “Iris recognition using shape – guided approach and game theoryâ€, Pattern Analysis and Application, Springer – Verlag, vol. 14, no. 4, pp. 329 – 348, 2011.
Pramoda R et al., Emotion Recognition and Drowsiness Detection using Digital Image Processing and Python, Int J Sci Res Sci & Technol. May-June-2021, 8 (3): 1037-1043, doi: https://doi.org/10.32628/IJSRST2183209.
C. David Mortensen, challenges of emotion recognition in images and video, https://www.apriorit.com/dev-blog/642-ai-emotion-recognition
Circumplex Models of Personality and Emotions Circumplex models of personality and emotions. – APA PsycNet. https://www.apa.org/pubs/books/4317770
Sonali Singh, Comparison Between Facial Expression Recognition Algorithms - For Effective Method, Int J Sci Res CSE & IT, November-December-2020; 6 (6) 146-154. doi: https://doi.org/10.32628/CSEIT206612 146