IMF-Tokenized Transformers for Joint EEG-Based Emotion and Motion Sickness Encoding
DOI:
https://doi.org/10.68337/cpsm.v1.i2.2026-028Keywords:
EEG, CEEMDAN, intrinsic mode functions, Transformer, emotion recognition, motion sickness, multitask learning, brain-computer interfaceAbstract
Emotion recognition and motion sickness detection engage the same frontal-parietal alpha-theta circuitry, yet they are almost always treated as separate electroencephalography (EEG) problems. This paper presents CEEMDAN-TransNet, which addresses both tasks in one model. Instead of feeding raw time windows to a Transformer, each EEG channel is decomposed by complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) into intrinsic mode functions (IMFs), one per canonical frequency band, and each IMF is presented as a token to a shared encoder. Three task heads, for valence, arousal, and motion sickness severity, share the encoder and are trained together under an uncertainty-weighted multi-task loss with a supervised contrastive objective. The framework was tested on two public datasets under leave-one-subject-out (LOSO) cross-validation. Using IMFs instead of raw windows produced a clear accuracy gain. Joint training narrowed cross-subject variability on the sickness task far more than it shifted mean accuracy, which is the more useful outcome for a system that has to work for any user. The ablations showed that none of the design choices is redundant, and the attention maps concentrated on frontal and parietal electrodes in the alpha and theta bands, where the neurophysiology predicts them.
References
[1] S. Koelstra, C. Muhl, M. Soleymani, J.-S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras, "DEAP: A database for emotion analysis using physiological signals," IEEE Trans. Affect. Comput., vol. 3, no. 1, pp. 18-31, 2012, doi: 10.1109/T-AFFC.2011.15.
[2] Vernon J. Lawhern, Amelia J. Solon, Nicholas R. Waytowich, Stephen M. Gordon, Chou P. Hung, and Brent J. Lance, "EEGNet: A compact convolutional neural network for EEG-based brain-computer interfaces," J. Neural Eng., vol. 15, no. 5, Art. no. 056013, 2018, doi: 10.1088/1741-2552/aace8c.
[3] Panagiotis Matsangas and Michael E. McCauley, "Yawning as a behavioral marker of mild motion sickness and sopite syndrome," Aviat. Space Environ. Med., vol. 85, no. 6, pp. 658-661, 2014, doi: 10.3357/ASEM.3897.2014.
[4] Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu, "Large brain model for learning generic representations with EEG data," in Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
[5] Chaoqi Yang, M. Westover, and Jimeng Sun, "BIOT: Biosignal transformer for cross-data learning in the wild," in Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023, pp. 78240-78260, doi: 10.52202/075280-3420.
[6] Y. Xu, K. Yang, and W. Zhang, "EEGPT: Pretrained transformer for universal EEG representation," in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2024.
[7] Maria E. Torres, Marcelo A. Colominas, Gaston Schlotthauer, and Patrick Flandrin, "A complete ensemble empirical mode decomposition with adaptive noise," in Proc. 2011 IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2011, pp. 4144-4147, doi: 10.1109/ICASSP.2011.5947265.
[8] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin, "Attention is all you need," in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017.
[9] Dan Hendrycks and Kevin Gimpel, "Gaussian error linear units (GELUs)," arXiv:1606.08415, 2016.
[10] Alex Kendall, Yarin Gal, and Roberto Cipolla, "Multi-task learning using uncertainty to weigh losses for scene geometry and semantics," in Proc. 2018 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 7482-7491, doi: 10.1109/CVPR.2018.00781.
[11] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan, "Supervised contrastive learning," in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 33, 2020.
[12] Ilya Loshchilov and Frank Hutter, "Decoupled weight decay regularization," in Proc. Int. Conf. Learn. Represent. (ICLR), 2019.
[13] Berken Utku Demirel, Adnan Harun Dogan, Juliete Rossie, Max Möbus, and Christian Holz, "Beyond subjectivity: Continuous cybersickness detection using EEG-based multitaper spectrum estimation," IEEE Trans. Vis. Comput. Graph., vol. 31, no. 5, pp. 2525-2534, 2025, doi: 10.1109/TVCG.2025.3549132.
[14] Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal, "Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness," Int. J. Aviat. Psychol., vol. 3, no. 3, pp. 203-220, 1993, doi: 10.1207/s15327108ijap0303_3.
[15] Federico Del Pup, Andrea Zanola, Louis Fabrice Tshimanga, Alessandra Bertoldo, Livio Finos, and Manfredo Atzori, "The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: A preliminary study," Comput. Biol. Med., vol. 196, Art. no. 110608, 2025, doi: 10.1016/j.compbiomed.2025.110608.
[16] Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao, "EEG conformer: Convolutional transformer for EEG decoding and visualization," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 31, pp. 710-719, 2023, doi: 10.1109/TNSRE.2022.3230250.
[17] Nalini Pusarla, Anurag Singh, and Shrivishal Tripathi, "Decoding emotional patterns using NIG modeling of EEG signals in the CEEMDAN domain," Int. J. Inf. Technol., vol. 17, no. 9, pp. 5625-5636, 2024, doi: 10.1007/s41870-024-02001-x.
[18] Geoffrey Brookshire, Jake Kasper, Nicholas M. Blauch, Yunan Charles Wu, Ryan Glatt, David A. Merrill, Spencer Gerrol, Keith J. Yoder, Colin Quirk, and Ché Lucero, "Data leakage in deep learning studies of translational EEG," Front. Neurosci., vol. 18, Art. no. 1373515, 2024, doi: 10.3389/fnins.2024.1373515.
Downloads
Published
Data Availability Statement
The DEAP dataset is available at http://eecs.qmul.ac.uk/mmv/datasets/deap/. The ETH Cybersickness dataset is available at https://github.com/eth-siplab/EEG_Cybersickness_Estimation.Conference Proceedings Volume
Section
Categories
License
Copyright (c) 2026 The Authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles published in Conference Proceedings in Science and Management are made available under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Authors retain copyright in their work and grant the journal first publication rights. Readers are free to share and adapt the published material for any purpose, including commercial use, provided appropriate credit is given to the author, a link to the license is provided, and any changes are indicated.
