On time series cross-validation for deep learning classification model of mental workload levels based on EEG signals

The determination of a subject's mental workload (MWL) from an electroencephalogram (EEG) is a well-studied area in the brain-computer interface (BCI) field. A high MWL level can significantly contribute to mental fatigue, decreased performance, and long-term health problems. Inspired by the success of machine learning in various areas, researchers have investigated the use of deep learning models to classify subjects' MWL levels. A common approach that is used to evaluate such classification models is the cross-validation (CV) technique. However, the CV technique used for such models does not take into account the time series nature of EEG signals. Therefore, in this paper we propose a modification of CV techniques, i.e. a blocked form of CV with rolling window and expanding window strategies, which are more suitable for EEG signals. Then, we investigate the effectiveness of the two strategies and also explore the effects of different block sizes for each strategy. We then apply these models to several state-of-the-art deep learning models used for MWL classification from EEG signals using a publicly available dataset, STEW. There were two classification tasks: Task 1- resting vs testing state, and Task 2- low vs moderate vs high MWL. Our results show that the model evaluated by the expanding window strategy, when it was trained using the 90% of data, provided a better performance than the rolling window strategy and that the BGRU-GRU model outperformed the other models for both tasks.