A high mental workload level could significantly contribute to mental fatigue, decreased performance, or long-term health problems . Recently, deep learning models have been trained on Electroencephalogram (EEG) signals to detect users' mental workload. While such approaches show promising results, they either ignore the noise element inherent in the EEG signals or apply a random set of preprocessing techniques to reduce the noise. Such a lack of uniform preprocessing techniques in cleaning the EEG signals would not allow the comparison of the effectiveness of deep learning models across different studies even when they use the data collected from the same experiment. Therefore, in this study, we aim to investigate the effect of preprocessing techniques defined by neuroscientists in the effectiveness of deep learning models. To do so, we focused on the preprocessing techniques that can be automated and do not need any human intervention, namely a high-pass filter, the ADJUST algorithm, and a re-referencing. Using a publicly available mental workload dataset, STEW, we investigate the effect of these preprocessing techniques in three state-of-the-art deep learning models named Stacked LSTM, BLSTM, and BLSTM-LSTM. Our results show that ADJUST has the most significant effect on the performance of our models compare to other steps. Our findings also show that EEG signals that were prepossessed using the high-pass filter, ADJUST algorithm and re-referencing provided the highest classification performance across the investigated deep learning models. We believe this paper provides an important step towards defining a uniform methodological framework for using deep learning models on EEG signals.