Advancing Personality Type Prediction: Utilizing Enhanced Machine and Deep Learning Models with the Myers-Briggs Type Indicator

This study addresses key gaps in personality prediction research by conducting a comprehensive comparison of machine learning and deep learning models on a new, large dataset of MBTI personality types. Previous studies predominantly focused on the Big Five framework and overlooked MBTI due to limited datasets. Moreover, basic hyperparameter tuning techniques, label imbalance, and insufficient text lengths in training samples have constrained the accuracy and generalizability of past models. To address these issues, this research employs a large balanced MBTI dataset with sufficient text lengths and optimizes models using Bayesian optimization. Models compared include Logistic Regression (LR), Random Forest (RF), Support Vector Machine (SVC), Naïve Bayes, LightGBM, XGBoost, Multilayer Perceptron (MLP), and Bidirectional Encoder Representations from Transformers (BERT). Results demonstrate that deep learning models outperform traditional methods, with BERT achieving the highest accuracy (93%), followed by XGBoost (86%) and SVC (85%). The BERT model also significantly outperformed the models implemented in previous works in this field. This work provides actionable insights into model selection and optimization, showcasing the utility of advanced techniques like Bayesian optimization in enhancing predictive performance. By addressing these gaps, the study lays the foundation for robust, scalable personality prediction models applicable in psychology, career counselling, and personalized marketing.