A Hybrid BERTopic–NAMAA Framework for Arabic News Topic Modeling, Classification, and Evaluation

Abstract

Topic detection is a powerful technique for uncovering the latent semantic structures in large-scale unlabeled text. Yet, the application to Arabic is still an open challenge due to the complex morphology of the language and the absence of comprehensive solution that extends topic discovery by systematic evaluation. In this paper, a novel hybrid architecture based on BERTopic (for unsupervised topic detection) in conjunction with NAMAA (an Arabic classification technique) for topic category assignment and external evaluation is presented. The suggested architecture involves an Arabic-specific preprocessing layer, a contextual document embedding through four Arabic pre-trained transformer models (AraBERTv0.2, AraBERTv2, Asafaya, and QARiB), a topic extraction layer by utilizing the BERTopic model, an automatic topic category assignment layer via NAMAA, and an evaluation step for the entire architecture based on both intrinsic (NPMI, C_v coherence, and Topic Diversity) and extrinsic (Accuracy and Cohen's Kappa) metrics.          The proposed framework was tested on two Arabic news related Economy and Sports domains: an imbalanced dataset (D1) of 7,212 documents and a balanced dataset (D2) of 3,158 documents. BERTopic yielded between 59 and 116 topics on D1 and between 62 and 125 topics on D2. On the balanced dataset Asafaya scored the best C_v coherence (0.579) and Cohen Kappa (0.937), while AraBERTv2 ranked the best NPMI (0.289) and Topic Diversity (0.702). The best classification accuracy with Asafaya on D2 was 96.8%, while 98.2 with AraBERTv2 on D1. To sum up, these results indicate that our framework delivers a robust and efficient solution for Arabic news topic discovery and evaluation, revealing also the impact of embedding model choice and class distribution on topic quality, classification accuracy and agreement with the original dataset labels.