ARCHIVES

Early and reliable identification of pulmonary disease is important because several respiratory conditions exhibit overlapping clinical and radiological characteristics. This research presents a multimodal deep learning framework for classifying Normal, COVID-19, Pneumonia, and Tuberculosis by jointly learning from chest X-ray/CT images and structured clinical information. The proposed architecture uses EfficientNet-B3 to extract a 1536-dimensional visual representation and a three-layer Clinical Multi-Layer Perceptron (MLP) to transform patient metadata into a 64-dimensional representation. The two representations are concatenated and processed by a fully connected fusion network that generates a 128-dimensional multimodal representation for Softmax classification. Grad-CAM is incorporated to visualize image regions contributing to the predicted class, while a Flask-based web interface provides real-time prediction, confidence, probability distribution, and explanation. The reported evaluation achieved 98.49% test accuracy, 98.50% precision, 98.49% recall, and 98.48% F1-score. Class-wise F1-scores were 98.03% for Normal, 98.56% for COVID-19, 97.74% for Pneumonia, and 99.60% for Tuberculosis. The results indicate that multimodal fusion can provide strong multiclass classification performance while explainability improves transparency. However, external validation on independent multi-center clinical datasets is required before clinical use.