Multi-Modal Diagnosis Model Project

Project Summary

The goal of this project is to create a multi-modal diagnostic system that can detect COVID-19, pneumonia, and other respiratory illnesses by combining medical imaging data (like chest X-rays and CT scans) with clinical records (such as symptoms, lab values, and demographic details).

The first and most crucial step toward building a high-performing AI model is data preprocessing — this project focuses deeply on that phase, ensuring the data is clean, normalized, well-aligned, and suitable for training a robust diagnostic model.

Objective

To preprocess and prepare a high-quality dataset that integrates:

Clinical data: Patient symptoms, vital signs, lab reports
Medical imaging: Chest X-rays and CT scans

To build a machine learning model that predicts whether a person has COVID-19, pneumonia, or another lung-related disease

Types of Input Data

1. Clinical Features

Patient age, gender
Symptoms (fever, cough, breathlessness)
Blood markers (CRP, WBC count, oxygen saturation)
Coexisting conditions (e.g., diabetes, hypertension)

2. Medical Imaging

Chest X-rays (CXR)-2D grayscale images
CT scan slices-2D/3D images of lungs
Each image is linked to the corresponding clinical record

Preprocessing Steps

A. Clinical Data Preprocessing

Problem	Solution
Missing values	Filled using statistical imputation (mean, median) or kNN
Inconsistent entries	Unified using mapping (e.g., "Fever", "fever" → "fever")
Mixed data types	Encoded categorical data (e.g., gender: 0 for Male, 1 for Female)
Different scales	Normalized numerical values (like CRP, oxygen saturation)
Redundant features	Removed irrelevant or highly correlated columns

B. Imaging Data Preprocessing

Task	Technique
Image format	Converted DICOM to PNG or JPG
Resize	Standardized to fixed size (e.g., 224224 or 256256)
Normalize	Pixel intensity scaled to 0-1 or standardized
Data augmentation	Flip, rotate, zoom, shift (to prevent overfitting)
Noise removal	Denoising filters or histogram equalization
Optional step	Lung segmentation using pre-trained U-Net model

C. Multi-Modal Alignment

Matched imaging and clinical data by patient ID and scan date
Ensured consistent labeling across modalities (COVID, Pneumonia, Normal)

Output Dataset Features

Balanced dataset: COVID-19, Pneumonia, Normal
Each entry includes:

Preprocessed image input
Corresponding clinical features
Diagnosis label

Tools & Technologies

Python, Pandas, NumPy-clinical data preprocessing
OpenCV, PIL, Pydicom-medical image handling
Scikit-learn-encoding, normalization, imputation
TensorFlow / PyTorchfor future classification model
Matplotlib, Seaborn-visualization and analysis

Advantages of Multi-Modal Diagnosis

Combines image and clinical data for higher accuracy
Helps distinguish between COVID and pneumonia (similar symptoms)
Works even with weak or incomplete data in one modality

Outcome of Preprocessing

Created a structured, clean dataset for training ML/DL models
Ready for use in models like CNN + clinical MLP
Effective in detecting:

COVID-19
Pneumonia (bacterial/viral)
Normal lung cases

Real World Applications

Hospital screening and triage tool
Useful in rural/low-resource settings
Foundation for mobile/web-based AI healthcare tools

What I Learned

Hands-on experience with healthcare data preprocessing
Worked with real medical imaging formats like DICOM
Addressed issues in data quality, integration, and balancing
Prepared data for a practical, real-world AI diagnostic system

Project Title: Data Preprocessing-Multi-Modal Diagnosis Model for COVID-19 & Other Respiratory Diseases