Urban Sound Classification using Deep Learning

---

Overview

This project investigates deep learning architectures for environmental sound classification using the UrbanSound8K dataset.

Several neural network architectures are evaluated, including multilayer perceptrons (MLPs), recurrent neural networks (RNNs), and bidirectional RNNs, together with feature extraction, hyperparameter optimization, and robustness analysis.

Environmental audio signals present unique challenges due to non-stationary noise, overlapping acoustic events, variable durations, and complex temporal dynamics. This repository provides an end-to-end framework ranging from raw waveform pre-processing and psychoacoustic feature extraction to hyperparameter grid search and perturbation stress-testing.

---

Repository Topics

deep-learning • audio-classification • urban-sound • machine-learning • tensorflow • keras • neural-networks • rnn • speech-processing • environmental-sound-classification

---

Project Structure

UrbanSound-DeepLearning/
├── README.md                           # Comprehensive portfolio documentation
├── LICENSE                             # MIT License
├── requirements.txt                    # Python dependencies
├── environment.yml                     # Conda environment definition
│
├── notebooks/
│   ├── 01_data_preprocessing.ipynb     # Audio loading, resampling, 4s chunk standardization
│   ├── 02_feature_extraction.ipynb     # MFCC, Mel-Spectrogram, and ZCR extraction
│   ├── 03_model_architectures.ipynb    # MLP, RNN, and BiRNN topology definitions
│   ├── 04_hyperparameter_search.ipynb  # TensorBoard automated grid search
│   ├── 05_robustness_analysis.ipynb    # Additive noise & perturbation stress testing
│   ├── 06_model_evaluation.ipynb       # 10-fold cross-validation & evaluation metrics
│   └── 07_final_model.ipynb            # Final Bidirectional RNN model & predictions
│
├── datasets/                           # Dataset download script & local cache placeholder
├── models/                             # Saved Keras model checkpoints (.h5)
├── figures/                            # Visual plots, diagrams, and confusion matrices
├── results/                            # Benchmark text logs (mlp, rnn, birnn)
└── docs/                               # Project documentation reports (PDF)

---

Workflow Pipeline

The processing and modeling pipeline is structured sequentially from raw acoustic signals to model robustness evaluation:

+------------------------+
|   UrbanSound8K Data    |
+------------------------+
           |
           v
+------------------------+
|   Audio Preprocessing  |  (Resampling, 4s Chunk Padding / Truncation)
+------------------------+
           |
           v
+------------------------+
|   Feature Extraction   |  (MFCCs 13/25, Mel-Spectrograms, ZCR)
+------------------------+
           |
           v
+------------------------+
|    Neural Networks     |  (MLP Baseline, Unidirectional RNN, BiRNN)
+------------------------+
           |
           v
+------------------------+
| Hyperparameter Search  |  (Learning Rate, Batch Size, Epochs, L2 Lambda)
+------------------------+
           |
           v
+------------------------+
|    Model Evaluation    |  (10-Fold Cross-Validation, Confusion Matrix)
+------------------------+
           |
           v
+------------------------+
|  Robustness Analysis   |  (Gaussian Additive Noise, Perturbation Decay)
+------------------------+

---

Feature Extraction

Audio signals in their raw time-domain representation contain high-dimensional amplitude fluctuations that are difficult for standard neural networks to model directly. We extract key psychoacoustic and spectro-temporal features using librosa:

1. Mel-Frequency Cepstral Coefficients (MFCCs)

MFCCs represent the short-term power spectrum of a sound signal based on a linear cosine transform of a log power spectrum on a nonlinear Mel scale of frequency. The Mel scale models human auditory perception, which is more sensitive to frequency variations at lower pitches than higher ones.

• Config 1 (MFCC 13): 13 coefficients per frame (compact, lower parameter count).
• Config 2 (MFCC 25): 25 coefficients per frame (captures fine-grained spectral envelope detail).

2. Spectrograms (STFT)

Short-Time Fourier Transform (STFT) computes discrete Fourier transforms over overlapping windowed segments of the signal: $$X(m, \omega) = \sum_{n=-\infty}^{\infty} x[n] w[n - m] e^{-j \omega n}$$ Spectrograms display acoustic energy distribution across time ($m$) and frequency ($\omega$).

3. Mel Spectrogram

Mel Spectrograms map the linear Hertz frequency scale onto the perceptual Mel scale: $$m = 2595 \cdot \log_{10}\left(1 + \frac{f}{700}\right)$$ This reduces dimensionality while preserving perceptually relevant harmonic and Formant structures crucial for separating urban classes (e.g., sirens vs. engine idling).

4. Zero Crossing Rate (ZCR)

ZCR measures the rate at which the audio signal changes sign from positive to negative or vice versa: $$\text{ZCR} = \frac{1}{2N} \sum_{n=1}^{N} \left| \text{sgn}(x[n]) - \text{sgn}(x[n-1]) \right|$$ ZCR serves as a key indicator for noisiness and percussive content, distinguishing unvoiced/percussive sounds (e.g., jackhammer, gun shots) from harmonic sounds (e.g., street music, sirens).

---

Model Comparison

We evaluated three main deep learning architectures on the UrbanSound8K 10-fold cross-validation scheme.

Model	Accuracy	F1-Score	Parameters	Architecture Summary
MLP Baseline	68.5%	0.678	~4.8M	3 Dense Layers (256 units), L2 Reg ($10^{-4}$), Flattened MFCC Input
RNN (Unidirectional)	71.8%	0.712	~280k	1x SimpleRNN (256 units) + Dense (128 units) + Dropout (0.3)
BiRNN (Bidirectional LSTM)	76.4%	0.759	~580k	1x Bi-LSTM (256 units) + Recurrent Dropout (0.2) + Dense (128 units)

Detailed benchmark logs are stored in the results/ folder:

• results/mlp_results.txt
• results/rnn_results.txt
• results/birnn_results.txt

---

Visualizations & Evaluation

Training Dynamics & Loss Convergence

10-Class Confusion Matrix

---

Robustness Analysis

A dedicated phase of this investigation (notebooks/05_robustness_analysis.ipynb) evaluates model stability when subjected to real-world acoustic perturbations:

Key findings:

1. Additive Noise Tolerance: The BiRNN exhibits superior resilience to additive zero-mean Gaussian noise compared to standard MLPs. At $10,\text{dB}$ SNR, BiRNN maintains over $60%$ accuracy, whereas MLP performance drops sharply below $46%$.
2. Generalization Across Folds: Bidirectional LSTMs effectively capture forward and backward temporal dependencies, reducing overfitting to fold-specific acoustic backgrounds.
3. Perturbation Resilience: Recurrent dropout and L2 regularization prevent over-reliance on high-frequency noise components in MFCC frames.

---

Future Work

• CNNs for Spectrogram Classification: Implementing 2D Convolutional Networks (e.g., ResNet-18, VGGish) directly on 2D Mel-Spectrogram images.
• Transformers & Conformer: Exploring Audio Spectrogram Transformer (AST) models for global context modeling.
• Self-Supervised Learning: Utilizing pre-trained self-supervised audio representations such as wav2vec 2.0 or HuBERT.
• Data Augmentation: Applying SpecAugment (time and frequency masking), pitch shifting, and time stretching to improve generalization.
• Transfer Learning & Attention: Incorporating channel and spatial attention mechanisms (CBAM) for selective frequency band focusing.

---

Installation & Setup

Environment Setup (Conda)

conda env create -f environment.yml
conda activate urban-sound-dl

Installation via Pip

pip install -r requirements.txt

---

License

This project is licensed under the MIT License - see the LICENSE file for details.