Hand Gesture System Control Application

A real-time gesture-based system control application using MediaPipe for hand tracking and geometric analysis for gesture recognition. Control your computer's volume, media playback, brightness, and scrolling through intuitive hand gestures.

Features

• Real-time hand detection and gesture recognition at 30+ FPS
• Two-handed gesture control system:
  • Volume Control: One finger on mode hand + pinch on control hand
  • Media Control: Three fingers on mode hand + pinch gestures on control hand
  • Brightness Control: Two fingers on mode hand + vertical movement on control hand
• Cross-platform support (Windows, macOS, Linux)
• Visual feedback for active control modes
• Configurable via command-line arguments
• No machine learning training required - uses direct geometric analysis
• Simplified distance-based finger detection (rotation-invariant)

Requirements

• Python 3.8 or higher
• Webcam or compatible video capture device
• Operating System: Windows, macOS, or Linux

Installation

Quick Install

1. Clone the repository:

git clone <repository-url>
cd hand-gesture-recognition

2. Create a virtual environment (recommended):

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

For detailed platform-specific installation instructions, see docs/INSTALLATION.md

Quick Start

Run the application with default settings:

python app.py

The application will:

1. Initialize the camera (device 0 by default)
2. Start real-time hand detection
3. Analyze hand gestures using geometric calculations
4. Enable two-handed gesture-based system controls
5. Display the video feed with visual feedback

Press ESC to exit the application.

Gesture Controls

The system uses two hands for control:

• Mode Selection Hand (left hand by default): Show 1-3 fingers to select control mode
  • 1 finger = Volume control
  • 2 fingers = Brightness control
  • 3 fingers = Media control
• Control Hand (right hand by default): Perform control actions
  • Pinch gesture for volume adjustment or media actions
  • Vertical movement for brightness adjustment

Both hands must be visible and stable for 0.3 seconds before controls activate.

Usage Examples

Basic Usage

Run with default settings:

python app.py

Custom Camera Settings

Use a specific camera device and resolution:

python app.py --device 1 --width 1280 --height 720

Adjust Model Complexity

Choose between lite (faster) and full (more accurate) models:

python app.py --model-complexity 0  # Lite model for better performance
python app.py --model-complexity 1  # Full model for better accuracy

Adjust Hand Dominance

Switch which hand is used for mode selection vs control:

python app.py --hand-dominance left  # Use left hand for control, right for mode selection

For all configuration options, see docs/CONFIGURATION.md.

Keyboard Controls

• ESC - Exit application

Documentation

• Gesture Controls Guide - Complete guide to all gestures and control modes
• Installation Guide - Detailed platform-specific installation instructions
• Configuration Guide - All configuration parameters and tuning tips
• Troubleshooting Guide - Common issues and solutions

Architecture

Camera → Hand Detection → Gesture Analysis → Two-Handed Gesture Manager → System Control → Display

Main components:

• CameraCapture: Video capture with error recovery
• HandDetector: MediaPipe-based hand detection
• GestureAnalyzer: Geometric analysis of hand landmarks using distance-based detection
• GestureManager: Coordinates two-handed gesture control
• SystemController: Cross-platform system control (volume, media, brightness)
• ControlRenderer: Visual feedback for active control modes
• GestureRecognitionApp: Main application controller

Project Structure

├── app.py                          # Main entry point
├── config/                         # Configuration management
├── core/                           # Core application modules
│   ├── app.py                      # Application controller
│   ├── camera.py                   # Camera capture
│   ├── detector.py                 # Hand detection
│   ├── gesture_analyzer.py         # Geometric gesture analysis (distance-based)
│   ├── gesture_manager.py         # Two-handed gesture coordination
│   ├── system_controller.py        # System control interface
│   ├── renderer.py                 # Visualization
│   └── control_renderer.py         # Control UI rendering
├── utils/                          # Utility modules
├── tests/                          # Unit tests
└── requirements.txt

Platform-Specific Dependencies

Windows

pip install pycaw comtypes

macOS

# Optional: Install brightness CLI tool
brew install brightness

Linux

# Ensure system tools are installed
sudo apt-get install pulseaudio-utils xbacklight  # Ubuntu/Debian
# or
sudo dnf install pulseaudio-utils xbacklight      # Fedora

Acknowledgments

• MediaPipe by Google for hand tracking
• OpenCV for computer vision operations
• pycaw for Windows audio control
• screen-brightness-control for cross-platform brightness control
• pyautogui for cross-platform input simulation