Real-time YOLO11 object detection in C++ (OpenCV DNN, CUDA/CPU)

Webcam object detection with YOLO11x exported to ONNX, run through OpenCV's DNN module in C++20 with a runtime-switchable CUDA / CPU backend. No Python at inference time: the YOLO output tensor is decoded by hand and the only dependency is OpenCV.

What the code does

Stage	Where	Notes
Load ONNX, pick backend	YoloModel	cv::dnn::readNetFromONNX; DNN_BACKEND_CUDA/DNN_TARGET_CUDA (fusion disabled, see below) or OpenCV CPU, switchable at runtime with SetUseCuda()
Pre-processing	ObjectDetector::Detect	blobFromImage to 640×640, 1/255 scaling, BGR→RGB
Output decoding (by hand)	ObjectDetector::processDetections	Reshape + transpose the [1, 84, 8400] tensor, argmax over the 80 COCO scores, convert normalised cx, cy, w, h to pixel cv::Rect
NMS	cv::dnn::NMSBoxes	score threshold 0.5, IoU threshold 0.5 (OpenCV's implementation, not custom)
Backend check	main.cpp	getAvailableBackends(): warns and runs on CPU if the OpenCV build has no CUDA backend
Capture / display	VideoCaptureManager	camera index or video file, optional headless mode, FPS overlay

Requirements

• OpenCV 4.10 with the DNN module; for the GPU backend it must be built from source with CUDA/cuDNN (recipe below)
• A YOLO11 model in ONNX format (see below)
• Visual Studio 2022+ project in projects/vs-2022/ (C++20 in both Debug and Release; builds with the v145 toolset too)

Exporting YOLO11 to ONNX

from ultralytics import YOLO   # Python 3.9+
YOLO("yolo11x.pt").export(format="onnx")   # -> yolo11x.onnx, copy it to models/

yolo11x is the most accurate and slowest variant; yolo11n/yolo11s run several times faster on CPU.

Re-export with a current ultralytics. A yolo11x ONNX exported in early 2025 loaded without any error in OpenCV 4.10's C++ importer but produced garbage scores — zero detections, silently — while the very same file ran fine in Python's cv2 4.11. A fresh export (opset 12) fixed it. If the demo draws nothing, re-export the model before debugging your own code.

Run

Run from a folder two levels below the repo root (e.g. binaries/<build>/), next to opencv_world4100.dll:

computer-vision.exe                              # camera 1, CUDA backend if the OpenCV build has it
computer-vision.exe --cpu --source 0             # camera 0, CPU
computer-vision.exe --source clip.mp4 --model ./../../models/yolo11n.onnx
computer-vision.exe --source clip.mp4 --frames 60 --headless        # benchmark, no window
computer-vision.exe --source clip.mp4 --frames 90 --headless --dump frames/   # annotated JPEGs

Press any key to quit. The mean detection latency and FPS are printed on exit.

Measured performance

Headless, 60 frames of a 1920×1080 driving clip, 640×640 network input, RTX 4070 Ti SUPER, CUDA 12.8 + cuDNN 9.7. Latency is ObjectDetector::Detect only (pre-processing + forward + decoding + NMS + drawing), so it includes the host↔device copies of every frame. Every raw run is in docs/bench.csv, so these numbers can be checked rather than quoted.

Model	CPU (ms / FPS)	CUDA (ms / FPS)	Speed-up
yolo11n	56.8 / 17.6	16.0 / 62.6	3.6×
yolo11x	499.3 / 2.0	32.2 / 31.1	15.5×

Each cell is the mean of two 60-frame runs on an otherwise idle machine. The CUDA column is stable (repeats within 1 ms); the CPU column is the one that moves with whatever else the machine is doing. An earlier run of the same binary measured yolo11x at 591.9 ms on CPU, which reads as 18.1× instead of 15.5×, and a draft taken while a training job shared the GPU was ~45% slower still. Treat “CUDA vs CPU on yolo11x” as 15–18× on this box rather than a single figure, and re-measure on your own.

The gap between the two speed-ups is the point worth understanding: the nano model is small enough that per-frame fixed costs (blob creation, host↔device copies, decoding) still weigh, so the GPU buys 3.6×. On yolo11x the forward pass is the bottleneck and CUDA wins by 15×, turning a 2 FPS model into a real-time one (31 FPS, on par with a 30 FPS camera) on the same binary.

Getting the CUDA backend to actually run

Two things had to happen before the table above could contain a CUDA column, and both are easy to miss:

1. The prebuilt opencv_world from opencv.org has no CUDA. Requesting DNN_BACKEND_CUDA on it makes OpenCV fall back to CPU silently; --cpu and the "CUDA" run gave identical timings, which is how it was caught. The program now checks cv::dnn::getAvailableBackends() at start-up and prints a warning instead of claiming a GPU it is not using. OpenCV was rebuilt from source (4.10.0 + contrib, Ninja, MSVC, CUDA arch 8.9):

   cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DOPENCV_EXTRA_MODULES_PATH=opencv_contrib/modules ^
     -DBUILD_LIST=core,imgproc,imgcodecs,videoio,highgui,dnn,cudev,cudaarithm -DBUILD_opencv_world=ON ^
     -DWITH_CUDA=ON -DWITH_CUDNN=ON -DOPENCV_DNN_CUDA=ON -DCUDA_ARCH_BIN=8.9 -DCUDA_ARCH_PTX= ^
     -DCUDNN_INCLUDE_DIR=<cudnn>/include/12.8 -DCUDNN_LIBRARY=<cudnn>/lib/12.8/x64/cudnn.lib ^
     -DWITH_FFMPEG=ON -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -B build -S opencv
   

   and the Visual Studio project takes the install location through two MSBuild properties: msbuild ... /p:OPENCV_DIR=<install> /p:OPENCV_LIB=<install>\lib (defaults point at the stock layout). cudnn64_9.dll and the CUDA runtime must be on PATH at run time.

2. OpenCV's CUDA backend asserts on YOLO11 graphs during layer fusion (biasLayerData->outputBlobsWrappers.size() == 1) — verified on 4.10.0 and 4.11.0, both crash identically with fusion on. YoloModel disables fusion on the CUDA target (net->enableFusion(false)); with it off, 4.11 measured on par with 4.10 (29.8 vs 31.3 ms on the same clip), so the workaround costs nothing measurable and makes the model run.

Decoding bug fixed (Aug 2026). The original decoder treated the YOLO11 output boxes as normalised [0, 1] coordinates and multiplied them by the frame size. Ultralytics ONNX exports emit cx, cy, w, h in pixels of the network input (640×640), so every box landed off-screen and the demo drew nothing. Boxes are now rescaled per axis by frame / input; see ObjectDetector::processDetections.

Project layout

code/headers, code/source   YoloModel · ObjectDetector · VideoCaptureManager · LabelLoader · main
models/                     coco.txt (class names) — put your .onnx here
projects/vs-2022/           Visual Studio solution
binaries/                   a Debug build; needs opencv_world4100d.dll next to it

Licenses

• YOLO11 models and export code: Ultralytics, AGPL-3.0.
• OpenCV: Apache 2.0.
• This repository: MIT.