Target: Ubuntu 24.04 · NVIDIA · see Hardware Requirements. This is a host benchmark, not a laptop pip install project.
git clone https://github.com/garys-gpu-benchmarks/214-gpu-bench-nvidia-memcpy-transfer-bandwidth-ubu2404.git
cd 214-gpu-bench-nvidia-memcpy-transfer-bandwidth-ubu2404
sudo bash setup.sh --assume-yes
bash run_benchmark.sh --profile smoke --validateResults are written to results/benchmark.db and results/summary.json.
This workload is executed on the validation host after the repository is copied there. setup.sh and run_benchmark.sh do not open an outbound SSH session.
Prerequisites: Ubuntu 24.04; NVIDIA; Python 3.12.3; root or sudo for setup.sh. Framework: Bash, SQLite, Python, PyYAML, CUDA Runtime, NVCC. This is a host benchmark, not a laptop pip install project.
flowchart LR
setup.sh --> run_benchmark.sh --> parse_results.py --> results/benchmark.db
This repository is one of the 32 workloads in the NVIDIA Ubuntu 24.04 bundle, bundle-nvidia-ubuntu-2404, which holds them as git submodules pinned to a release. To put every workload for this platform on a GPU host in one step, run this as root (or put sudo in front of the apt-get and git clone commands):
apt-get update && apt-get install -y git
git clone --recurse-submodules https://github.com/garys-gpu-benchmarks/bundle-nvidia-ubuntu-2404 /opt/benchmarks \
&& cd /opt/benchmarks \
&& ./run_benchmark_suite.sh --list \
&& cd /opt/benchmarks/214-gpu-bench-nvidia-memcpy-transfer-bandwidth-ubu2404 \
&& bash run_benchmark.sh --profile smoke --validate \
&& cd /opt/benchmarks \
&& echo "Done: 32 workloads in /opt/benchmarks; 214-gpu-bench-nvidia-memcpy-transfer-bandwidth-ubu2404 smoke run passed. Next: ./run_benchmark_suite.sh --help"Every line in ./run_benchmark_suite.sh --list should show ready; it exits with an error if any workload is not fetched. The commands are joined with &&, so if the clone, the list or the smoke run fails, the chain stops there, and the closing message appears only when everything succeeded. The chain ends in /opt/benchmarks, where the suite runner is. For more extensive testing:
./run_benchmark_suite.sh --smoke # smoke test of all 32 workloads
./run_benchmark_suite.sh -w 207 --baseline # workload 207, baseline profile
./run_benchmark_suite.sh --help # all optionsA workload that is not set up yet sets itself up on its first run (its run_benchmark.sh calls scripts/ensure_setup.sh). See Suite Tools for more.
To move to the newest published release later: cd /opt/benchmarks && git pull && git submodule update --init --recursive.
| Bundle repository | Platform | Workloads |
|---|---|---|
bundle-amd-ubuntu-2404 |
AMD · Ubuntu 24.04 | 101–132 |
bundle-nvidia-ubuntu-2404 |
NVIDIA · Ubuntu 24.04 | 201–232 |
bundle-amd-ubuntu-2604 |
AMD · Ubuntu 26.04 | 301–332 |
bundle-nvidia-ubuntu-2604 |
NVIDIA · Ubuntu 26.04 | 401–432 |
Three operator scripts from garys-gpu-benchmarks/gpu-bench-suite work across the whole bundle. run_benchmark_suite.sh and show_metrics.sh are also at the top of every bundle (and the generator's remote copy installs show_metrics.sh at /opt/benchmarks); each script prints full usage with --help.
run_benchmark_suite.sh(on the GPU host) — runs the installed workloads one after another for each profile, writes one log to/opt/benchmarks/benchmark_suite_log/, and ends with a pass/fail summary.cd /opt/benchmarks ./run_benchmark_suite.sh --dry-run # list what would run ./run_benchmark_suite.sh -w 214 --smoke --baseline # just this workload ./run_benchmark_suite.sh --smoke # every workload, smoke profile ./run_benchmark_suite.sh -r 20 # soak: everything, 20 times
get_remote_info.sh(on your laptop; Git Bash on Windows) — over one SSH connection, copies the runtime ledger and every workload'sresults/raw,results/parsedand suite logs intoRUNTIME_LEDGERS/andDIRECTORIES/.git clone https://github.com/garys-gpu-benchmarks/gpu-bench-suite && cd gpu-bench-suite ./scripts/get_remote_info.sh --host 203.0.113.10 # root@host, port 22 ./scripts/get_remote_info.sh --host ubuntu@203.0.113.10 --port 2222 -i ~/.ssh/id_ed25519 -o ~/results
show_metrics.sh(laptop or GPU host) — prints one workload's runs from the runtime ledger in fixed-width metric columns with per-profile means. On the GPU host it is/opt/benchmarks/show_metrics.sh.cd /opt/benchmarks && ./show_metrics.sh 214 # on the GPU host ./scripts/show_metrics.sh 214 --host 203.0.113.10 # over SSH, all profiles ./scripts/show_metrics.sh 214 --baseline --file RUNTIME_LEDGERS/<ledger>.csv
Builds src/cuda_memcpy.cu and sweeps transfer sizes from min_bytes to max_bytes by step_factor. Each size always runs H2D, D2H, D2D, and H2D_pageable. device_id: GPU index. warmup_iters and num_iterations time cudaMemcpy with CUDA events. Sweep dimensions: device_id, direction, pinned_memory, min_bytes, max_bytes, step_factor, warmup_iters, num_iterations.
- Validates positive bandwidth for every swept size. Metric #5 is a pinned/pageable ratio
- #1: H2D bandwidth at max transfer size, GB/s (bandwidth_h2d_gbps); is present and physically sensible.
- #2: D2H bandwidth at max transfer size, GB/s (bandwidth_d2h_gbps); is present and physically sensible.
- #3: D2D bandwidth at max transfer size, GB/s (bandwidth_d2d_gbps); is present and physically sensible.
- #4: H2D latency at min transfer size, us (latency_h2d_us); is present and physically sensible.
- #5: D2H latency at min transfer size, us (latency_d2h_us); is present and physically sensible.
- #6: D2D latency at min transfer size, us (latency_d2d_us); is present and physically sensible.
- #7: Pinned-over-pageable H2D bandwidth ratio (pinned_vs_pageable_bandwidth_ratio) is present and physically sensible.
- #1: H2D bandwidth at max transfer size, GB/s — stored as
bandwidth_h2d_gbps. - #2: D2H bandwidth at max transfer size, GB/s — stored as
bandwidth_d2h_gbps. - #3: D2D bandwidth at max transfer size, GB/s — stored as
bandwidth_d2d_gbps. - #4: H2D latency at min transfer size, us — stored as
latency_h2d_us. - #5: D2H latency at min transfer size, us — stored as
latency_d2h_us. - #6: D2D latency at min transfer size, us — stored as
latency_d2d_us. - #7: Pinned-over-pageable H2D bandwidth ratio — stored as
pinned_vs_pageable_bandwidth_ratio.
- OS: Ubuntu 24.04
- GPU vendor: NVIDIA
- Framework family: Bash, SQLite, Python, PyYAML, CUDA Runtime, NVCC
- Python: Python 3.12.3
The tables below describe the machine used to generate the reference results. They are not a requirement that every user buy that exact cloud instance.
Builds src/cuda_memcpy.cu and sweeps transfer sizes from min_bytes to max_bytes by step_factor. Each size always runs H2D, D2H, D2D, and H2D_pageable. device_id: GPU index.
Ubuntu 24.04 / NVIDIA / Bash, SQLite, Python, PyYAML, CUDA Runtime, NVCC
| Component | Version |
|---|---|
| OS | Ubuntu 24.04 |
| Kernel | kernel 6.8.0 |
| Python | Python 3.12.3 |
| ROCm | CUDA 12.6 (nvcc 12.6.85 preinstalled in the container; setup installs 12.8 only when nvcc is missing). Driver 580.159.04 reports CUDA 13.0 |
| rocBLAS | N/A - cuBLAS not used |
Builds src/cuda_memcpy.cu and sweeps transfer sizes from min_bytes to max_bytes by step_factor. Each size always runs H2D, D2H, D2D, and H2D_pageable. device_id: GPU index.
Compile cuda_memcpy.cu with nvcc; run bin/cuda_memcpy for H2D, D2H, D2D, and H2D_pageableCompile cuda_memcpy.cu with nvcc; run bin/cuda_memcpy for H2D, D2H, D2D, and H2D_pageableValidating results separately:
export BENCHMARK_PYTHON=/usr/bin/python3.13 # optional
python3 -m venv .venv
source ".venv/bin/activate"
".venv/bin/python" scripts/validate_results.pyCSV with H2D, D2H, and D2D bandwidth and latency on each transfer-size row
sample_index,status,transfer_size_bytes,bandwidth_h2d_gbps,bandwidth_d2h_gbps,bandwidth_d2d_gbps,latency_h2d_us,latency_d2h_us,latency_d2d_us,pinned_vs_pageable_host_memory_bandwidth_gb_s,pinned_vs_pageable_bandwidth_ratio,error_message 0,ok,268435456,25,24,700,8,9,2,20,1.4,
Compile cuda_memcpy.cu with nvcc; run bin/cuda_memcpy for H2D, D2H, D2D, and H2D_pageableConsolidated metrics from the most recent run — suitable for CI artifact upload or dashboard ingestion.
CSV with H2D, D2H, and D2D bandwidth and latency on each transfer-size row
sample_index,status,transfer_size_bytes,bandwidth_h2d_gbps,bandwidth_d2h_gbps,bandwidth_d2d_gbps,latency_h2d_us,latency_d2h_us,latency_d2d_us,pinned_vs_pageable_host_memory_bandwidth_gb_s,pinned_vs_pageable_bandwidth_ratio,error_message 0,ok,268435456,25,24,700,8,9,2,20,1.4,
Expected ranges and gates live in config/benchmark_config.yaml under baselines: or thresholds:. To update them, edit that file — never edit validation code directly.
setup.sh missing collector
Create cannot finish without scripts/collect_workload.py.
self_check overlay rewritten
Locked overlay rewritten means the sibling file and the sha256 in results/overlay_lock.json disagree. The repair is to make implementation_components/<component-id>/files/<path>, the sibling path, and that digest the same bytes, then recheck with create_one_workload.py --validate-only. Do not replace the overlay with a new collector, and do not regenerate a non-empty sibling.
Remote SSH drop during setup
Reconnect and resume bash setup.sh --assume-yes. Do not wipe .venv or .cache.
Native NVIDIA CUDA workload. Execute on the stated Ubuntu release with the host NVIDIA driver and CUDA userspace. ROCm porting notes do not apply.
.
├── setup.sh
├── run_benchmark.sh
├── benchmark_specification.json
├── .github/workflows/ # thin CI callers (see Continuous Integration)
├── config/
├── scripts/
├── src/
├── tests/
├── docs/
├── results/
└── LICENSE
| Workflow | Runs on | When | What it does |
|---|---|---|---|
| CI | GitHub-hosted runner | every pull request, and every push to main |
shellcheck, ruff, bash -n, compileall, run_benchmark.sh --help, specification schema, the results validator on a seeded fixture, required files, and actionlint. No GPU and no benchmark run. |
| GPU Smoke Benchmark | self-hosted runner labeled gpu, nvidia, ubu2404 |
only when started by hand: Actions → GPU Smoke Benchmark → Run workflow (choose smoke, baseline or extended) |
Verifies the pre-provisioned GPU stack, records results/environment.json (driver, runtime, kernel, GPU), runs the profile with --validate, shows headline metrics on the run page, and uploads the results. |
Both files are short callers. The steps themselves live once, for every workload in the suite, in garys-gpu-benchmarks/shared-workflows, pinned at @v1. The GPU workflow is never triggered by pull requests, so code from a fork cannot run on the GPU host.
The GPU Smoke Benchmark workflow runs one workload. To install and run the whole NVIDIA Ubuntu 24.04 bundle by hand on the same machines, see Download as a Bundle and Suite Tools.