360 Secure SkillHub
360 安全技能中心

llama.cpp Benchmark

这是基于llama.cpp的llama-bench打造的GGUF模型基准测试工具，核心测试不同prompt长度的处理速度及不同生成长度的token生成速度，支持CUDA、Vulkan、ROCm、CPU等多后端。适用于LLM模型性能对比、推理速度验证，可通过参数定制测试项，支持批量测试多模型，结果可保存，还附构建脚本便于部署配置。

alexhegit
alexhegit
热度 42.13
最近更新 2026-05-11 17:40:15
分类：开发依赖：代码执行 / 读写文件应用场景：DevOps / 数据处理 / 自动化脚本

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add llamacpp-bench
Metadata
name
llamacpp-bench
description
Run llama.cpp benchmarks on GGUF models to measure prompt processing (pp) and token generation (tg) performance. Use when the user wants to benchmark LLM models, compare model performance, test inference speed, or run llama-bench on GGUF files. Supports Vulkan, CUDA, ROCm, and CPU backends.

llamacpp-bench

Run standardized benchmarks on GGUF models using llama.cpp's llama-bench tool.

Quick Start

# Basic benchmark
llama-bench -m model.gguf -p 512,1024,2048 -n 128,256 -ngl 99

# With specific backend
LLAMA_BACKEND=vulkan llama-bench -m model.gguf -p 512,1024,2048 -n 128,256 -ngl 99

Benchmark Parameters

Parameter Description Default
-m Model path (GGUF file) required
-p Prompt sizes to test 512
-n Generation lengths to test 128
-ngl GPU layers to offload 99
-t CPU threads auto
-dev Device selection auto

Standard Test Suite

For consistent comparisons across models, use:

-p 512,1024,2048 -n 128,256 -ngl 99

This tests:

  • Prompt processing: 512, 1024, 2048 tokens
  • Token generation: 128, 256 tokens

Interpreting Results

Metric Meaning Good Performance
pp512 Prompt processing speed at 512 tokens >1000 t/s
pp1024 Prompt processing speed at 1024 tokens >1000 t/s
pp2048 Prompt processing speed at 2048 tokens >1000 t/s
tg128 Token generation speed (128 tokens) >50 t/s
tg256 Token generation speed (256 tokens) >50 t/s

Backend Selection

llama-bench auto-detects available backends. Priority order:

  1. CUDA (NVIDIA GPUs)
  2. ROCm (AMD GPUs)
  3. Vulkan (cross-platform GPU)
  4. CPU (fallback)

To force a backend, set environment variable or check build:

# Check available backends
llama-bench --help | grep -i "backend\|cuda\|rocm\|vulkan"

Batch Benchmarking

Use the provided script for benchmarking multiple models:

./scripts/benchmark_models.sh /path/to/models/*.gguf

Saving Results

Output can be redirected to a file:

llama-bench -m model.gguf -p 512,1024,2048 -n 128,256 -ngl 99 > results.txt

Or use the benchmark script which auto-saves to timestamped files.

Common Issues

  1. Out of memory: Reduce -ngl (GPU layers) or test smaller prompt sizes
  2. Slow CPU performance: Ensure -t matches CPU core count
  3. Backend not found: Check llama.cpp was built with the desired backend

Building / Updating llama.cpp

Check Current Version

./scripts/build_llamacpp.sh -v

Shows:

  • Current Git commit and branch
  • Build date
  • Whether behind upstream
  • Available backends

Build or Update

# Interactive mode (prompts for backend selection)
./scripts/build_llamacpp.sh -u

# Specify backend directly
./scripts/build_llamacpp.sh -u -b vulkan   # Vulkan (AMD/Intel GPUs)
./scripts/build_llamacpp.sh -u -b cuda     # CUDA (NVIDIA GPUs)
./scripts/build_llamacpp.sh -u -b rocm     # ROCm (AMD GPUs)
./scripts/build_llamacpp.sh -u -b cpu      # CPU only

# Clean rebuild
./scripts/build_llamacpp.sh -c -b vulkan

# Custom build directory
./scripts/build_llamacpp.sh -u -b cuda -d /custom/path

Build Options

Flag Description
-v Show version info and exit
-u Update to latest from GitHub
-c Clean build (remove existing)
-b Backend: vulkan, cuda, rocm, cpu
-d Build directory path
-j Parallel jobs (default: CPU count)

Finding llama-bench

The benchmark script auto-detects llama-bench in these locations:

  • /DATA/Benchmark/llama.cpp/build/bin/llama-bench
  • ~/Repo/llama.cpp/build/bin/llama-bench
  • ~/lab/build/bin/llama-bench

If not found, it will search your home directory or you can build it using the script above.