Llmfit

通过命令行扫描本机 GPU、CPU、RAM 与显存，联网查询 HuggingFace 模型元数据，按量化、上下文长度和预估推理速度筛选并排名可本地运行的 LLM 模型；支持服务模式供 CI 集成，显存估算为近似值。

johnsonbuilds

热度 13.82

最近更新 2026-09-11 10:03:04

分类：搜索依赖：代码执行 / 第三方 API应用场景：推荐 / 研究 / DevOps

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add llmfit-2

Metadata

name: llmfit
description: CLI tool to find which LLM models run on your local hardware — checks GPU/CPU/RAM and returns compatible models
version: 1.0.0
author: AlexsJones | Weekly Discovery
license: MIT
platforms: ["linux","macos","windows"]
metadata: {"hermes":{"tags":["llm","hardware-detection","local-inference","model-selection"]}}

llmfit — Local LLM Hardware Fitter

Find which open-weight LLM models can run on your local hardware. Hundreds of models, one command.

Source

GitHub: https://github.com/AlexsJones/llmfit
Stars: ~35,700 (trending)
License: MIT
Language: Rust

What It Does

Scans your system (GPU, CPU, RAM, VRAM) and queries HuggingFace to find models that will fit and run efficiently. Returns a ranked list with estimated inference speed.

Installation

# Homebrew (macOS/Linux)
brew install llmfit

# uv / pip (Python wrapper)
uv pip install llmfit

# Pre-built binary
# Download from releases: https://github.com/AlexsJones/llmfit/releases

Usage

# Basic usage — shows compatible models for your hardware
llmfit

# Filter by context length
llmfit --context 8192

# Filter by quantization
llmfit --quant q4_k_m

# Show top N results
llmfit --limit 10

# Specific model family
llmfit --family llama

# With server mode (for CI/automation)
llmfit --server --port 8765

Output Format

Model                          | Quant  | Context | Est. Speed | VRAM Required
qwen2.5-7b-instruct            | Q4_K_M | 32k     | ~45 tok/s  | 4.8 GB
phi4-mini-instruct             | Q4_K_M | 16k     | ~60 tok/s  | 3.2 GB
mistral-7b-instruct            | Q5_K_M | 8k      | ~38 tok/s  | 5.1 GB

Use Cases for Agents

Model selection — when building a local AI agent, query llmfit to pick the best model for available hardware
CI/CD pipelines — use --server mode to integrate model compatibility checks into deployment workflows
Cost estimation — compare quantization options to balance speed vs quality

Integration with Hermes Agent

# In a skill, use llmfit to determine best model before API calls
import subprocess
result = subprocess.run(["llmfit", "--limit", "5"], capture_output=True, text=True)
# Parse output to select optimal model

Pitfalls

Requires local hardware — must run on the machine you want to test
Internet required — queries HuggingFace API for model metadata
VRAM estimation is approximate — actual usage may vary by implementation (llama.cpp vs transformers)
Some models need authentication — gated models (e.g., Llama 3) require HuggingFace token

Verification

llmfit --limit 3
# Should return 3 models compatible with your hardware
# If empty, your hardware may not meet minimum requirements (typically 4GB+ RAM)

References

GitHub: https://github.com/AlexsJones/llmfit
Documentation: https://github.com/AlexsJones/llmfit#readme