GPU Manager MCP

Read-only GPU monitoring and AI-workload planning over the Model Context Protocol. One stdio server, any MCP client, no LLM required.

Discover NVIDIA, AMD and Intel GPUs, watch utilization / VRAM / temperature / power / clocks, keep bounded local history in SQLite, and estimate whether your LLM actually fits before you load it.

• Cross-platform backends — NVML (via pynvml) and nvidia-smi for NVIDIA; amdgpu sysfs (no ROCm needed) and rocm-smi fallback for AMD; i915/xe sysfs for Intel.
• Honest telemetry — every reading carries a timestamp, unit and source. Unsupported sensors are reported as available: false with a reason, never silently zeroed.
• Read-only by design — no shell tool, no overclocking, no voltage, no fan control, no process termination. External helpers run from a fixed allowlist with timeouts and no shell interpolation.
• AI-aware — estimate_model_fit() computes weights + runtime overhead + KV-cache needs for a given quantization, context length and parallel request count, then checks it against live free VRAM.

Installation

pip install gpu-manager-mcp            # (once published; see below)
# or from source
git clone https://github.com/quxlzx/gpu-manager-mcp
cd gpu-manager-mcp
python -m venv .venv && .venv/bin/pip install -e .

Optional NVIDIA NVML bindings (recommended on NVIDIA hosts):

pip install "gpu-manager-mcp[nvidia]"   # or: pip install nvidia-ml-py

Requirements: Python 3.11+. Linux is fully supported; on Windows the AMD backend is not yet implemented (see Known limitations).

Supported GPUs

✅* = depends on card/driver; unavailable sensors are reported with the driver's own error, not guessed.

Connecting an MCP client

Generic stdio configuration (any MCP client):

{
  "mcpServers": {
    "gpu-manager": {
      "command": "/path/to/.venv/bin/gpu-manager-mcp",
      "args": ["stdio"]
    }
  }
}

Hermes Agent (~/.hermes/config.yaml):

mcp_servers:
  gpu-manager:
    command: /path/to/.venv/bin/gpu-manager-mcp
    args: ["stdio"]
    timeout: 60

Claude Desktop / Cursor / Cline: same command + args shape as above.

Tools

Discovery: list_gpus, get_gpu_info, get_system_gpu_summary Monitoring: get_gpu_utilization, get_gpu_memory, get_gpu_temperature, get_gpu_power, get_gpu_clocks, get_gpu_processes, get_all_gpu_metrics History: get_gpu_history, get_gpu_peak_usage, get_gpu_health AI planning: estimate_model_fit, recommend_gpu_for_model, find_available_gpu, compare_gpus, detect_gpu_bottleneck

Always call list_gpus() first — every other tool takes a gpu_id from it.

Example

estimate_model_fit(gpu_id="amd-0000:03:00.0", model_size_b=7, quantization="q4", context_length=4096, parallel_requests=1, architecture="qwen2-7b") — real output from an 8 GiB RX 6600 XT:

{
  "gpu_id": "amd-0000:03:00.0",
  "available": true,
  "weights_gib": 3.26,
  "runtime_overhead_gib": 0.5,
  "kv_cache_gib_per_request": 0.22,
  "total_gib": 3.98,
  "vram_total_gib": 7.98,
  "vram_free_gib": 7.97,
  "fits": true,
  "headroom_gib": 3.99,
  "assumptions": ["runtime overhead = max(0.5 GiB, 5% of weights)",
                   "KV cache from architecture 'qwen2-7b'"],
  "warnings": []
}

A metric that the hardware cannot provide looks like:

{
  "available": false,
  "reason": "sensor busy — the GPU power management interface refused the read (common when the card is in a low-power state)",
  "source": "amd-sysfs"
}

Configuration

Environment variables (all optional):

Security guidance

• The server exposes monitoring only. There is no tool that executes arbitrary commands, changes clocks/fans/voltages, or kills processes.
• Helper CLIs (nvidia-smi, rocm-smi, intel_gpu_top) are invoked from a fixed allowlist with argument arrays and timeouts — never via a shell and never with user-supplied strings.
• gpu_id values are validated against discovered devices; unknown or crafted ids are rejected with a clean error.
• The optional HTTP transport binds to loopback by default and refuses non-loopback hosts unless you explicitly change it — and you should not, without adding authentication in front of it. Do not expose this server to the public internet.
• History is local SQLite with a bounded row cap; nothing leaves the machine.

Known limitations

• Windows: AMD telemetry via sysfs is Linux-only; a Windows backend is planned (AMD-ADL / PDH or amdsmi bindings). The server runs on Windows but will report no AMD devices.
• AMD hwmon reads can return EBUSY when the card is in a low-power state; the reading is reported as unavailable with that explanation.
• Intel VRAM is not exposed by i915/xe sysfs; memory metrics are reported unavailable rather than estimated.
• KV-cache estimates are architecture-dependent. Without an architecture hint the server uses documented default assumptions and flags them. Treat all fit numbers as engineering estimates.
• Per-process VRAM requires NVML or rocm-smi; plain sysfs cannot provide it.

Troubleshooting

• No GPUs found on Linux — check /sys/class/drm/card*/device/vendor exists and the driver is amdgpu/i915/xe; NVIDIA needs the proprietary driver + nvidia-ml-py or nvidia-smi on PATH.
• sensor busy on AMD — normal at idle on some cards; retry under load or check power_state in sysfs.
• Permission denied — NVML on some setups needs root or CAP_SYS_ADMIN; sysfs sensors are world-readable on standard distros.
• History empty — the sampler runs inside the server process; give it a few sample intervals, or call get_all_gpu_metrics() to force samples.

Development

pip install -e ".[dev]"
pytest                 # unit + MCP client tests (mocked hardware)
ruff check src tests
mypy src
python -m build        # package validation

License

MIT — see LICENSE.