Compare GPU price-to-performance inference benchmarks for common open source large language models using vllm.
Qwen7bqwen2-7b-instruct
1 GPU
NVIDIA HGX B200
NVIDIA HGX B200
AMD MI325X
AMD MI300X
NVIDIA HGX H100
NVIDIA HGX A100
Maker
Model
Cost /GPU/hr
# GPUs
TTFT (ms)
Latency (ms)
Throughput (tok/s)
Actions
Performance metrics used for Custom Models is user-supplied and is not validated by Vultr or any GPU manufacturer as being accurate, and do not represent official results. These are not endorsed by Vultr or any hardware manufacturer, may not reflect actual real-world performance, and should be used for informational or comparative purposes only. By using this tool, you agree not to use this calculator or its results in marketing materials, promotional content, or for making any claims about a particular GPU's performance or price-to-performance.
Price-to-Performance Comparison
GPU - Metric
All GPUs Avg. Performance (ms)0
All GPUs Avg. Price to Performance (ms per USD)0
GPU Absolute Performance (ms)0
GPU Normalized Performance (multiplier)0
GPU Normalized Price to Performance (multiplier)0
Display
Metrics
Table
Large-scale dedicated clusters and flexible on-demand VMs, accelerated by AMD and NVIDIA GPUs
Vultr Cloud GPU, underpinned by Vultr's collaborations with AMD and NVIDIA, simplifies next-gen GPU-accelerated infrastructure. Sidestepping the usual complications of driver setups and licensing, it offers users a direct conduit to the raw power of AMD and NVIDIA GPUs for any computational endeavor.
Start your GPU-accelerated project now by signing up for a free Vultr account. Or, if you’d like to speak with us regarding your needs, please reach out.