Terms that will help you better understand hardware requirements for AI.
Artificial intelligence that runs entirely on your own hardware. Unlike ChatGPT, no data is sent to third-party servers, guaranteeing 100% privacy and security.
The core "brain" of the AI (e.g., ChatGPT/GPT-5.6, Google Gemini, Claude Fable 5/Opus 5, Llama-4, or Qwen) trained on vast amounts of text. It can understand context, write code, or generate creative content.
The basic building block of text processed by an AI. One token corresponds to approximately 4 characters or 3/4 of a word. Models do not read full words directly; they split them into tokens.
A pioneer in commercial AI and the creator of the world-famous ChatGPT chatbot, whose flagship version runs on the GPT-5.6 model (Sol, Terra, and Luna series). It is renowned for the GPT-4 and GPT-4o series, as well as the reasoning models o1 and o3. These closed-source models are accessible via API.
AI models developed by Google. While their flagship multimodal Gemini series (Flash, Pro, Ultra) operates as a powerful cloud service featuring an exceptionally large context window, the Gemma models are open-weights and optimized to run locally on your own hardware.
The Claude model family (e.g., the latest high-performance Claude Fable 5, Claude Opus 5, or the highly intelligent Claude Sonnet 5) developed by Anthropic. These closed-source cloud models focus on high safety alignment, precise coding capabilities, and advanced logical reasoning.
A rapidly expanding ecosystem of both open-weights and proprietary models (such as DeepSeek-V3/R1 and Alibaba's Qwen). They offer high computational efficiency, excellent price-to-performance, and are highly popular for local, on-premise business deployments.
Graphics card optimized for AI computations. Unlike a CPU, it has thousands of small cores and can process millions of mathematical operations simultaneously, making it the engine of AI.
Graphics memory on AI accelerators. The entire AI model and its computations must fit in here. It is the most critical hardware metric for an AI workstation.
A metric (in tokens per second) showing how fast the AI generates answers (words). For local execution, this directly depends on the bandwidth and performance of your GPU's VRAM.
The model's "brain size" indicated in billions (B) of parameters. A higher number means the model is smarter and more capable, but requires much larger VRAM.
The maximum amount of text a model can read and process at one time within a single request. Measured in "tokens" (approximately syllables or parts of words).
A method for connecting AI with a company's internal files in real-time. The AI searches your database before answering, preventing hallucinations (making things up).
The process of taking a pre-trained LLM and further training it on your own company data (like contracts or manuals) so it gains your specific domain expertise.
Memory used by the AI to "remember" context during an ongoing conversation or when analyzing long documents. As the chat history grows, the KV Cache consumes VRAM exponentially.
The phase where the model is no longer being trained, but is actively being used (answering questions, generating code). This process is also extremely demanding on compute and memory.
Model compression by reducing the mathematical precision of its weights (e.g., from 16-bit to 4-bit). It significantly lowers VRAM requirements with minimal generation quality loss.
A standardized test to measure AI model capabilities (e.g., MMLU for general knowledge, GSM8k for math reasoning). It allows objective comparison of intelligence and speed between models.
A popular, user-friendly open-source tool designed to simplify running local LLMs (like Llama 3, Qwen) on personal computers and business workstations. It packages model weights, configurations, and runtime code into easily executable bundles and exposes a local API interface.
A high-throughput, memory-efficient serving engine designed to host LLM models on production servers and API workstations. It features an innovative *PagedAttention* algorithm (similar to virtual memory paging in OS), enabling servers to handle dozens of concurrent user interactions at extreme speed while saving VRAM.
A foundational, low-level open-source library written in pure C/C++ that serves as the engine for many local AI applications (including Ollama). It allows running quantized LLM models on consumer-grade hardware (including CPUs and graphics cards from various vendors) with minimal dependencies and maximum raw performance.