Was ist Inference

Definition

In Machine Learning, inference refers to the phase where a fully trained model is applied to new, unseen, real-time data to make predictions or drive decisions. Unlike the training phase, the model's internal weights remain fixed during inference.

Explanation

While model training demands significant computational power and typically runs in the background, inference is the critical juncture where AI delivers its business value. In a B2B SaaS context, for instance, inference occurs when software calculates the conversion probability of a newly captured lead in real-time, or when an AI assistant generates an email response. Because end-users expect immediate responses, low latency and high efficiency are paramount during this phase. To minimize infrastructure costs for live operations, models are often optimized before deployment, for example, through model compression or quantization. A high-performance inference pipeline is therefore a fundamental prerequisite for AI functionalities to scale seamlessly and cost-effectively within software solutions.