Introducing an accessible open source AI platform

RHEL AI simplifies deployment across hybrid infrastructure environments by integrating enterprise-ready versions ofInstructLab and Granite language models, along with a bootable image of Red Hat Enterprise Linux. This forms a foundation model platform for bringing open source-licensed gen AI models into the enterprise, lowering costs and removing barriers to testing and experimentation. The platform supports tools like vLLM, DeepSpeed, and PyTorch, and leverages containerization with Podman, RHEL Image mode, and scalable deployments using Kubernetes and OpenShift. 

Users can join a growing community of AI contributors through InstructLab, enhancing large language models with additional skills and data cost-effectively.

Introducing Red Hat AI Inference Server

Deploy your preferred models faster and more cost-effectively across the hybrid cloud with Red Hat AI Inference Server. Its vLLM runtime maximizes inference throughput and minimizes latency. A pre-optimized model repository ensures rapid model serving, while the LLM compressor reduces compute costs without sacrificing accuracy. Experience fast, accurate inference for a wide range of applications.

Red Hat AI Inference Server is included in Red Hat OpenShift AI and Red Hat Enterprise Linux AI and supported on Red Hat OpenShift and Red Hat Enterprise Linux.

View on Red Hat Catalog

Learn More

Introducing Red Hat AI Inference Server

Deploy your preferred models faster and more cost-effectively across the hybrid cloud with Red Hat AI Inference Server. Its vLLM runtime maximizes inference throughput and minimizes latency. A pre-optimized model repository ensures rapid model serving, while the LLM compressor reduces compute costs without sacrificing accuracy. Experience fast, accurate inference for a wide range of applications.

Red Hat AI Inference Server is included in Red Hat OpenShift AI and Red Hat Enterprise Linux AI and supported on Red Hat OpenShift and Red Hat Enterprise Linux.