Products & documentation Red Hat AI
A platform of products and services for the development and deployment of AI across the hybrid cloud.
Red Hat AI Inference Server
Optimize model performance with vLLM for fast and cost-effective inference at scale.
Red Hat Enterprise Linux AI
Develop, test, and run generative AI models to power enterprise applications.
Red Hat OpenShift AI
Build and deploy AI-enabled applications and models at scale across hybrid environments.
Learn AI partners
- Home
- Products
- Red Hat AI
- Red Hat OpenShift AI
Red Hat OpenShift AI
Red Hat® OpenShift® AI is a platform for managing the lifecycle of predictive and generative AI (gen AI) models, at scale, across hybrid cloud environments.
What is Red Hat OpenShift AI?
Built using open source technologies, OpenShift AI provides trusted, operationally consistent capabilities for teams to experiment, serve models, and deliver innovative applications.
OpenShift AI enables data acquisition and preparation, model training and fine-tuning, model serving and model monitoring, and hardware acceleration. With an open ecosystem of hardware and software partners, OpenShift AI delivers the flexibility you need for your specific use cases.
This video can't play due to privacy settings
To change your settings, select the "Cookie Preferences" link in the footer and opt in to "Advertising Cookies."
Launch AI faster in any environment. Video duration: 5:26
Bring AI-enabled applications to production faster
Combine the proven capabilities of Red Hat OpenShift AI and Red Hat OpenShift in a single enterprise-ready AI application platform that brings teams together. Data scientists, engineers, and app developers can collaborate in a single destination that promotes consistency, security, and scalability.
The latest release of OpenShift AI includes a curated collection of optimized, production-ready, third-party models, validated for Red Hat OpenShift AI. Access to this third-party model catalog gives your team more control over model accessibility and visibility to help meet security and policy requirements.
Additionally, OpenShift AI helps manage costs of inferencing with distributed serving through an optimized vLLM framework. To further reduce operational complexity, it offers advanced tooling to automate deployments and self-service access to models, tools, and resources.
Features & benefits
Less time managing AI infrastructure
Provide your teams with on-demand access to resources, so they can self-service and scale their model training and serving environments as needed. Additionally, reduce operational complexity by managing AI accelerators (GPUs) and workload resources across a scalable clustered environment.
Tested and supported AI/ML tooling
Red Hat tests, integrates, and supports AI/ML tooling and model serving, so you don’t have to. OpenShift AI draws from years of incubation in our Open Data Hub community project and open source projects like Kubeflow.
Our experience and open source expertise allow us to provide a generative AI-ready foundation, so customers can have greater choice and confidence in their generative AI strategies.
Flexibility across the hybrid cloud
Offered as either self-managed software or as a fully managed cloud service on top of OpenShift, Red Hat OpenShift AI provides a secure and flexible platform that gives you the choice of where you develop and deploy your models–whether on-premise, the public cloud, or even at the edge.
Leverage our best practices
Red Hat Services provides expertise, training, and support that helps you overcome the challenges of AI no matter where you are in your adoption journey.
Whether you’d like to prototype an AI solution, streamline the deployment of your AI platform, or advance your MLOps strategies, Red Hat Consulting will provide support and mentorship.
Optimize with vLLM for fast and cost-effective inference at scale.
Red Hat AI Inference Server is part of the Red Hat AI platform. It is available as a standalone product and is included in both Red Hat Enterprise Linux® AI and Red Hat OpenShift® AI.
Partnerships
Get more from the Red Hat OpenShift AI platform by extending it with other integrated services and products.
NVIDIA and Red Hat offer customers a scalable platform that accelerates a diverse range of AI use cases with unparalleled flexibility.
Intel® and Red Hat help organizations accelerate AI adoption and rapidly operationalize their AI/ML models.
IBMand Red Hat provide open source innovation to accelerate AI development, including through IBM watsonx.aiTM, an enterprise-ready AI studio for AI builders.
Starburst Enterprise and Red Hat support better and more timely insights through rapid data analysis across multiple disparate and distributed data platforms.
Scalable Kubernetes Infrastructure for AI platforms
Learn how to apply machine learning operations (MLOPs) principles and practices to build AI-powered applications.
Collaborating through model workbenches
Provide pre-built or customized cluster images to your data scientists to work with models using their preferred IDE, such as JupyterLab. Red Hat OpenShift AI tracks changes to Jupyter, TensorFlow, and PyTorch, and other open source AI technologies.
Scaling model serving with Red Hat OpenShift AI
Models can be served using an optimized version of vLLM (or other model servers of your choice) for integration into AI-enabled applications on-premise, in the public cloud or at the edge. These models can be rebuilt, redeployed, and monitored based on changes to the source notebook.
Bias and drift can compromise the integrity of your models and make it harder to scale. To help maintain fairness, safety, and scalability, OpenShift AI allows data practitioners to monitor alignment between model outputs and training data.
Drift detection tools can monitor when live data used for model inference deviates from original training data. AI guardrails are also included to protect your model inputs and outputs from harmful information such as abusive and profane speech, personal data, or domain-specific restrictions.
Solution Pattern
Red Hat AI applications with NVIDIA AI Enterprise
Create a RAG application
Red Hat OpenShift AI is a platform for building data science projects and serving AI-enabled applications. You can integrate all the tools you need to support retrieval-augmented generation (RAG), a method for getting AI answers from your own reference documents. When you connect OpenShift AI with NVIDIA AI Enterprise, you can experiment with large language models (LLMs) to find the optimal model for your application.
Build a pipeline for documents
To make use of RAG, you first need to ingest your documents into a vector database. In our example app, we embed a set of product documents in a Redis database. Since these documents change frequently, we can create a pipeline for this process that we’ll run periodically, so we always have the latest versions of the documents.
Browse the LLM catalog
NVIDIA AI Enterprise gives you access to a catalog of different LLMs, so you can try different choices and select the model that delivers the best results. The models are hosted in the NVIDIA API catalog. Once you’ve set up an API token, you candeploy a model using the NVIDIA NIM model serving platform directly from OpenShift AI.
Choose the right model
As you test different LLMs, your users can rate each generated response. You can set up a Grafana monitoring dashboard to compare the ratings, as well as latency and response time for each model. Then you can use that data to choose the best LLM to use in production.
How to try Red Hat OpenShift AI
Developer Sandbox
For developers and data scientists who want to experiment with building AI-enabled applications in a preconfigured and flexible environment.
60-day trial
When your organization is ready to evaluate the full capabilities of OpenShift AI, explore them with a 60-day product trial. An existing Red Hat OpenShift cluster is required.
