- Notifications
You must be signed in to change notification settings - Fork6.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
docs: update ray serve section#48770
base:master
Are you sure you want to change the base?
Conversation
Signed-off-by: Saihajpreet Singh <c-saihajpreet.singh@anyscale.com>
| Ray Serve is particularly well suited for [model composition](serve-model-composition) and many model serving, enabling you to build a complex inference service consisting of multiple ML models and business logic all in Python code. | ||
| Ray Serve is built on top of Ray, so it easily scales to many machines and offers flexible scheduling support such as fractional GPUs so you can share resources and serve many machine learning models at low cost. | ||
| Ray Serve is built on top of Ray, so it easily scales to many machines and offers flexible scheduling support such as fractional GPUs—so you can share resources and serve many machine learning models at low cost. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
| Ray Serve is built on top of Ray, so it easily scales to many machines and offers flexible scheduling support such as fractional GPUs—so you can share resources and serve many machine learning models at low cost. | |
| Ray Serve is built on top of Ray, so it easily scales to many machines and offers flexible scheduling support such as fractional GPUs. You can share resources and serve many machine learning models at low cost. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Just getting rid of the secondso.
| @@ -35,12 +35,9 @@ api/index | |||
| (rayserve-overview)= | |||
| Ray Serve is a scalable model serving library for building online inference APIs. | |||
| Serve is framework-agnostic, so you can use a single toolkit to serve everything from deep learning models built with frameworks like PyTorch, TensorFlow, and Keras, to Scikit-Learn models, to arbitrary Python business logic. It has several features and performance optimizations for serving Large Language Models such as response streaming, dynamic request batching, multi-node/multi-GPU serving, etc. | |||
| Ray Serve is a scalable, framework-agnostic model serving library for building online inference APIs. Serve integrates with any ML framework including PyTorch, TensorFlow, Keras, Scikit-Learn, and more. It's particularly well suited for model composition and many model serving, and includes performance optimizations for serving LLMs such as response streaming, dynamic request batching, multi-mode/multi-GPU serving, and more. | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Could you add a link to the first occurence ofRay Serve that goes to where the marketing content for Serve moved to?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
let's not forget this piece - "to arbitrary Python business logic" -> that's an important highlight. Also model composition and many model serving are more differentiated than the LLM serving features
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.
|
Signed-off-by: Saihajpreet Singhc-saihajpreet.singh@anyscale.com
Moving marketing type content to anyscale.com so we can keep user documentation concise. The removed content will now live here:https://www.anyscale.com/product/library/ray-serve
See this doc for more context:
ttps://docs.google.com/document/d/10xTHUhFDDD214xIeKPZK4Jrls_bdCvjR8phKQXygmgY/edit?tab=t.0