Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

llama 3.1 release#1579

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
montanalow merged 2 commits intomasterfrommontana/llama3
Jul 23, 2024
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletionpgml-apps/pgml-chat/pgml_chat/main.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -123,7 +123,7 @@ def handler(signum, frame):
"--chat_completion_model",
dest="chat_completion_model",
type=str,
default="meta-llama/Meta-Llama-3-8B-Instruct",
default="meta-llama/Meta-Llama-3.1-8B-Instruct",
)

parser.add_argument(
Expand Down
37 changes: 37 additions & 0 deletionspgml-cms/blog/announcing-support-for-meta-llama-3.1.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
---
description:>-
Today we’re taking the next steps towards open source AI becoming the industry standard. We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. In addition to having significantly better cost/performance relative to closed models.
featured:false
tags:[engineering]
image:".gitbook/assets/image (2) (2).png"
---

#Announcing Support for Meta Llama 3.1

<divalign="left">

<figure><imgsrc=".gitbook/assets/montana.jpg"alt="Author"width="125"><figcaption></figcaption></figure>

</div>

Montana Low

July 23, 2024

We're pleased to offer Meta Llama 3.1 running in our serverless cloud today. Mark Zuckerberg explained[his company's reasons for championing open source AI](https://about.fb.com/news/2024/07/open-source-ai-is-the-path-forward/), and it's great to see a strong ecosystem forming. These models are now available in our serverless cloud with optimized kernels for maximum throughput.

- meta-llama/Meta-Llama-3.1-8B-Instruct
- meta-llama/Meta-Llama-3.1-70B-Instruct
- meta-llama/Meta-Llama-3.1-405B-Instruct

##Is open-source AI right for you?

We think so. Open-source models have made remarkable strides, not only catching up to proprietary counterparts but also surpassing them across multiple domains. The advantages are clear:

***Performance & reliability:** Open-source models are increasingly comparable or superior across a wide range of tasks and performance metrics. Mistral and Llama-based models, for example, are easily faster than GPT 4. Reliability is another concern you may reconsider leaving in the hands of OpenAI. OpenAI’s API has suffered from several recent outages, and their rate limits can interrupt your app if there is a surge in usage. Open-source models enable greater control over your model’s latency, scalability and availability. Ultimately, the outcome of greater control is that your organization can produce a more dependable integration and a highly reliable production application.
***Safety & privacy:** Open-source models are the clear winner when it comes to security sensitive AI applications. There are[enormous risks](https://www.infosecurity-magazine.com/news-features/chatgpts-datascraping-scrutiny/) associated with transmitting private data to external entities such as OpenAI. By contrast, open-source models retain sensitive information within an organization's own cloud environments. The data never has to leave your premises, so the risk is bypassed altogether – it’s enterprise security by default. At PostgresML, we offer such private hosting of LLM’s in your own cloud.
***Model censorship:** A growing number of experts inside and outside of leading AI companies argue that model restrictions have gone too far. The Atlantic recently published an[article on AI’s “Spicy-Mayo Problem''](https://www.theatlantic.com/ideas/archive/2023/11/ai-safety-regulations-uncensored-models/676076/) which delves into the issues surrounding AI censorship. The titular example describes a chatbot refusing to return commands asking for a “dangerously spicy” mayo recipe. Censorship can affect baseline performance, and in the case of apps for creative work such as Sudowrite, unrestricted open-source models can actually be a key differentiating value for users.
***Flexibility & customization:** Closed-source models like GPT3.5 Turbo are fine for generalized tasks, but leave little room for customization. Fine-tuning is highly restricted. Additionally, the headwinds at OpenAI have exposed the[dangerous reality of AI vendor lock-in](https://techcrunch.com/2023/11/21/openai-dangers-vendor-lock-in/). Open-source models such as MPT-7B, Llama V2 and Mistral 7B are designed with extensive flexibility for fine tuning, so organizations can create custom specifications and optimize model performance for their unique needs. This level of customization and flexibility opens the door for advanced techniques like DPO, PPO LoRa and more.

For a full list of models available in our cloud, check out our[plans and pricing](/pricing).

View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -100,7 +100,7 @@ async def main():
"aggregate": {"join": "\n"},
},
"chat": {
"model": "meta-llama/Meta-Llama-3-8B-Instruct",
"model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
"messages": [
{
"role": "system",
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -44,7 +44,7 @@ The Switch Kit is an open-source AI SDK that provides a drop in replacement for
const korvus = require("korvus");
const client = korvus.newOpenSourceAI();
const results = client.chat_completions_create(
"meta-llama/Meta-Llama-3-8B-Instruct",
"meta-llama/Meta-Llama-3.1-8B-Instruct",
[
{
role: "system",
Expand All@@ -65,7 +65,7 @@ console.log(results);
import korvus
client = korvus.OpenSourceAI()
results = client.chat_completions_create(
"meta-llama/Meta-Llama-3-8B-Instruct",
"meta-llama/Meta-Llama-3.1-8B-Instruct",
[
{
"role": "system",
Expand DownExpand Up@@ -96,7 +96,7 @@ print(results)
],
"created": 1701291672,
"id": "abf042d2-9159-49cb-9fd3-eef16feb246c",
"model": "meta-llama/Meta-Llama-3-8B-Instruct",
"model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
"object": "chat.completion",
"system_fingerprint": "eecec9d4-c28b-5a27-f90b-66c3fb6cee46",
"usage": {
Expand All@@ -113,7 +113,7 @@ We don't charge per token, so OpenAI “usage” metrics are not particularly re

!!!

The above is an example using our open-source AI SDK with Meta-Llama-3-8B-Instruct, an incredibly popular and highly efficient 8 billion parameter model.
The above is an example using our open-source AI SDK with Meta-Llama-3.1-8B-Instruct, an incredibly popular and highly efficient 8 billion parameter model.

Notice there is near one to one relation between the parameters and return type of OpenAI’s `chat.completions.create` and our `chat_completion_create`.

Expand All@@ -125,7 +125,7 @@ Here is an example of streaming:
const korvus = require("korvus");
const client = korvus.newOpenSourceAI();
const it = client.chat_completions_create_stream(
"meta-llama/Meta-Llama-3-8B-Instruct",
"meta-llama/Meta-Llama-3.1-8B-Instruct",
[
{
role: "system",
Expand All@@ -150,7 +150,7 @@ while (!result.done) {
import korvus
client = korvus.OpenSourceAI()
results = client.chat_completions_create_stream(
"meta-llama/Meta-Llama-3-8B-Instruct",
"meta-llama/Meta-Llama-3.1-8B-Instruct",
[
{
"role": "system",
Expand DownExpand Up@@ -182,7 +182,7 @@ for c in results:
],
"created": 1701296792,
"id": "62a817f5-549b-43e0-8f0c-a7cb204ab897",
"model": "meta-llama/Meta-Llama-3-8B-Instruct",
"model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
"object": "chat.completion.chunk",
"system_fingerprint": "f366d657-75f9-9c33-8e57-1e6be2cf62f3"
}
Expand All@@ -198,7 +198,7 @@ for c in results:
],
"created": 1701296792,
"id": "62a817f5-549b-43e0-8f0c-a7cb204ab897",
"model": "meta-llama/Meta-Llama-3-8B-Instruct",
"model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
"object": "chat.completion.chunk",
"system_fingerprint": "f366d657-75f9-9c33-8e57-1e6be2cf62f3"
}
Expand Down
16 changes: 8 additions & 8 deletionspgml-cms/blog/unified-rag.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -51,7 +51,7 @@ Here is an example of the pgml.transform function
SELECT pgml.transform(
task => ''{
"task": "text-generation",
"model": "meta-llama/Meta-Llama-3-8B-Instruct"
"model": "meta-llama/Meta-Llama-3.1-8B-Instruct"
}''::JSONB,
inputs => ARRAY[''AI is going to''],
args => ''{
Expand All@@ -64,7 +64,7 @@ Here is another example of the pgml.transform function
SELECT pgml.transform(
task => ''{
"task": "text-generation",
"model": "meta-llama/Meta-Llama-3-70B-Instruct"
"model": "meta-llama/Meta-Llama-3.1-70B-Instruct"
}''::JSONB,
inputs => ARRAY[''AI is going to''],
args => ''{
Expand DownExpand Up@@ -145,9 +145,9 @@ SELECT * FROM chunks limit 10;
| id | chunk | chunk_index | document_id |
| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | ------------- |
| 1 | Here is an example of the pgml.transform function | 1 | 1 |
| 2 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3-8B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); | 2 | 1 |
| 2 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3.1-8B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); | 2 | 1 |
| 3 | Here is another example of the pgml.transform function | 3 | 1 |
| 4 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3-70B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); | 4 | 1 |
| 4 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3.1-70B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); | 4 | 1 |
| 5 | Here is a third example of the pgml.transform function | 5 | 1 |
| 6 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "microsoft/Phi-3-mini-128k-instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); | 6 | 1 |
| 7 | ae94d3413ae82367c3d0592a67302b25 | 1 | 2 |
Expand DownExpand Up@@ -253,8 +253,8 @@ LIMIT 6;
| 1 | 0.09044166306461232 | Here is an example of the pgml.transform function |
| 3 | 0.10787954026965096 | Here is another example of the pgml.transform function |
| 5 | 0.11683694289239333 | Here is a third example of the pgml.transform function |
| 2 | 0.17699128851412282 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3-8B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 4 | 0.17844729798760672 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3-70B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 2 | 0.17699128851412282 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3.1-8B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 4 | 0.17844729798760672 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3.1-70B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 6 | 0.17520464423854842 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "microsoft/Phi-3-mini-128k-instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |

!!!
Expand DownExpand Up@@ -330,8 +330,8 @@ FROM (

| cosine_distance | rank_score | chunk |
| -------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 0.2124727254737595 | 0.3427378833293915 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3-70B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 0.2109014406365579 | 0.342184841632843 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3-8B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 0.2124727254737595 | 0.3427378833293915 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3.1-70B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 0.2109014406365579 | 0.342184841632843 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "meta-llama/Meta-Llama-3.1-8B-Instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 0.21259646694819168 | 0.3332781493663788 | SELECT pgml.transform(\n task => ''{\n "task": "text-generation",\n "model": "microsoft/Phi-3-mini-128k-instruct"\n }''::JSONB,\n inputs => ARRAY[''AI is going to''],\n args => ''{\n "max_new_tokens": 100\n }''::JSONB\n ); |
| 0.19483324929456136 | 0.03163915500044823 | Here is an example of the pgml.transform function |
| 0.1685870257610742 | 0.031176624819636345 | Here is a third example of the pgml.transform function |
Expand Down
2 changes: 1 addition & 1 deletionpgml-cms/docs/SUMMARY.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -146,7 +146,7 @@
* [Explain plans]()
* [Composition]()
* [LLMs]()
* [LLama]()
* [Llama]()
* [GPT]()
* [Facon]()
* [Glossary]()
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -42,7 +42,7 @@ const pgml = require("pgml");
const main = () => {
const client = pgml.newOpenSourceAI();
const results = client.chat_completions_create(
"meta-llama/Meta-Llama-3-8B-Instruct",
"meta-llama/Meta-Llama-3.1-8B-Instruct",
[
{
role: "system",
Expand All@@ -66,7 +66,7 @@ import pgml
async def main():
client = pgml.OpenSourceAI()
results = client.chat_completions_create(
"meta-llama/Meta-Llama-3-8B-Instruct",
"meta-llama/Meta-Llama-3.1-8B-Instruct",
[
{
"role": "system",
Expand Down
Loading
Loading
[8]ページ先頭
©2009-2025 Movatter.jp