Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Dan blog landing page#1294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
chillenberger merged 10 commits intomasterfromdan-blog-landing-page
Jan 19, 2024
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
View file
Open in desktop
Loading
Sorry, something went wrong.Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View file
Open in desktop
Loading
Sorry, something went wrong.Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View file
Open in desktop
Loading
Sorry, something went wrong.Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View file
Open in desktop
Loading
Sorry, something went wrong.Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,9 @@ description: >-
GPTQ & GGML allow PostgresML to fit larger models in less RAM. These
algorithms perform inference significantly faster on NVIDIA, Apple and Intel
hardware.
featured: false
tags: [engineering]
image: ".gitbook/assets/image (14).png"
---

# Announcing GPTQ & GGML Quantized LLM support for Huggingface Transformers
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
---
description: >-
We added aws us east 1 to our list of support aws regions.
featured: false
tags: [product]
---
# Announcing Support for AWS us-east-1 Region

<div align="left">
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,6 +2,8 @@
description: >-
A story about including Scikit-learn into our Rust extension and preserving
backwards compatibility in the process.
tags: [engineering]
featured: false
---

# Backwards Compatible or Bust: Python Inside Rust Inside Postgres
Expand Down
2 changes: 2 additions & 0 deletionspgml-cms/blog/data-is-living-and-relational.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,8 @@ description: >-
A common problem with data science and machine learning tutorials is the
published and studied datasets are often nothing like what you’ll find in
industry.
featured: false
tags: [engineering]
---

# Data is Living and Relational
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,6 +2,8 @@
description: >-
How to use the pgml.embed(...) function to generate embeddings with free and
open source models in your own database.
image: ".gitbook/assets/blog_image_generating_llm_embeddings.png"
features: true
---

# Generating LLM embeddings with open source models in PostgresML
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,9 @@ description: >-
PostgresML makes it easy to use machine learning on your data and scale
workloads horizontally in our cloud. One of the most common use cases is to
improve search results.
featured: true
image: ".gitbook/assets/image (2) (2).png"
tags: ["Engineering"]
---

# How-to Improve Search Results with Machine Learning
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
---
image: https://postgresml.org/dashboard/static/images/open_source_ai_social_share.png
featured: true
tags: [engineering, product]
description: >-
Quickly and easily transition from the confines of the OpenAI APIs to higher
quality embeddings and unrestricted text generation models.
image: ".gitbook/assets/blog_image_switch_kit.png"
---

# Introducing the OpenAI Switch Kit: Move from closed to open-source AI in minutes
Expand Down
1 change: 1 addition & 0 deletionspgml-cms/blog/mindsdb-vs-postgresml.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,6 +2,7 @@
description: >-
PostgresML is more opinionated, more scalable, more capable and several times
faster than MindsDB.
image: ".gitbook/assets/image (17).png"
---

# MindsDB vs PostgresML
Expand Down
1 change: 1 addition & 0 deletionspgml-cms/blog/postgres-full-text-search-is-awesome.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,6 +2,7 @@
description: >-
If you want to improve your search results, don't rely on expensive O(n*m)
word frequency statistics. Get new sources of data instead.
image: ".gitbook/assets/image (53).png"
---

# Postgres Full Text Search is Awesome!
Expand Down
3 changes: 3 additions & 0 deletionspgml-cms/blog/speeding-up-vector-recall-5x-with-hnsw.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,9 @@ description: >-
HNSW indexing is the latest upgrade in vector recall performance. In this post
we announce our updated SDK that utilizes HNSW indexing to give world class
performance in vector search.
tags: [engineering]
featured: true
image: ".gitbook/assets/blog_image_hnsw.png"
---

# Speeding up vector recall 5x with HNSW
Expand Down
1 change: 1 addition & 0 deletionspgml-dashboard/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

2 changes: 1 addition & 1 deletionpgml-dashboard/Cargo.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,7 +15,7 @@ anyhow = "1"
aho-corasick = "0.7"
base64 = "0.21"
comrak = "0.17"
chrono = "0.4"
chrono ={ version ="0.4", features = ["serde"] }
csv-async = "1"
console-subscriber = "*"
convert_case = "0.6"
Expand Down
163 changes: 132 additions & 31 deletionspgml-dashboard/src/api/cms.rs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,8 @@ use std::{
path::{Path, PathBuf},
};

use std::str::FromStr;

use comrak::{format_html_with_plugins, parse_document, Arena, ComrakPlugins};
use lazy_static::lazy_static;
use markdown::mdast::Node;
Expand All@@ -16,7 +18,6 @@ use crate::{
templates::docs::*,
utils::config,
};

use serde::{Deserialize, Serialize};

lazy_static! {
Expand DownExpand Up@@ -61,35 +62,66 @@ lazy_static! {
);
}

#[derive(PartialEq, Debug, Serialize, Deserialize)]
pub enum DocType {
Blog,
Docs,
Careers,
}

impl FromStr for DocType {
type Err = ();

fn from_str(s: &str) -> Result<DocType, Self::Err> {
match s {
"blog" => Ok(DocType::Blog),
"Doc" => Ok(DocType::Docs),
"Careers" => Ok(DocType::Careers),
_ => Err(()),
}
}
}

#[derive(Debug, Serialize, Deserialize)]
pub struct Document {
/// The absolute path on disk
pub path: PathBuf,
pub description: Option<String>,
pub author: Option<String>,
pub author_image: Option<String>,
pub featured: bool,
pub date: Option<chrono::NaiveDate>,
pub tags: Vec<String>,
pub image: Option<String>,
pub title: String,
pub toc_links: Vec<TocLink>,
pub html: String,
pub contents: String,
pub doc_type: Option<DocType>,
}

// Gets document markdown
impl Document {
pub async fn from_path(path: &PathBuf) -> anyhow::Result<Document, std::io::Error> {
warn!("path: {:?}", path);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Never noticed the ^warn! above. It's a bit too noisy for production, could we move that todebug!?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

yeah, I can do that in my next merge.

let regex = regex::Regex::new(r#".*/pgml-cms/([^"]*)/(.*)\.md"#).unwrap();

let doc_type = match regex.captures(&path.clone().display().to_string()) {
Some(c) => DocType::from_str(&c[1]).ok(),
_ => None,
};

let contents = tokio::fs::read_to_string(&path).await?;

let parts = contents.split("---").collect::<Vec<&str>>();

let (description, contents) = if parts.len() > 1 {
let (meta, contents) = if parts.len() > 1 {
match YamlLoader::load_from_str(parts[1]) {
Ok(meta) => {
if meta.len() == 0 || meta[0].as_hash().is_none() {
(None, contents)
} else {
let description: Option<String> = match meta[0]["description"].is_badvalue() {
true => None,
false => Some(meta[0]["description"].as_str().unwrap().to_string()),
};
(description, parts[2..].join("---").to_string())
(Some(meta[0].clone()), parts[2..].join("---").to_string())
}
}
Err(_) => (None, contents),
Expand All@@ -98,16 +130,78 @@ impl Document {
(None, contents)
};

// parse meta section
let (description, image, featured, tags) = match meta {
Some(meta) => {
let description = if meta["description"].is_badvalue() {
None
} else {
Some(meta["description"].as_str().unwrap().to_string())
};

let image = if meta["image"].is_badvalue() {
Some(".gitbook/assets/blog_image_placeholder.png".to_string())
} else {
Some(meta["image"].as_str().unwrap().to_string())
};

let featured = if meta["featured"].is_badvalue() {
false
} else {
meta["featured"].as_bool().unwrap()
};

let tags = if meta["tags"].is_badvalue() {
Vec::new()
} else {
let mut tags = Vec::new();
for tag in meta["tags"].as_vec().unwrap() {
tags.push(tag.as_str().unwrap_or_else(|| "").to_string());
}
tags
};

(description, image, featured, tags)
}
None => (
None,
Some(".gitbook/assets/blog_image_placeholder.png".to_string()),
false,
Vec::new(),
),
};

// Parse Markdown
let arena = Arena::new();
let spaced_contents = crate::utils::markdown::gitbook_preprocess(&contents);
let root = parse_document(&arena, &spaced_contents, &crate::utils::markdown::options());

// Title of the document is the first (and typically only) <h1>
let root = parse_document(&arena, &contents, &crate::utils::markdown::options());
let title = crate::utils::markdown::get_title(root).unwrap();
let toc_links = crate::utils::markdown::get_toc(root).unwrap();
let image = crate::utils::markdown::get_image(root);
crate::utils::markdown::wrap_tables(root, &arena).unwrap();
let (author, date, author_image) = crate::utils::markdown::get_author(root);

let document = Document {
path: path.to_owned(),
description,
author,
author_image,
date,
featured,
tags,
image,
title,
toc_links,
contents,
doc_type,
};
Ok(document)
}

pub fn html(self) -> String {
let contents = self.contents;

// Parse Markdown
let arena = Arena::new();
let spaced_contents = crate::utils::markdown::gitbook_preprocess(&contents);
let root = parse_document(&arena, &spaced_contents, &crate::utils::markdown::options());

// MkDocs, gitbook syntax support, e.g. tabs, notes, alerts, etc.
crate::utils::markdown::mkdocs(root, &arena).unwrap();
Expand All@@ -122,31 +216,23 @@ impl Document {
format_html_with_plugins(root, &crate::utils::markdown::options(), &mut html, &plugins).unwrap();
let html = String::from_utf8(html).unwrap();

let document = Document {
path: path.to_owned(),
description,
image,
title,
toc_links,
html,
};
Ok(document)
html
}
}

/// A Gitbook collection of documents
#[derive(Default)]
struct Collection {
pubstruct Collection {
/// The properly capitalized identifier for this collection
name: String,
/// The root location on disk for this collection
root_dir: PathBuf,
pubroot_dir: PathBuf,
/// The root location for gitbook assets
asset_dir: PathBuf,
pubasset_dir: PathBuf,
/// The base url for this collection
url_root: PathBuf,
/// A hierarchical list of content in this collection
index: Vec<IndexLink>,
pubindex: Vec<IndexLink>,
/// A list of old paths to new paths in this collection
redirects: HashMap<&'static str, &'static str>,
}
Expand DownExpand Up@@ -303,7 +389,7 @@ impl Collection {
}

// Sets specified index as currently viewed.
fn open_index(&self, path: PathBuf) -> Vec<IndexLink> {
fn open_index(&self, path:&PathBuf) -> Vec<IndexLink> {
self.index
.clone()
.iter_mut()
Expand All@@ -330,10 +416,10 @@ impl Collection {

match Document::from_path(&path).await {
Ok(doc) => {
let index = self.open_index(doc.path);
let index = self.open_index(&doc.path);

let mut layout = crate::templates::Layout::new(&doc.title, Some(cluster));
if let Some(image) = doc.image {
if let Some(image) =&doc.image {
layout.image(&config::asset_url(image.into()));
}
if let Some(description) = &doc.description {
Expand All@@ -351,7 +437,7 @@ impl Collection {
.footer(cluster.context.marketing_footer.to_string());

Ok(ResponseOk(
layout.render(crate::templates::Article { content: doc.html }),
layout.render(crate::templates::Article { content: doc.html() }),
))
}
// Return page not found on bad path
Expand DownExpand Up@@ -438,6 +524,20 @@ async fn get_docs(
DOCS.get_content(path, cluster, origin).await
}

#[get("/blog")]
async fn blog_landing_page(cluster: &Cluster) -> Result<ResponseOk, crate::responses::NotFound> {
let layout = crate::components::layouts::marketing::Base::new("Blog landing page", Some(cluster))
.footer(cluster.context.marketing_footer.to_string());

Ok(ResponseOk(
layout.render(
crate::components::pages::blog::LandingPage::new(cluster)
.index(&BLOG)
.await,
),
))
}

#[get("/user_guides/<path..>", rank = 5)]
async fn get_user_guides(
path: PathBuf,
Expand All@@ -449,6 +549,7 @@ async fn get_user_guides(

pub fn routes() -> Vec<Route> {
routes![
blog_landing_page,
get_blog,
get_blog_asset,
get_careers,
Expand Down
Loading

[8]ページ先頭

©2009-2025 Movatter.jp