Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gitbook assets#1052

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
montanalow merged 2 commits intomasterfrommontana/gitbook
Oct 11, 2023
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 6 additions & 7 deletionsREADME.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -138,6 +138,10 @@ SELECT pgml.predict(
# Installation
PostgresML installation consists of three parts: PostgreSQL database, Postgres extension for machine learning and a dashboard app. The extension provides all the machine learning functionality and can be used independently using any SQL IDE. The dashboard app provides an easy to use interface for writing SQL notebooks, performing and tracking ML experiments and ML models.

## Serverless Cloud

If you want to check out the functionality without the hassle of Docker, [sign up for a free PostgresML account](https://postgresml.org/signup). You'll get a free database in seconds, with access to GPUs and state of the art LLMs.

## Docker

```
Expand All@@ -150,19 +154,14 @@ docker run \
sudo -u postgresml psql -d postgresml
```

For more details, take a look at our [Quick Start with Docker](https://postgresml.org/docs/guides/setup/quick_start_with_docker) documentation.

## Serverless Cloud

If you want to check out the functionality without the hassle of Docker, [sign up for a free PostgresML account](https://postgresml.org/signup). You'll get a free database in seconds, with access to GPUs and state of the art LLMs.
For more details, take a look at our [Quick Start with Docker](https://postgresml.org/docs/guides/developer-docs/quick-start-with-docker) documentation.

# Getting Started

## Option 1

- On local installation, go to dashboard app at `http://localhost:8000/` to use SQL notebooks.

- On the cloud console click on the **Dashboard** button to connect to your instance with a SQL notebook, or connect directly with tools listed below.
- On local installation, go to dashboard app at `http://localhost:8000/` to use SQL notebooks.

## Option 2

Expand Down
1 change: 1 addition & 0 deletionsdocker/dashboard.sh
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@ set -e
export DATABASE_URL=postgres://postgresml:postgresml@127.0.0.1:5432/postgresml
export DASHBOARD_STATIC_DIRECTORY=/usr/share/pgml-dashboard/dashboard-static
export DASHBOARD_CONTENT_DIRECTORY=/usr/share/pgml-dashboard/dashboard-content
export DASHBOARD_CONTENT_DOCS=/usr/share/pgml-docs
export SEARCH_INDEX_DIRECTORY=/var/lib/pgml-dashboard/search-index
export ROCKET_SECRET_KEY=$(openssl rand -hex 32)
export ROCKET_ADDRESS=0.0.0.0
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@ StartLimitIntervalSec=0
Environment=RUST_LOG=info
Environment=DASHBOARD_STATIC_DIRECTORY=/usr/share/pgml-dashboard/dashboard-static
Environment=DASHBOARD_CONTENT_DIRECTORY=/usr/share/pgml-dashboard/dashboard-content
Environment=DASHBOARD_CONTENT_DIRECTORY=/usr/share/pgml-docs
Environment=ROCKET_ADDRESS=0.0.0.0
Environment=GITHUB_STARS=${GITHUB_STARS}
Environment=SEARCH_INDEX_DIRECTORY=/var/lib/pgml-dashboard/search-index
Expand Down
1 change: 1 addition & 0 deletionspgml-dashboard/.env.development
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
DATABASE_URL=postgres:///pgml_dashboard_development
DEV_MODE=true
RUST_LOG=debug,tantivy=error,rocket=info
27 changes: 27 additions & 0 deletionspgml-dashboard/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

3 changes: 3 additions & 0 deletionspgml-dashboard/Cargo.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,12 +17,14 @@ base64 = "0.21"
comrak = "0.17"
chrono = "0.4"
csv-async = "1"
convert_case = "0.6"
dotenv = "0.15"
env_logger = "0.10"
itertools = "0.10"
parking_lot = "0.12"
lazy_static = "1.4"
log = "0.4"
markdown = "1.0.0-alpha.13"
num-traits = "0.2"
once_cell = "1.18"
rand = "0.8"
Expand All@@ -39,6 +41,7 @@ sqlx = { version = "0.6.3", features = [ "runtime-tokio-rustls", "postgres", "js
tantivy = "0.19"
time = "0.3"
tokio = { version = "1", features = ["full"] }
url = "2.4"
yaml-rust = "0.4"
zoomies = { git="https://github.com/HyperparamAI/zoomies.git", branch="master" }
pgvector = { version = "0.2.2", features = [ "sqlx", "postgres" ] }
Expand Down
2 changes: 1 addition & 1 deletionpgml-dashboard/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,4 +2,4 @@

PostgresML provides a dashboard with analytical views of the training data and model performance, as well as integrated notebooks for rapid iteration. It is primarily written in Rust using [Rocket](https://rocket.rs/) as a lightweight web framework and [SQLx](https://github.com/launchbadge/sqlx) to interact with the database.

Please see the [quick start instructions](https://postgresml.org/user_guides/setup/quick_start_with_docker/) for general information on installing or deploying PostgresML. A [developer guide](https://postgresml.org/developer_guide/overview/) is also available for those who would like to contribute.
Please see the [quick start instructions](https://postgresml.org/docs/guides/getting-started/sign-up) for general information on installing or deploying PostgresML. A [developer guide](https://postgresml.org/developer_guide/overview/) is also available for those who would like to contribute.
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
# Dashboard

PostgresML comes with a web app to provide visibility into models and datasets in your database. If you're running [our Docker container](/docs/guides/setup/quick_start_with_docker/), you can view it running on [http://localhost:8000/](http://localhost:8000/).
PostgresML comes with a web app to provide visibility into models and datasets in your database. If you're running [our Docker container](/docs/guides/developer-docs/quick-start-with-docker), you can view it running on [http://localhost:8000/](http://localhost:8000/).


## Generate example data
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -22,7 +22,7 @@ psql \
-f dump.sql
```

If you're using our <a href="/docs/guides/setup/quick_start_with_docker">Docker</a> stack, you can import the data there:</p>
If you're using our <a href="/docs/guides/developer-docs/quick-start-with-docker">Docker</a> stack, you can import the data there:</p>

```
psql \
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ The extension can be installed by compiling it from source, or if you're using U

!!! tip

If you're just looking to try PostgresML without installing it on your system, take a look at our [Quick Start with Docker](/docs/guides/setup/quick_start_with_docker) guide.
If you're just looking to try PostgresML without installing it on your system, take a look at our [Quick Start with Docker](/docs/guides/developer-docs/quick-start-with-docker) guide.

!!!

Expand Down
85 changes: 35 additions & 50 deletionspgml-dashboard/src/api/docs.rs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -24,54 +24,36 @@ async fn search(query: &str, index: &State<markdown::SearchIndex>) -> ResponseOk
)
}

#[get("/docs/<path..>", rank = 10)]
async fn doc_handler<'a>(path: PathBuf, cluster: &Cluster) -> Result<ResponseOk, Status> {
let guides = vec![
NavLink::new("Setup").children(vec![
NavLink::new("Installation").children(vec![
NavLink::new("v2").href("/docs/guides/setup/v2/installation"),
NavLink::new("Upgrade from v1.0 to v2.0")
.href("/docs/guides/setup/v2/upgrade-from-v1"),
NavLink::new("v1").href("/docs/guides/setup/installation"),
]),
NavLink::new("Quick Start with Docker")
.href("/docs/guides/setup/quick_start_with_docker"),
NavLink::new("Distributed Training").href("/docs/guides/setup/distributed_training"),
NavLink::new("GPU Support").href("/docs/guides/setup/gpu_support"),
NavLink::new("Developer Setup").href("/docs/guides/setup/developers"),
]),
NavLink::new("Training").children(vec![
NavLink::new("Overview").href("/docs/guides/training/overview"),
NavLink::new("Algorithm Selection").href("/docs/guides/training/algorithm_selection"),
NavLink::new("Hyperparameter Search")
.href("/docs/guides/training/hyperparameter_search"),
NavLink::new("Preprocessing Data").href("/docs/guides/training/preprocessing"),
NavLink::new("Joint Optimization").href("/docs/guides/training/joint_optimization"),
]),
NavLink::new("Predictions").children(vec![
NavLink::new("Overview").href("/docs/guides/predictions/overview"),
NavLink::new("Deployments").href("/docs/guides/predictions/deployments"),
NavLink::new("Batch Predictions").href("/docs/guides/predictions/batch"),
]),
NavLink::new("Transformers").children(vec![
NavLink::new("Setup").href("/docs/guides/transformers/setup"),
NavLink::new("Pre-trained Models").href("/docs/guides/transformers/pre_trained_models"),
NavLink::new("Fine Tuning").href("/docs/guides/transformers/fine_tuning"),
NavLink::new("Embeddings").href("/docs/guides/transformers/embeddings"),
]),
NavLink::new("Vector Operations").children(vec![
NavLink::new("Overview").href("/docs/guides/vector_operations/overview")
]),
NavLink::new("Dashboard").href("/docs/guides/dashboard/overview"),
NavLink::new("Schema").children(vec![
NavLink::new("Models").href("/docs/guides/schema/models"),
NavLink::new("Snapshots").href("/docs/guides/schema/snapshots"),
NavLink::new("Projects").href("/docs/guides/schema/projects"),
NavLink::new("Deployments").href("/docs/guides/schema/deployments"),
]),
];

render(cluster, &path, guides, "Guides", &Path::new("docs")).await
use rocket::fs::NamedFile;

#[get("/docs/guides/.gitbook/assets/<path>", rank = 10)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Can't use the fileserver because we need the path to not be exact, but rather the relative gitbook path.

pub async fn gitbook_assets(path: PathBuf) -> Option<NamedFile> {
let path = PathBuf::from(&config::docs_dir())
.join("docs/guides/.gitbook/assets/")
.join(path);

NamedFile::open(path).await.ok()
}

#[get("/docs/<path..>", rank = 5)]
async fn doc_handler(path: PathBuf, cluster: &Cluster) -> Result<ResponseOk, Status> {
let root = PathBuf::from("docs/guides/");
let index_path = PathBuf::from(&config::docs_dir())
.join(&root)
.join("SUMMARY.md");
let contents = tokio::fs::read_to_string(&index_path).await.expect(
format!(
"could not read table of contents markdown: {:?}",
index_path
)
.as_str(),
);
let mdast = ::markdown::to_mdast(&contents, &::markdown::ParseOptions::default())
.expect("could not parse table of contents markdown");
let guides = markdown::parse_summary_into_nav_links(&mdast)
.expect("could not extract nav links from table of contents");

render(cluster, &path, guides, "Guides", &Path::new("docs"), &config::docs_dir()).await
}

#[get("/blog/<path..>", rank = 10)]
Expand DownExpand Up@@ -134,6 +116,7 @@ async fn blog_handler<'a>(path: PathBuf, cluster: &Cluster) -> Result<ResponseOk
],
"Blog",
&Path::new("blog"),
&config::blogs_dir(),
)
.await
}
Expand All@@ -144,14 +127,16 @@ async fn render<'a>(
mut nav_links: Vec<NavLink>,
nav_title: &'a str,
folder: &'a Path,
content: &'a str,
) -> Result<ResponseOk, Status> {
let url = path.clone();

// Get the document content
let path = Path::new(&config::content_dir())
let path = Path::new(&content)
.join(folder)
.join(&(path.to_str().unwrap().to_string() + ".md"));

info!("path: {:?}", path);
// Read to string
let contents = match tokio::fs::read_to_string(&path).await {
Ok(contents) => contents,
Expand DownExpand Up@@ -244,7 +229,7 @@ async fn render<'a>(
}

pub fn routes() -> Vec<Route> {
routes![doc_handler, blog_handler, search]
routes![gitbook_assets,doc_handler, blog_handler, search]
}

#[cfg(test)]
Expand Down
2 changes: 1 addition & 1 deletionpgml-dashboard/src/components/navbar/template.html
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,7 +15,7 @@
<div class="collapse navbar-collapse" id="navbarSupportedContent">
<ul class="navbar-nav flex-grow-1 gap-4 me-auto mb-4 mb-lg-0">
<li class="nav-item d-flex align-items-center">
<a class="nav-link p-0" href="/docs/guides/setup/quick_start_with_docker">Docs</a>
<a class="nav-link p-0" href="/docs/guides/developer-docs/quick-start-with-docker">Docs</a>
</li>
<% if !standalone_dashboard { %>
<!-- <li class="nav-item d-flex align-items-center">
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@
</li>

<li class="nav-item d-flex align-items-center">
<a class="nav-link p-lg-0" href="/docs/guides/setup/quick_start_with_docker">Docs</a>
<a class="nav-link p-lg-0" href="/docs/guides/README">Docs</a>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Not very SEO friendly imo

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Yeah, but we can't control this lever in gitbook... and then again... it is kinda devvy.

</li>

<div class="vr my-2 opacity-100 d-lg-block d-none" style="width: 2px"></div>
Expand DownExpand Up@@ -78,7 +78,7 @@
</li>

<li class="menu-item rounded-0 d-flex align-items-center">
<a href="/docs/guides/setup/quick_start_with_docker/">Docs</a>
<a href="/docs/guides/README/">Docs</a>
</li>

<li class="menu-item rounded-0 d-flex align-items-center">
Expand Down
2 changes: 1 addition & 1 deletionpgml-dashboard/src/main.rs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -286,7 +286,7 @@ mod test {
async fn test_docs() {
let client = Client::tracked(rocket().await).await.unwrap();
let response = client
.get("/docs/guides/setup/quick_start_with_docker")
.get("/docs/guides/README")
.dispatch()
.await;
assert_eq!(response.status().code, 200);
Expand Down
6 changes: 3 additions & 3 deletionspgml-dashboard/src/templates/docs.rs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -19,16 +19,16 @@ impl NavLink {
pub fn new(title: &str) -> NavLink {
NavLink {
id: crate::utils::random_string(25),
title: title.to_string(),
href: "#".to_string(),
title: title.to_owned(),
href: "#".to_owned(),
children: vec![],
open: false,
}
}

/// Set the link href.
pub fn href(mut self, href: &str) -> NavLink {
self.href = href.to_string();
self.href = href.to_owned();
self
}

Expand Down
9 changes: 8 additions & 1 deletionpgml-dashboard/src/utils/config.rs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -34,13 +34,20 @@ pub fn static_dir() -> String {
}
}

pub fncontent_dir() -> String {
pub fnblogs_dir() -> String {
match var("DASHBOARD_CONTENT_DIRECTORY") {
Ok(dir) => dir,
Err(_) => "content".to_string(),
}
}

pub fn docs_dir() -> String {
match var("DASHBOARD_DOCS_DIRECTORY") {
Ok(dir) => dir,
Err(_) => "../pgml-docs/".to_string(),
}
}

pub fn search_index_dir() -> String {
match var("SEARCH_INDEX_DIRECTORY") {
Ok(path) => path,
Expand Down
Loading

[8]ページ先頭

©2009-2025 Movatter.jp