Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Setup - PostgresML#541

Feb 11, 2023· 4 comments· 4 replies
Discussion options

Setup - PostgresML

Train and deploy models to make online predictions using only SQL, with an open source Postgres extension.

https://postgresml.org/user_guides/transformers/setup/

You must be logged in to vote

Replies: 4 comments 4 replies

Comment options

Two things.

First, running the installation commands as shown withsudo causes this warning:

WARNING: Running pip as the'root' user can resultin broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Is it necessary to usesudo? Installation instructions that cause the system to emit warnings will worry people kicking the tires and will scare some of them away.

Second, and more important, after following the instructions to the letter, I get this error when trying to use transformers.

psql (14.6 (Ubuntu14.6-0ubuntu0.22.04.1))Type"help" for help.postgres=# SELECT pgml.transform('translation_en_to_fr',    inputs=> ARRAY['Welcome to the future!','Where have you been all this time?'    ])AS french;ERROR:  called`Result::unwrap()`on an`Err` value: PyErr { type:<class'ModuleNotFoundError'>, value: ModuleNotFoundError("No module named 'transformers'"), traceback: Some(<traceback object at 0x7fbaea7dc700>) }CONTEXT:  src/bindings/transformers.rs:19:62

This is for postgresql-14 and pgml-14 on Ubuntu 22.

You must be logged in to vote
0 replies
Comment options

  1. Unfortunately, python dependency management is fraught with peril and installing in the system Python is required for Postgres to be able to find the modules for most generally distributed versions.

image

There may be ways we could get around this by using individual package managers (e.g. apt on debian/ubuntu) to install Python dependencies system wide, but we'd need documentation for every distribution and OS. I think a better path forward would be to sandbox the Postgres plPython dependencies (or at least just our own) using a virtualenv forpy03.

  1. I think you need to also install the transformers pip package (and probably torch and/or tensorflow). I'll see what docs need to be updated.
You must be logged in to vote
0 replies
Comment options

Hello, would you please provide guidance on how to install postgresML extension on CentOS ? Many thanks !

You must be logged in to vote
0 replies
Comment options

Hi@ahmedrachid, we don't maintain a CentOS package, although you can convert thedeb package we distribute withalien. Otherwise, you'll need to build from source.

You must be logged in to vote
4 replies
@ahmedrachid
Comment options

Oh i see.. I will give it a try then, many thanks for your answer. I'll be testing it in a distributed MPP PostgreSQL, do you think that model training can be distributed ?

@montanalow
Comment options

PostgresML training happens in a single connection process, so it will not work out of the box to distribute training throughout a Postgres-XL cluster. Parallelization happens within the training algorithm, although training can be offloaded to a replica.

@ahmedrachid
Comment options

That's great to know, many thanks for your help :)

@ahmedrachid
Comment options

Hello@montanalow , can you please share with me the "deb" package link ? I'm unable to find it and would like to use Alien to convert it to RPM. Many thanks

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
None yet
3 participants
@montanalow@ahmedrachid@dventimihasura

[8]ページ先頭

©2009-2025 Movatter.jp