Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Add a way to inject modules to rustpython_wasm#2402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
coolreader18 merged 2 commits intomasterfromwasm-hooks
Feb 3, 2021

Conversation

@coolreader18
Copy link
Member

cc@satyajeetkolhapure

To add your own modules, you should make your own crate that depends on

# for wasm-bindgen[lib]crate-type = ["cdylib","rlib"]# can remove `branch="..."` once  this PR is mergedrustpython_wasm = {git ="https://github.com/RustPython/RustPython",branch ="wasm-hooks" }rustpython_vm = {git ="https://github.com/RustPython/RustPython",branch ="wasm-hooks" }wasm-bindgen ="0.2"# might also want this so that compilation doesn't take forever[package.metadata.wasm-pack.profile.release]wasm-opt =false

Then your lib.rs should look like:

pubuse rustpython_wasm::exports::*;use wasm_bindgen::prelude::*;use rustpython_vm::VirtualMachine;#[wasm_bindgen(start)]fninit(){    rustpython_wasm::add_init_func(init_vm);}fninit_vm(vm:&mutVirtualMachine){    vm.add_frozen(rustpython_vm::py_freeze!(dir ="deps"));}

deps/ should be a directory next to your Cargo.toml, with a subdirectorydeps/pytezos with stuff likedeps/pytezos/__init__.py.

@satyajeetkolhapure
Copy link

satyajeetkolhapure commentedJan 20, 2021
edited
Loading

Thanks@coolreader18 for the quick fix.

I created new crate and cargo.toml looks like -

[package]name = "PytezosRustPython"version = "0.1.0"authors = ["satyajeetkolhapure"]edition = "2018"[lib]crate-type = ["cdylib", "rlib"][dependencies]rustpython_wasm = { git = "https://github.com/RustPython/RustPython", branch = "wasm-hooks" }rustpython-vm = { git = "https://github.com/RustPython/RustPython", branch = "wasm-hooks" }wasm-bindgen = "0.2"[package.metadata.wasm-pack.profile.release]wasm-opt = false

And lib.rs looks like

pub use rustpython_wasm::exports::*;use wasm_bindgen::prelude::*;use rustpython-vm::VirtualMachine;#[wasm_bindgen(start)]fn init() {    rustpython_wasm::add_init_func(init_vm);}fn init_vm(vm: &mut VirtualMachine) {    vm.add_frozen(rustpython-vm::py_freeze!(dir = "deps"));}

I have added simple mymodule.py file in deps folder which has one simple function

When I try to runwasm-pack build --target nodejs I get below error

Compiling rustpython-pylib v0.1.0 (https://github.com/RustPython/RustPython?branch=wasm-hooks#304a47af)error[E0599]: no method named `into_pyexception` found for struct `std::io::Error` in the current scope   --> /Users/satyajeetkolhapure/.cargo/git/checkouts/rustpython-f8ef4d934ac33cd8/304a47a/vm/src/stdlib/thread.rs:246:28    |246 |         .map_err(|err| err.into_pyexception(vm))    |                            ^^^^^^^^^^^^^^^^ method not found in `std::io::Error`    |    = help: items from traits can only be used if the trait is implemented and in scopenote: `IntoPyException` defines an item `into_pyexception`, perhaps you need to implement it   --> /Users/satyajeetkolhapure/.cargo/git/checkouts/rustpython-f8ef4d934ac33cd8/304a47a/vm/src/exceptions.rs:42:1    |42  | pub trait IntoPyException {    | ^^^^^^^^^^^^^^^^^^^^^^^^^error: aborting due to previous errorFor more information about this error, try `rustc --explain E0599`.error: could not compile `rustpython-vm`

Any idea, if I am missing something?

@coolreader18
Copy link
MemberAuthor

Oh whoops, yeah, adddefault-features = false to the rustpython-vm dependency

@satyajeetkolhapure
Copy link

Sorry@coolreader18 for too many questions. This time cargo build succeeds, but when I ranwasm-pack build --target nodejs I got below error

[INFO]: ⬇️  Installing wasm-bindgen...error: cannot specify two `start` functionsError: Running the wasm-bindgen CLICaused by: failed to execute `wasm-bindgen`: exited with exit code: 1

May be becauserustpython_wasm package already has start method. Not sure how to fix this.

@coolreader18
Copy link
MemberAuthor

Oh, hmmmm. I didn't think that the one in rustpython_wasm would count since it isn't really exported.

@coolreader18
Copy link
MemberAuthor

@satyajeetkolhapure try addingfeatures = ["no-start-func"] to your rustpython_wasm dependency

@satyajeetkolhapure
Copy link

Hi@coolreader18

I managed to run new package with new changes. I am not sure how to use it though.

I created mymodule.py inside deps folder in my new crate. Created pkg using wasm-pack command.

mymodule.py looks like below

def getPrintString():    return 'Hello world'

Then I created main.py like below

from mymodule import getPrintStringprint(getPrintString())

I then created runwasm.js like below

const rp = require('/Users/satyajeetkolhapure/TruffleProject/TezosRustPython/PytezosRustPython/pkg_old/PytezosRustPython.js')const fs = require('fs')const content = fs.readFileSync('main.py').toString()rp.pyEval(content)

But I am getting below error

Error parsing Python code: Got unexpected token 'print' at line 2 column 1print(getPrintString())

Any idea?

@coolreader18
Copy link
MemberAuthor

You should callpyExec, notpyEval,pyEval expects one expression. Also, you have to havemymodule.py in the deps/ folder that gets frozen, wasm doesn't have access to the file system.

@satyajeetkolhapure
Copy link

@coolreader18 : You are awesome. It worked. Thank you so much for all your help and solving this problem. I will keep you posted when I try it with pytezos library.

Copy link

@satyajeetkolhapuresatyajeetkolhapure left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I tested it. It works.

coolreader18 reacted with rocket emoji
@satyajeetkolhapure
Copy link

Hi@coolreader18

While testing it for pytezos, I am getting error because of other dependencies which pytezos needs. Is it possible to copy whole venv as well with this? When I copy the whole venv folder inside deps folder, somehow it does not find packages.

e.g I have venv folder with all site-packages and one python file which uses those dependencies. How can I achieve it? Is there any way to add venv?

@coolreader18
Copy link
MemberAuthor

@satyajeetkolhapure you should be able to point thepy_freeze!(dir = "...") to your venv packages folder, though keep in mind that the more you include the larger the wasm binary will be

@satyajeetkolhapure
Copy link

satyajeetkolhapure commentedJan 25, 2021
edited
Loading

@coolreader18 Somehow when I add packages folder with py_freeze, it is not able to detect it.

I did below steps -

  1. Created new python project with venv python 3.7 version
  2. Installed pytezos package
  3. Copied all the folders and files from vevn/lib/python3.7/site-packages folder to deps folder inside my new crate rust project
  4. Built pkg using wasm command
  5. Tried to import pytezos, it failed with module not found.
  6. Probably site-packages are treated differently. If I manage to add pytezos as std-library in vm as others like html, email etc. I might be able to use it.

Is it possible to add it as standard library. I mean if I copy all pytezos libraries inside pylib-crate/Lib folder on my local machine vm repository, will those be added as standard libraries? or do I need something more than just copying?

@coolreader18
Copy link
MemberAuthor

@satyajeetkolhapure that's really weird, you can definitely try copying everything in deps/ torustpython/Lib. Could you post what the contents of your deps/ folder looks like? Like justls deps or something?

@satyajeetkolhapure
Copy link

@coolreader18 : Below are the folders and files inside deps folder

pycache
_distutils_hack
base58-1.0.3.dist-info
base58.py
certifi
certifi-2020.12.5.dist-info
chardet
chardet-4.0.0.dist-info
dateutil
distutils-precedence.pth
fire
fire-0.4.0-py3.7.egg-info
idna
idna-2.10.dist-info
loguru
loguru-0.5.3.dist-info
mnemonic
mnemonic-0.19.dist-info
netstruct-1.1.2-py3.7.egg-info
netstruct.py
pendulum
pendulum-2.1.2.dist-info
pip
pip-21.0.dist-info
pkg_resources
ply
ply-3.11.dist-info
pyblake2-1.1.2-py3.7.egg-info
pyblake2.cpython-37m-darwin.so
pytezos
pytezos-2.2.8.dist-info
python_dateutil-2.8.1.dist-info
pytzdata
pytzdata-2020.1.dist-info
requests
requests-2.25.1.dist-info
setuptools
setuptools-52.0.0.dist-info
simplejson
simplejson-3.17.2.dist-info
six-1.15.0.dist-info
six.py
termcolor-1.1.0-py3.7.egg-info
termcolor.py
testmain.py
tqdm
tqdm-4.56.0.dist-info
urllib3
urllib3-1.26.2.dist-info

I will give it a try copying all these inside Lib folder of rustpython

@satyajeetkolhapure
Copy link

Hi@coolreader18 : I am using python3.7 to create those packages. Probably, this is causing it, may be because of obsolete import rather than relative.

Which version of python interpreter RustPython uses?

@coolreader18
Copy link
MemberAuthor

We don't really have a specific version we target, we're a mishmash of anywhere from 3.5 to 3.9. Have you changed your lib.rs at all? It's stillvm.add_frozen(py_freeze!(dir = "deps")) ?

@satyajeetkolhapure
Copy link

yes. it is still the samevm.add_frozen(py_freeze!(dir = "deps"))

I tried even copying all those files in Libs folder in vm. Somehow it found the main module. But it failed to find submodule. the paths used are obsolute. Not sure if that is the reason.

@coolreader18
Copy link
MemberAuthor

Which paths? (assuming you mean absolute). Like the imports areimport pytezos.bar.baz instead ofimport ..bar.baz?

@coolreader18
Copy link
MemberAuthor

There might be some unresolved issues with recursive relative imports, I know there are a few instances in the standard library we've had to work around.

@satyajeetkolhapure
Copy link

correct import pytezos.rpc is throwing errorNo module named 'pytezos.rpc'

could you help me with the workaround? if possible. if you point me to any example of workaround

@coolreader18
Copy link
MemberAuthor

Were you ever able to get it working in the first place? Trying to import pytezos just using the native rustpython binary seems to error in a bunch of different ways.

@satyajeetkolhapure
Copy link

When I create new python project and installed pytezos package, I called pytezos as below

import pytezos from pytezosprint(pytezos)

Then I ran it in pycharm. I also tried commandpython3 filename.py

It gave me output

<pytezos.client.PyTezosClient object at 0x103cf59d0>Properties.key  # tz1grSQDByRpnVs7sPtaprNZRp531ZKz6Jmm.shell  # https://rpc.tzkt.io/delphinet/ (delphinet)Helpers.account().activate_account().balance().ballot().contract().delegation().double_baking_evidence().double_endorsement_evidence().endorsement().nft_app().now().operation().operation_group().origination().proposals().reveal().seed_nonce_revelation().transaction().using()Process finished with exit code 0

I had to install below two on mac os

$ brew tap cuber/homebrew-libsecp256k1$ brew install libsodium libsecp256k1 gmp

pytezos documentation link -https://pypi.org/project/pytezos/

Appreciate your help@coolreader18

@coolreader18
Copy link
MemberAuthor

Right, but were you ever able to get it working in rustpython? RustPython isn't fully compatible with CPython yet, and looking at whatpytezos depends on, it looks like there are some native modules, which RustPython probably won't support any time soon.

@satyajeetkolhapure
Copy link

I was not able to run it using rustpython yet.

Possible. I think, I need to wait for those native modules be part of rustpython then.

Will it be a long task to add those native modules? Just to get an idea

@coolreader18
Copy link
MemberAuthor

Well it wouldn't really be adding them, it would be adding support for native extensions. Alternatively, you could translate every native module that pytezos depends on to rust and include it in your wasm binary, but I don't even know if that's possible - it looks like there's lots of crypto stuff as well as http requests, which can't be made from wasm

@coolreader18
Copy link
MemberAuthor

Sorry for "leading you on" for so long, I hadn't ever looked up what pytezos is until today, I don't think it's possible to run in rustpython at the moment

@satyajeetkolhapure
Copy link

@coolreader18 no problem mate

@coolreader18coolreader18 merged commitf49448a intomasterFeb 3, 2021
@youknowoneyouknowone deleted the wasm-hooks branchFebruary 4, 2021 06:14
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@youknowoneyouknowoneyouknowone approved these changes

+1 more reviewer

@satyajeetkolhapuresatyajeetkolhapuresatyajeetkolhapure approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@coolreader18@satyajeetkolhapure@youknowone

[8]ページ先頭

©2009-2025 Movatter.jp