Codecov Report

❗ No coverage uploaded for pull request base (python-sdk@8441038).Click here to learn what that means.
The diff coverage isn/a.

❗ Current headc5d2b37 differs from pull request most recent head8cfc065. Consider uploading reports for the commit8cfc065 to get more accurate results

@@              Coverage Diff              @@##             python-sdk     #633   +/-   ##=============================================  Coverage              ?   80.02%           =============================================  Files                 ?      131             Lines                 ?    18950             Branches              ?     3996           =============================================  Hits                  ?    15164             Misses                ?     2749             Partials              ?     1037

📣 We’re building smart automated test selection to slash your CI/CD build times.Learn more

You can find the definition of theenum WasmEdge_Proposal in<wasmedge/enum_configure.h> after0.9.0-rc.1 pre-release. And we should not use the0.8.2 version, I think?

That's better. I'll update it. Thanks.

@apepkuss Please help review this PR as it develops. Thanks.

Currently, theVM::run function which does not determine the return type and parameter type on it's own is improper as it uses I32 for all the values. It should accept the following parameters based on the newly addedType in python.

• @param _FileName Name of .wasm binary
• @param _FuncName Wasm Function name
• @param _params Wasm Function Parameters
• @param _param_types List ofType of parameters
• @param _ret_types List ofType of Return values. This length must be same
• as return length of wasm function.

Hi@SAtacker, I noticed that PySDK uses Boost.Python as the bridge library to implement the C++ to Python conversion. How do you think about another useful library, PyBind11?

According to the description from PyBind11 (https://pybind11.readthedocs.io/en/stable/index.html), the main issue of Boost.Python:

Boost is an enormously large and complex suite of utility libraries that works with almost every C++ compiler in existence. This compatibility has its cost: arcane template tricks and workarounds are necessary to support the oldest and buggiest of compiler specimens. Now that C++11-compatible compilers are widely available, this heavy machinery has become an excessively large and unnecessary dependency.

In the context of WasmEdge project, PyBind11 could be better than Boost.Python for the following reasons:

• Shorter compilation time (https://pybind11.readthedocs.io/en/stable/benchmark.html#compilation-time).
• Smaller generated Python plugin (https://pybind11.readthedocs.io/en/stable/benchmark.html#module-size).
• PyBind11 is header only, while Boost.Python is part of Boost. From the view of development, we can use the header file of PyBind11 directly, do not need to care about build, installation and dependencies.
• WasmEdge itself is not based on Boost.

The above is my thought for the choice of the bridge library. Please correct me if my description was inappropriate.

Hi@SAtacker, I noticed that PySDK uses Boost.Python as the bridge library to implement the C++ to Python conversion. How do you think about another useful library, PyBind11?

According to the description from PyBind11 (https://pybind11.readthedocs.io/en/stable/index.html), the main issue of Boost.Python:

Boost is an enormously large and complex suite of utility libraries that works with almost every C++ compiler in existence. This compatibility has its cost: arcane template tricks and workarounds are necessary to support the oldest and buggiest of compiler specimens. Now that C++11-compatible compilers are widely available, this heavy machinery has become an excessively large and unnecessary dependency.

In the context of WasmEdge project, PyBind11 could be better than Boost.Python for the following reasons:

• Shorter compilation time (https://pybind11.readthedocs.io/en/stable/benchmark.html#compilation-time).
• Smaller generated Python plugin (https://pybind11.readthedocs.io/en/stable/benchmark.html#module-size).
• PyBind11 is header only, while Boost.Python is part of Boost. From the view of development, we can use the header file of PyBind11 directly, do not need to care about build, installation and dependencies.
• WasmEdge itself is not based on Boost.

The above is my thought for the choice of the bridge library. Please correct me if my description was inappropriate.

Thanks for bringing it up. I should have researched prior to implementation.
I started with the headers provided by CPython and wrapped it over WasmEdge. Then I realized that it took some extra effort to "glue" things together. This is when I jumped to Boost.Python.

I definitely agree with your proposal of moving to PyBind11 however I want to clarify that; Currently, I am re-implementing the C++ code using WasmEdge-C-API. Do you recommend it or any other method for example use the existing code which is used to implement the wasmedge CLI tool?

Thanks,@apepkuss I have moved it over to PyBind11.

@apepkuss 1. Can we have some use cases or examples of most of the C-API documentations?
2. For testing purpose, I have to test the functionalities similar toWasmEdge/test files, is it so?

Hi@SAtacker, sorry for the late reply, I missed the message notification.

1. Yes. Those examples in the C-API documents are to help the C-API users to understand how to use the C APIs. For the same purpose, hence, it would be better to cover the examples (or even create some new ones) using Python APIs. Of course, the Python APIs could have some different designs from the C APIs, therefore the examples for our Python APIs don't have to be exactly the same as those for C APIs.
2. PySDK is a Python binding to C APIs, and it should have the entire capabilities of the C APIs, so we should have all test cases for it as for C APIs. For the current phase, I think it would be better and reasonable for PySDK to have the unit test cases similar toWasmEdge/test/api/APIUnitTest.cpp.

Hi@SAtacker, sorry for the late reply, I missed the message notification.

1. Yes. Those examples in the C-API documents are to help the C-API users to understand how to use the C APIs. For the same purpose, hence, it would be better to cover the examples (or even create some new ones) using Python APIs. Of course, the Python APIs could have some different designs from the C APIs, therefore the examples for our Python APIs don't have to be exactly the same as those for C APIs.
2. PySDK is a Python binding to C APIs, and it should have the entire capabilities of the C APIs, so we should have all test cases for it as for C APIs. For the current phase, I think it would be better and reasonable for PySDK to have the unit test cases similar toWasmEdge/test/api/APIUnitTest.cpp.

Thanks for this, will let you know once it is done.

@apepkuss could you please look at#464 (comment)

@apepkuss could you please look at#464 (comment)

@SAtacker I'll check it, thanks!

@apepkuss I have tried a workaround for adding a python function as a host function. But when I list the function after adding it to VM Context, it gives a zero and consequently, the rest of the code does not work as expected.

@apepkuss I have tried a workaround for adding a python function as a host function. But when I list the function after adding it to VM Context, it gives a zero and consequently, the rest of the code does not work as expected.

Hi@SAtacker, could you please attach your python demo code here? so that we first get to know how you use the Python APIs, then we'll check the details under the hood. Thanks!

Cleaned.

I am going to create a new branch calledpython-sdk based on the latest master.

Could you also add a workflow to build and test the python SDK? If should fail at the current stage, however, we just need it to check the status.

Yes, I already have the testing in place, CI needs to be added. Not sure if it works with the current versions of WasmEdge.

I am going to merge this into the branchpython-sdk. So we can have a working branch for the python developers to contribute the WasmEdge python SDK.

Please also notice that:

1. The python SDK is in alpha development, and we don't recommend to use it in a production environment.
2. The current implementation is based on the WasmEdge 0.9.x APIs, which may change when there are newer versions released. Developers should upgrade the APIs to the latest version to continue the development.

Hi@SAtacker
Could you also create a new issue to describe the roadmap and the current status of the python SDK. IMO, the#396 and#464 are too old to track the current status. We should have an integrated version.
Once you create the issue, I will add a new project called python-sdk to collect the corresponding issues.

I am going to merge this into the branchpython-sdk. So we can have a working branch for the python developers to contribute the WasmEdge python SDK.

Thanks that makes sense.

Please also notice that:

1. The python SDK is in alpha development, and we don't recommend to use it in a production environment.
2. The current implementation is based on the WasmEdge 0.9.x APIs, which may change when there are newer versions released. Developers should upgrade the APIs to the latest version to continue the development.

Yes, I agree.

Hi@SAtacker Could you also create a new issue to describe the roadmap and the current status of the python SDK. IMO, the#396 and#464 are too old to track the current status. We should have an integrated version. Once you create the issue, I will add a new project called python-sdk to collect the corresponding issues.

Right on it.

Hi@SAtacker
I would like to know the plan for this Python SDK.
Is it still alive? Or do you want to convert it into a community issue? So people who are interested in contribution can help.

Hi@SAtacker
I would like to know the plan for this Python SDK.
Is it still alive? Or do you want to convert it into a community issue? So people who are interested in contribution can help.

It would be better if we convert it to community issue because I don't really have much time outside of my work and academics currently.
Thanks!

I think that this PR is based on a pretty old version of WasmEdge. Is it possible to clean it up with minimum components, which only contain limited features but can be compatible with the latest version? And then raise an issue with several sub-tasks for the community.

Or we can have a roadmap or plan for the Python SDK and make all changes inside this branch. And we can decide when to get it merged back into the master branch.

WDYT?

I think that this PR is based on a pretty old version of WasmEdge. Is it possible to clean it up with minimum components, which only contain limited features but can be compatible with the latest version? And then raise an issue with several sub-tasks for the community.

Yes, i think so. But before I spend more time on this I would like to ask the below question.

Or we can have a roadmap or plan for the Python SDK and make all changes inside this branch. And we can decide when to get it merged back into the master branch.

Do we really plan to keep this part of the SDK alive? Maintaining it would be a part time job for anybody (and I have gotten way busier than before and the chances of me continuing it are frankly very less for now). The way I implemented it is mostly sane but without any real world use case and insights from you guys, it's not possible to move further.

The question is this: Do we need the Python binding for WasmEdge?

1. One big issue is that most maintainers, including me, use something other than Python as their primary programming language to build side projects. It will take more work to retrieve the real-world use cases in such a situation. Someone outside of the WasmEdge repository could help. So far, this kind of somebody seems nonexistent or rare.
2. If the users are few and no one has time to maintain it, Java Binding is an excellent example of this case; it exists, but no one knows its actual status. Then, users try to use it and finally figure out it no longer works.

One possible way is merging this PR into thepython-sdk branch and creating an issue for the community to call for contributors for this. No more work is needed; keep the current status and look for someone who wants to take over this. Does this make sense for you?

The question is this: Do we need the Python binding for WasmEdge?

Yeah

1. One big issue is that most maintainers, including me, use something other than Python as their primary programming language to build side projects. It will take more work to retrieve the real-world use cases in such a situation. Someone outside of the WasmEdge repository could help. So far, this kind of somebody seems nonexistent or rare.

I agree.

2. If the users are few and no one has time to maintain it, Java Binding is an excellent example of this case; it exists, but no one knows its actual status. Then, users try to use it and finally figure out it no longer works.
   One possible way is merging this PR into thepython-sdk branch and creating an issue for the community to call for contributors for this. No more work is needed; keep the current status and look for someone who wants to take over this. Does this make sense for you?

Yes it does, thank you. We should do that.