Python bindings for my custom model

DmitriyMarin

@dmitriymarin said in Python bindings for my custom model:

Dear @Elisa ,
This is not totally related to the Python Scripting Element but to the fact that now your Element is linked to a Python library and users will need to install Python. The Python Scripting Element also takes care of loading the Python library.
There are several possibilities to solve this:
1 . You can ship 2 SAMSON Elements: with Python bindings and without. But it is not that convenient.
2 . Users will need to install Python.
3 . You can ship a Python library together with your SAMSON Element.
I will check the 3rd option and will return to you.

Concerning the 3rd option. A simple workaround might be to ship your SAMSON Element together with a Python library used for compilation for each platform. You can just copy the Python library in the SAMSON Binaries folder (the folder with SAMSON executable). Then even if a user has no Python installed, your SAMSON Element will be loaded since a dependency to the Python library is resolved.

Elisa

Thank you very much, I will investigate how to ship it with a virtual python environment with pip freeze and venv.

DmitriyMarin

@Elisa , if you came up with a solution, please, share it. In the future, we want to ship the Python Scripting Element with python interpreter and libraries, so that users won't need to install Python themselves.

Elisa

I will! I am dealing with a weird unrelated bug in my programming environment, but will try to get on with this after that :)

DmitriyMarin

Thanks, @Elisa !

DmitriyMarin

I have added a tutorial on how to create Python bindings for functions that return or receive SAMSON physical quantities and types (i.e., SBQuantity, SBVector3, etc). The code for the tutorial can be found in the sample PyBindTutorial SAMSON Element.

Elisa

Thanks for the new tutorial!!

I was looking into shipping my module with a Python virtual environment, to avoid users having to install Anaconda3. So far this is what I got:

If I understood it alright, to develop and compile the module only the variables PYTHON_INCLUDE_DIR and PYTHON_LIBRARY matter, which should be Python 3.6's. I didn't check if they updated when upgrading the Anaconda installation, but it seems easy to keep an Anaconda 5.2 installation to access this in case it wouldn't work.
To execute the Python Scripting (and any other depending on it) it is necessary to specify the variable PYTHON_EXECUTABLE. For this I think you can use a virtual environment. What I did was to use pip freeze
to get a list of modules installed in Anaconda 5.2, then I used the virtualenv module to create a virtual environment out of Anaconda 5.2 and install this modules there. I set the PYTHON_EXECUTABLE variable to this virtual environment ENV_DIR\\Scripts\\python.exe (which you can place on the data folder and then set this using the plugin installation folder, I guess). The only problem is, you have to activate the virtual environment. I tested this by going to the virtual environment folder, activating it from there in the command line, and calling SAMSON-Core.exe then. It worked. I didn't had Anaconda in my path, nor it worked without activating the virtual environment.

I have to update now my Anaconda3 installation for some unrelated work, so I will test if this keeps working in the developers case. Hopefully, I will be able to test if at least users can avoid installing Anaconda3 5.2 soon with a user of our module.

This were the modules I installed in my virtual environment btw:

backcall==0.1.0
colorama==0.3.9
decorator==4.3.0
ipykernel==4.8.2
ipython==6.4.0
ipython-genutils==0.2.0
jedi==0.12.0
jupyter-client==5.2.3
jupyter-console==5.2.0
jupyter-core==4.4.0
numpy==1.14.3
parso==0.2.0
pickleshare==0.7.4
prompt-toolkit==1.0.15
Pygments==2.2.0
python-dateutil==2.7.3
pyzmq==17.0.0
simplegeneric==0.8.1
six==1.11.0
tornado==5.0.2
traitlets==4.3.2
wcwidth==0.1.7

Maybe you could do without numpy or with a more recent version. The virtual environment command was:

python -m virtualenv VENV_DIR --always-copy

I haven't understood yet how these specific versions of the modules are involved when compiling, but in principle the Include folder from my virtual environment, and the include folder from anaconda3 seem the same. Maybe for the Python Scripting module this is different, since it is the module actually creating the ipython console.

DmitriyMarin

Thank you, @Elisa ! This looks great!

Do you want users to use Python in your module or is it just for your module to be loaded in SAMSON if users do not have Python installed?

What is the size of the resulting environment?

I will try the same for the Python Scripting Element, since for the next release we are thinking about shipping it with a Python environment.

Elisa

Thanks!

It would be only for the case that users don't have python or they need very specific Python requirements (exactly those python modules versions, for example).
I think if you ship the Python Scripting Element with a virtual environment, probably other modules don't need to ship it.

The virtual environment is 101MB, but I created it copying everything, maybe it could work if the user has a Python36 installation somewhere, and we just override the modules with the versions we need.

DmitriyMarin

Yes, if Python Scripting is shipped with a Python environment, other SAMSON Elements would not need their own, except if they use some quite specific one.

Can the Python environment you are shipping with your module be extended by installing new modules?

Overriding of existing modules in the user's environment is not suitable since users might need this particular environment, we should not modify a user's environment in any way.

Elisa

The virtual environments doesn't really override existing modules, only once you have activated this environment, but once you exit it, the usual environment is back. It works like Anaconda environments. Once you activate the environment you can install any module you want, remove it or upgrade it, via pip as usual.

DmitriyMarin

Ah, sorry, I see what you meant in the previous comment, I've misunderstood.
I will check later the construction of a Python environment for Python Scripting so that users did not have to install Python. Activation of Python environment on Linux and Mac can be incorporated into the scripts that launch SAMSON, on Windows you can write a .bat file that will be first activating your Python env and then launching SAMSON.

Elisa

Even though I develop this for Windows, I am more familiar with Linux too, so I don't know if it would be possible to incorporate on the launcher the activation of the venv, but it is done with a script created by virtualenv/venv as in Linux.

DmitriyMarin

I think it should be possible, it will be just an additional line in the launcher script. If we will be shipping Python Scripting with the Python environment, I think we might ship the environment with a bare SAMSON installation so that to use SAMSON Elements, that depend on this environment, the user won't need to install the Python Scripting Element.

DmitriyMarin

To update the topic and for information:
If you are looking for instructions on how to create Python bindings for your SAMSON Extension, please refer to the Developer Guide: Creation of Python Bindings for SAMSON Extensions.