Python bindings for my custom model

[DmitriyMarin]

Dear @Elisa

I prepared a tutorial on how to create Python bindings for a SAMSON Element.

For now, it does not describe how to create Python bindings for functions which return or receive SBQuantity or SBDType* - I will add a tutorial on it later.

[Elisa]

Dear @DmitriyMarin ,

Thank you for the information. I was able to create python bindings for my model, although I had to add:

py::class_<SBStructuralModel>(m, "SBStructuralModel");

Before exposing my structural model, otherwise it was throwing an error when importing my module in python, that SBStructuralModel was an unknown type (in Python). After this, it is working great!!

I found a small problem, and it is to modify a structural model that has already been created in Python. For example, I create a simple structural model with a chain and some nucleotides, then through my own exposed functions I create it in the data graph. If I then try to add to this structural model (for which I still have a variable in python), another chain, it doesn't seem to be added properly and my visual model does weird things. It is not critical for what I could do, but perhaps I am missing something? I am noob with pybind11, and I'm still going over the doc.

Thanks again for your help and sorry for the late reply!

[DmitriyMarin]

Dear @Elisa ,

I am glad that you were able to create your own Python bindings, I hope it will make your work easier. ;)

Concerning the SBStructuralModel, it is exposed as SBMStructuralModel and can be found in Python bindings as follows: sam.Modeling.StructuralModel.StructuralModel. It is exposed like that:

py::class_<SBMStructuralModel, std::unique_ptr<SBMStructuralModel, py::nodelete>, SBMModel>(m, "StructuralModel")

Unfortunately, I cannot check right now the origin of the error you have. Could you, please, try adding into the exposure of your custom structural model the following: std::unique_ptr<MyCustomStructuralModel, py::nodelete>, where MyCustomStructuralModel is the name of your structural model class:

py::class_<MyCustomStructuralModel, std::unique_ptr<MyCustomStructuralModel, py::nodelete>, SBMStructuralModel>(m, "MyCustomStructuralModel")

See pybind11 documentation "Non-public destructors" section for more information. This might resolve your second issue as well.

[Elisa]

I also get the error if I use SBMStructuralModel:

ImportError: generic_type: type "MyCustomModel" referenced unknown base type "SBMStructuralModel"

It doesn't seem a problem with the typedef, as adding before exposing my model py::class_<SBStructuralModel>(m, "SBStructuralModel"); gets it working regardless of the alias used.

Best regards,

Elisa

[DmitriyMarin]

Yes, there is no difference between SBStructuralModel and SBMStructuralModel since it's a typedef, I just wanted to note that it is already exposed by the Python Scripting SAMSON Element and therefore, you do not need to expose it in your SAMSON Element.

When you expose your own custom structural model, you just need to provide the base class (i.e., SBStructuralModel) and the class type (the class type also depends on the class type of the base class). For a custom structural model it should be as follows (please, see the PyBindTutorial sample):

py::class_<
	CustomStructuralModel,					/* the class */
	std::unique_ptr<CustomStructuralModel, py::nodelete>,	/* the class type */
	SBStructuralModel					/* the base class */
	>
	c(m,							/* pybind11::module */
	"CustomStructuralModel"					/* the class name in python*/
	);

Then in Python Scripting you could do something like this:

import SE_F2078F9E_F2CB_BA72_EE86_1E01A10B63D4 as tutorial	# import your module
structuralModel = tutorial.CustomStructuralModel()		# construct a custom structural model
structuralModel.create()					# create it
layer = SAMSON.getActiveLayer()					# get active layer
layer.addChild(structuralModel)					# add the custom structural model to the active layer
structuralRoot = structuralModel.getStructuralRoot()		# get the structural root of the custom structural model
group1 = tutorial.CustomStructuralGroup('group 1')		# construct a custom structural group or other type of data graph node
group1.create()							# create it
structuralRoot.addChild(group1)					# add it to the custom structural model
group2 = tutorial.CustomStructuralGroup('group 2')
group2.create()
structuralRoot.addChild(group2)

[Elisa]

That is not working for me, I get the error I posted before unless before that statement I add py::class_<SBStructuralModel>(m, "SBStructuralModel"); before exposing my structural model when importing in the Python Scripting console.

[DmitriyMarin]

Could you, please, try to build the PyBindTutorial sample and check if you experience the same behavior.
Do you get this error when importing your module in the Jupyter QtConsole by Python Scripting?

[Elisa]

Hi, I have built the PyBindTutorial and I got the same error:

Jupyter QtConsole 4.3.1
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

Python bindings for the SAMSON API are done in the same structure as the SAMSON SDK itself (see Documentation for SAMSON SDK). Python bindings for the SAMSON API are imported in the following way:

import samson as sam
from samson.Facade import SAMSON        # SAMSON Facade - main interface of SAMSON
from samson.DataModel import Quantity   # Quantities: length, mass, time, etc
from samson.DataModel import Type       # Types: position3, etc

import SE_F2078F9E_F2CB_BA72_EE86_1E01A10B63D4 as pybindtutorial
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-1dded50a96a7> in <module>()
----> 1 import SE_F2078F9E_F2CB_BA72_EE86_1E01A10B63D4 as pybindtutorial

ImportError: generic_type: type "CustomStructuralModel" referenced unknown base type "SBMStructuralModel"

The only change I made to the code was in the CMakeList.txt to point to my pybind11 folder, but I don't think that should be related to this.

[DmitriyMarin]

No, I do not think that it can be related to a pybind11 version. Did you experience this issue on Windows? I will try to reproduce the problem and will return to you. For now, you may try to use your workaround with py::class_<SBStructuralModel>(m, "SBStructuralModel");

[Elisa]

Yes, I'm using Windows 10.

By the way, this is not entirely related, but now my module fails to load if the Python Scripting module is not loaded, do you know of a simple way to handle this? So the rest of functionality of my module can be loaded.

Best regards,

Elisa

[DmitriyMarin]

Dear @Elisa ,

This is not totally related to the Python Scripting Element but to the fact that now your Element is linked to a Python library and users will need to install Python. The Python Scripting Element also takes care of loading the Python library.

There are several possibilities to solve this:
1 . You can ship 2 SAMSON Elements: with Python bindings and without. But it is not that convenient.
2 . Users will need to install Python.
3 . You can ship a Python library together with your SAMSON Element.

I will check the 3rd option and will return to you.

[DmitriyMarin]

@elisa said in Python bindings for my custom model:

That is not working for me, I get the error I posted before unless before that statement I add py::class_<SBStructuralModel>(m, "SBStructuralModel"); before exposing my structural model when importing in the Python Scripting console.

I have tried to reproduce the problem you are having on Win10 and Win7 with the PyBindTutorial sample, but I could not - for me, it works without the need to expose SBStructuralModel (i.e., py::class_<SBStructuralModel>(m, "SBStructuralModel");) before exposing a custom structural model - there is no error when importing the module in the Python Scripting console.

Could you, please, try to point not to your pybind11 folder but to one provided with the PyBindTutorial sample, like it is already done in the CMakeLists.txt of this sample - just try to build the PyBindTutorial sample as it is.

[DmitriyMarin]

@dmitriymarin said in Python bindings for my custom model:

Dear @Elisa ,
This is not totally related to the Python Scripting Element but to the fact that now your Element is linked to a Python library and users will need to install Python. The Python Scripting Element also takes care of loading the Python library.
There are several possibilities to solve this:
1 . You can ship 2 SAMSON Elements: with Python bindings and without. But it is not that convenient.
2 . Users will need to install Python.
3 . You can ship a Python library together with your SAMSON Element.
I will check the 3rd option and will return to you.

Concerning the 3rd option. A simple workaround might be to ship your SAMSON Element together with a Python library used for compilation for each platform. You can just copy the Python library in the SAMSON Binaries folder (the folder with SAMSON executable). Then even if a user has no Python installed, your SAMSON Element will be loaded since a dependency to the Python library is resolved.

[Elisa]

Thank you very much, I will investigate how to ship it with a virtual python environment with pip freeze and venv.

[DmitriyMarin]

@Elisa , if you came up with a solution, please, share it. In the future, we want to ship the Python Scripting Element with python interpreter and libraries, so that users won't need to install Python themselves.

[Elisa]

I will! I am dealing with a weird unrelated bug in my programming environment, but will try to get on with this after that :)

[DmitriyMarin]

Thanks, @Elisa !

[DmitriyMarin]

I have added a tutorial on how to create Python bindings for functions that return or receive SAMSON physical quantities and types (i.e., SBQuantity, SBVector3, etc). The code for the tutorial can be found in the sample PyBindTutorial SAMSON Element.

[Elisa]

Thanks for the new tutorial!!

I was looking into shipping my module with a Python virtual environment, to avoid users having to install Anaconda3. So far this is what I got:

• If I understood it alright, to develop and compile the module only the variables PYTHON_INCLUDE_DIR and PYTHON_LIBRARY matter, which should be Python 3.6's. I didn't check if they updated when upgrading the Anaconda installation, but it seems easy to keep an Anaconda 5.2 installation to access this in case it wouldn't work.

• To execute the Python Scripting (and any other depending on it) it is necessary to specify the variable PYTHON_EXECUTABLE. For this I think you can use a virtual environment. What I did was to use pip freeze
  to get a list of modules installed in Anaconda 5.2, then I used the virtualenv module to create a virtual environment out of Anaconda 5.2 and install this modules there. I set the PYTHON_EXECUTABLE variable to this virtual environment ENV_DIR\\Scripts\\python.exe (which you can place on the data folder and then set this using the plugin installation folder, I guess). The only problem is, you have to activate the virtual environment. I tested this by going to the virtual environment folder, activating it from there in the command line, and calling SAMSON-Core.exe then. It worked. I didn't had Anaconda in my path, nor it worked without activating the virtual environment.

I have to update now my Anaconda3 installation for some unrelated work, so I will test if this keeps working in the developers case. Hopefully, I will be able to test if at least users can avoid installing Anaconda3 5.2 soon with a user of our module.

This were the modules I installed in my virtual environment btw:

backcall==0.1.0
colorama==0.3.9
decorator==4.3.0
ipykernel==4.8.2
ipython==6.4.0
ipython-genutils==0.2.0
jedi==0.12.0
jupyter-client==5.2.3
jupyter-console==5.2.0
jupyter-core==4.4.0
numpy==1.14.3
parso==0.2.0
pickleshare==0.7.4
prompt-toolkit==1.0.15
Pygments==2.2.0
python-dateutil==2.7.3
pyzmq==17.0.0
simplegeneric==0.8.1
six==1.11.0
tornado==5.0.2
traitlets==4.3.2
wcwidth==0.1.7

Maybe you could do without numpy or with a more recent version. The virtual environment command was:

python -m virtualenv VENV_DIR --always-copy

I haven't understood yet how these specific versions of the modules are involved when compiling, but in principle the Include folder from my virtual environment, and the include folder from anaconda3 seem the same. Maybe for the Python Scripting module this is different, since it is the module actually creating the ipython console.

[DmitriyMarin]

Thank you, @Elisa ! This looks great!

Do you want users to use Python in your module or is it just for your module to be loaded in SAMSON if users do not have Python installed?

What is the size of the resulting environment?

I will try the same for the Python Scripting Element, since for the next release we are thinking about shipping it with a Python environment.