Dear Elisa,
the rotation editor only works on atoms. Does your custom model contain atoms? If yes, you might be able to connect to the model's structural signal and react to the ParticlePositionChanged event (see this tutorial for more information on signals: https://documentation.samson-connect.net/developers/latest/page_tutorial_center_of_mass.html). Then, you would deduce the change in the coordinate system by looking at the change of positions in the atoms. Note that this wouldn't work if there would be a non-rigid deformation applied by another editor.
If your custom model does not contain any atom, yes you will have to develop an editor to rotate it, and extending the current rotation editor will not help (for 0.8.0, we're looking to expose the underlying controller, for this type of situations).
Best,
Stephane