Script utils in scripts repo?

[wmoore]

How to handle scripts that require additional functionality, E.g. in script_utils? when adding these scripts to the new scripts gh repo?

E.g.
The Kymograph script was originally a stand-alone script, but since it was in a branch with another script "Plot_Profile.py" that shared a function - we moved this common code into script_utils. See #8206.

​https://github.com/will-moore/openmicroscopy/commits/feature/kymograph_script

We don't really want to force users to checkout a corresponding branch of the main server, just to be able to run these scripts. Kinda defeats the object of a separate scripts repo. So, is there somewhere else under OmeroPy/scripts that we can put script_utils?

[jmoore]

Other scripts under lib/scripts are still import-able. I.e. if you adding omero/util_scripts.py then other scripts would be able to do "from omero.utils_scripts import X". We'd have to look at how that affects the UI. Will it show up as a user script? That'd be pretty annoying.

[wmoore]

I'm not getting this: how to import scripts under lib/scripts. When I do "import omero" I'm not importing dist/lib/scripts/omero ?

dist/lib/scripts is not on my $PYTHONPATH. This is dist/lib/python/

I have dist/lib/scripts/omero/analysis_utils.py but when I do:

 id   | Official scripts                            
------+---------------------------------------------
 1304 | /omero/analysis_utils.py                    
 1251 | /omero/export_scripts/Batch_Image_Export.py 
 1252 | /omero/export_scripts/Make_Movie.py         
 1253 | /omero/figure_scripts/Movie_Figure.py       
 1254 | /omero/figure_scripts/Movie_ROI_Figure.py   
 1255 | /omero/figure_scripts/ROI_Split_Figure.py   
 1256 | /omero/figure_scripts/Split_View_Figure.py  
 1257 | /omero/figure_scripts/Thumbnail_Figure.py   
 1258 | /omero/import_scripts/Populate_ROI.py       
 1259 | /omero/setup_scripts/FLIM_initialise.py     
 1260 | /omero/util_scripts/Channel_Offsets.py      
 1261 | /omero/util_scripts/Combine_Images.py       
 1262 | /omero/util_scripts/Dataset_To_Plate.py     
 1263 | /omero/util_scripts/Images_From_ROIs.py     
(14 rows)
jrs-macbookpro-25107:scripts will$ omero script upload omero/analysis_scripts/Kymograph.py --official
Using session feb334ba-bf43-4fdc-9c5b-df48214bd900 (root@localhost:4064). Idle timeout: 10.0 min. Current group: system
ApiUsageException: Can't find params for 1305. Stderr is in file 1306

---stderr---
Traceback (most recent call last):
  File "./script", line 40, in <module>
    from omero.analysis_utils import getLineData
ImportError: No module named analysis_utils

[jmoore]

Hmmm.....maybe I'm wrong. I thought we had decided to put lib/scripts on PYTHONPATH, but perhaps that was only a discussion. Sorry. In that case, there's not currently a good (non-code-duplicating) solution. The only thing I can think of off-hand is to download another original file from lib/scripts and exec it, but that seems quite nasty.

[wmoore]

OK, I'm happy to have lib/scripts on my PYTHONPATH.
Now, if I put my analysis_utils in lib/scripts then I can do

>>> from analysis_utils import getLineData

but if I put it in lib/scripts/omero (including a init.py in scripts/omero) I can't import it because "omero" is not this module

>>> import omero
>>> dir(omero)
['ClientError', 'UnloadedCollectionException', 'UnloadedEntityException', '__builtins__', '__doc__', '__file__', '__import_style__', '__name__', '__package__', '__path__', '_sys', 'client', 'client_wrapper', 'exceptions', 'warnings']
>>> from omero.analysis_scripts import getLineData
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named analysis_scripts

[jmoore]

Makes sense. This may have been on of the issues. You could try something like omero_utils/__init__.py but then we're creating just another utils location. Any other ideas of what we could try?

[wmoore]

I'm OK with another "utils location" within the scripts repo, since we kinda need that so that the script and it's dependancies can be in the same branch.
But will scritps/omero_utils/ get built into dist/lib/scripts/omero_utils/ ?
ie - if we put a scripts/omero_utils into the master branch and it becomes part of the main OMERO build, will this work OK?

[jmoore]

At the build level, there's no filtering of the stuff under OmeroPy/scripts. So whatever is in that submodule pointer, will go into lib/scripts in the server. Are you thinking of wanting to change this for existing servers (4.4.0) or just something we could update in 4.4.1?

[wmoore]

Committed to scripts submodule using scripts/analysis_utils for common code. See ​https://github.com/ome/scripts/pull/1