User Story #10178 (closed)
Opened 11 years ago
Closed 11 years ago
FS/Web stateful service interaction
Reported by: | jamoore | Owned by: | wmoore |
---|---|---|---|
Priority: | blocker | Milestone: | 5.0.0-beta1 |
Component: | Web | Keywords: | FS |
Cc: | web-team@…, fs@… | Story Points: | n.a. |
Sprint: | n.a. | Importance: | n.a. |
Total Remaining Time: | n.a. | Estimated Remaining Time: | n.a. |
Description (last modified by wmoore)
In 4.4, web methods were all converted to use a
svc = openStatefulService() try: svc.doWork(); finally: svc.close()
idiom. In other words, no stateful service lives longer than a single http requestl. With Bio-Formats now backing many of the services (renderingengine, rawpixelsservice, etc), this means that each method call (getPlane(), renderJpeg()) is performing a full setIdinvocation. For some formats, this can take several seconds.
Melissa is working on reducing setId times, and eventually the Bio-Formats state can be cached server-side in order to further speed up the individual lookups, but there may still be the need for properly handling state from the web client.
A first step is likely to evaluate the user impact of the slow down (especially when several users are accessing the web simultaneously). #10183
If the Blitz Gateway is to re-use existing stateful services (E.g. rendering engine already initialised on a particular imageId) then we have to support several steps
- Once we have joined a session...
- Identify stateful services that exist on the server
- Determine whether any of these are initialised with the correct ImageId?
- Recreate the client-side proxy
- Update the state (E.g. turn channels on/off etc) and get output (Render plane)
- Leave the service open.
- The server would need to purge or limit the number of these open stateful services.
NB: If multiple web workers are simultaneously following the above steps, I imagine it's possible that they could both try to work with the same stateful service, causing all sorts of problems!
Change History (10)
comment:1 Changed 11 years ago by wmoore
comment:2 Changed 11 years ago by jmoore
Will: a test like that we could have on both branches might be ideal. Something like ApacheBench
comment:3 Changed 11 years ago by wmoore
- Sprint set to 2013-02-12 (5)
comment:4 Changed 11 years ago by wmoore
- Sprint 2013-02-12 (5) deleted
comment:5 Changed 11 years ago by agilo
- Status changed from new to accepted
Updated status, related task in progress
comment:6 follow-up: ↓ 7 Changed 11 years ago by wmoore
- Description modified (diff)
Josh / Chris / Carlos - any idea how tricky are the steps I've outlined above? How bad is the risk of 2 workers accessing the same stateful service etc?
comment:7 in reply to: ↑ 6 Changed 11 years ago by jmoore
The server can definitely limit the number of stateful sessions per user/group combination. And then throw a special exception saying, "clean up your mess" ?
As for having multiple workers work on the save SF-service, to some extent your protected since the invocations are synchronized (can only happen in serial). So 2 simultaneous actions shouldn't cause a problem. However, if there are two conceptually different actions, (e.g. setPixelsId(4) and setPixelsId(6)) then things will get wonky. It might suffice to lay down rules about what is permissible under these circumstances. Perhaps even passing a {"shared":"true"} with the context map to tell the server to prevent certain actions. Alternatively, we provide a way to "lock" a service in the webclient so only one worker can get it at a time.
comment:8 Changed 11 years ago by jburel
- Keywords FS added
- Milestone changed from OMERO-4.5 to OMERO-5
comment:9 Changed 11 years ago by wmoore
If we have multiple workers simultaneously accessing a pool of active Rendering Engines, creating client-side proxies, setting rendering settings and then rendering, we have the potential for 2 client-side processes to be accessing the same server-side Rendering Engine?
E.g. These methods are called in sequence when we render a 3 Channel image.
setActive setChannelWindow setRGBA setActive setChannelWindow setRGBA setActive setChannelWindow setRGBA setModel setCompressionLevel renderCompressed
Multiple workers generating a Z-stack or T-series (all same channels on/off) would be unlikely to cause problems, but if different workers wanted to render with different channels / rendering settings, they could compete with each other and get out of sync (maybe)?
comment:10 Changed 11 years ago by wmoore
- Resolution set to wontfix
- Status changed from accepted to closed
Not going to try and re-use stateful services in web, without more server-side support. See https://github.com/openmicroscopy/openmicroscopy/pull/718
Josh: would it help to have a performance testing web page, which tries to render multiple planes from the same (or different) images and logs the time taken? This could be part of webtest and included in dev_4_4 and develop, then we can compare times etc.
E.g. sequentially render all the planes in a specified image / images.