Bug: hang during import with multiple tabs in importer dialog

[mtbcarroll]

I was running 5.0.0 latest with PRs 2230 and 2231 merged in and I started several imports in separate tabs in the import dialog. Some of them proceeded and completed but the remaining never got started, they all sat as "Pending" with the CPU idle. Stack traces attached.

[mtbcarroll]

some exceptions in Blitz log

[mtbcarroll]

Insight log is unremarkable,

2014-04-01 10:03:34,882 INFO  [       ome.formats.importer.ImportConfig] ( Thread-27) OMERO Version: 5.0.0-DEV-ice34 
2014-04-01 10:03:34,882 INFO  [       ome.formats.importer.ImportConfig] ( Thread-27) Bioformats version: 5.0.0-DEV-ice34 revision: 85f8d80 date: 1 April 2014 
2014-04-01 10:03:35,406 INFO  [      o.f.i.transfers.UploadFileTransfer] ( Thread-27) Transferring /home/mtbc/Win7/images/2chZT.lsm... 
2014-04-01 10:03:39,664 DEBUG [          o.o.shoola.util.NetworkChecker] ( Thread-27) Count: 1  1 
2014-04-01 10:03:39,664 DEBUG [          o.o.shoola.util.NetworkChecker] ( Thread-27) Network status: true (in 0 ms.) use cached value: true 
2014-04-01 10:03:42,476 DEBUG [          o.o.shoola.util.NetworkChecker] (1-thread-1) Count: 1  1 
2014-04-01 10:03:42,476 DEBUG [          o.o.shoola.util.NetworkChecker] (1-thread-1) Network status: true (in 1 ms.) use cached value: false 
2014-04-01 10:03:43,777 DEBUG [          o.o.shoola.util.NetworkChecker] (entQueue-0) Cached networkup: true 
2014-04-01 10:03:43,875 DEBUG [          o.o.shoola.util.NetworkChecker] ( Thread-29) Cached networkup: true 
2014-04-01 10:04:42,477 DEBUG [          o.o.shoola.util.NetworkChecker] (1-thread-1) Count: 1  1 
2014-04-01 10:04:42,477 DEBUG [          o.o.shoola.util.NetworkChecker] (1-thread-1) Network status: true (in 0 ms.) use cached value: false 
2014-04-01 10:04:51,301 DEBUG [          o.o.shoola.util.NetworkChecker] (entQueue-0) Count: 1  1 
2014-04-01 10:04:51,301 DEBUG [          o.o.shoola.util.NetworkChecker] (entQueue-0) Network status: true (in 0 ms.) use cached value: true 
2014-04-01 10:04:51,467 DEBUG [          o.o.shoola.util.NetworkChecker] ( Thread-30) Cached networkup: true 
2014-04-01 10:05:42,478 DEBUG [          o.o.shoola.util.NetworkChecker] (1-thread-1) Count: 1  1 
2014-04-01 10:05:42,479 DEBUG [          o.o.shoola.util.NetworkChecker] (1-thread-1) Network status: true (in 1 ms.) use cached value: false 
2014-04-01 10:06:42,480 DEBUG [          o.o.shoola.util.NetworkChecker] (1-thread-1) Count: 1  1 

[jamoore]

Moving this to 5.0.2. For all comments including "hanging" or "hung" please include a jstack of the client and if possible, the server. Can you also capture the relevant portion of the server log?

[jamoore]

(Sorry missed the attached stack traces. I don't see anything remarkable in any of them though, which is unfortunate)

[dlindner]

There is an issue with importing many files at once; maybe that was the problem here, too?
With bugfix for #12012 there is now a setting in importer.xml to limit the number of files, currently set to 2000. But maybe this number is still too high. I ran into problems with 750 images, with default max heap size of 128MB (wasn't a problem with 256MB though); it's just the huge amount of UI elements which are created. I didn't get a OutOfMemoryException? but the import just got slower and slower, finally the UI freezed.

[mtbcarroll]

I was probably importing less than 200 images, including well samples.

[jburel]

Probably time to rethink the strategy for the number of elements created in that case.

[jburel]

The number of UI elements is too high. Gus has generated a "lighter" import dialog.
Not doable for 5.1. Pushing to 5.1.1.

[jburel]

I have also fixed issues when importing OME-TIFF and dv with companion files.
Did you have some of those files in the queue?

[mtbcarroll]

I would guess probably not -- I don't have either in my usual testing set -- but I can't rule it out, it was a long time ago, so if the errors I saw are consistent with those issues then that may well be what the problem was.

[jburel]

This will require to review the importer so we have a "light version"