Task #9372 (closed)
Bug: big image fails with 'Negative size'
Reported by: | jamoore | Owned by: | rleigh |
---|---|---|---|
Priority: | blocker | Milestone: | OMERO-4.4.1 |
Component: | Bio-Formats | Version: | n.a. |
Keywords: | n.a. | Cc: | mlinkert, cxallan, cneves, rleigh |
Resources: | n.a. | Referenced By: | n.a. |
References: | n.a. | Remaining Time: | 0.0d |
Sprint: | n.a. |
Description
or "read beyond end of file". One simple example is:
Caused by: java.io.EOFException: Attempting to read beyond end of file. at loci.common.NIOFileHandle.readInt(NIOFileHandle.java:331) ... at ome.io.bioformats.OmeroPixelsPyramidReader.initFile(OmeroPixelsPyramidReader.java:111) at loci.formats.FormatReader.setId(FormatReader.java:1105) at ome.io.bioformats.OmeroPixelsPyramidReader.setId(OmeroPixelsPyramidReader.java:92) at ome.io.bioformats.BfPixelsWrapper.<init>(BfPixelsWrapper.java:60) at ome.io.bioformats.BfPixelBuffer.reader(BfPixelBuffer.java:68) ... 65 more Caused by: java.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:478) at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:336) at loci.common.NIOFileHandle.readInt(NIOFileHandle.java:329) ... 74 more
from forum thread: https://www.openmicroscopy.org/community/viewtopic.php?f=5&t=1256
Change History (11)
comment:1 Changed 12 years ago by jmoore
comment:2 Changed 12 years ago by cneves
This issue is easily reproduced by merging https://github.com/cneves/openmicroscopy/tree/postdecorators/7202_unittest_review on top of develop and running the omeropy unittest:
python setup.py test -s test.gatewaytest.image.ImageTest.testThumbnail
comment:3 Changed 12 years ago by rleigh
Definitely occuring during pyramid generation in the minimal tiff reader (for me):
2012-07-18 11:53:06,211 INFO [ loci.formats.in.MinimalTiffReader] (2-thread-1) Reading IFDs 2012-07-18 11:53:06,216 INFO [ loci.formats.in.MinimalTiffReader] (2-thread-1) Populating metadata 2012-07-18 11:53:06,219 ERROR [ ome.io.nio.PixelsService] (2-thread-1) Error closing pixel pyramid. java.io.EOFException: Attempting to read beyond end of file. at loci.common.NIOFileHandle.readInt(NIOFileHandle.java:331) at loci.common.RandomAccessInputStream.readInt(RandomAccessInputStream.java:362) at loci.formats.in.JPEG2000MetadataParser.parseBoxes(JPEG2000MetadataParser.java:164) at loci.formats.in.JPEG2000MetadataParser.<init>(JPEG2000MetadataParser.java:139) at loci.formats.in.MinimalTiffReader.initFile(MinimalTiffReader.java:431) at loci.formats.in.BaseTiffReader.initFile(BaseTiffReader.java:518) at loci.formats.FormatReader.setId(FormatReader.java:1105) at ome.io.bioformats.OmeroPixelsPyramidWriter.postProcess(OmeroPixelsPyramidWriter.java:85) at ome.io.bioformats.OmeroPixelsPyramidWriter.close(OmeroPixelsPyramidWriter.java:52) at ome.io.bioformats.BfPyramidPixelBuffer.closeWriter(BfPyramidPixelBuffer.java:415) at ome.io.bioformats.BfPyramidPixelBuffer.close(BfPyramidPixelBuffer.java:650) at ome.io.nio.PixelsService.makePyramid(PixelsService.java:210) at ome.services.pixeldata.PixelDataHandler.process(PixelDataHandler.java:137) at ome.services.pixeldata.PixelDataHandler.doWork(PixelDataHandler.java:88) at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:508) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at ome.security.basic.NullEventHandler.invoke(NullEventHandler.java:39) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) at $Proxy66.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:406) at ome.services.util.Executor$Impl.execute(Executor.java:350) at ome.services.pixeldata.PixelDataThread$1.call(PixelDataThread.java:146) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:480) at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:336) at loci.common.NIOFileHandle.readInt(NIOFileHandle.java:329) ... 44 more 2012-07-18 11:53:06,222 INFO [ ome.services.pixeldata.PixelDataHandler] (2-thread-1) Added StatsInfo:9550 for ome.model.core.Channel:Id_9579 - C:0 Max:255.0 Min:0.0 2012-07-18 11:53:06,222 INFO [ ome.services.pixeldata.PixelDataHandler] (2-thread-1) Added StatsInfo:9551 for ome.model.core.Channel:Id_9580 - C:1 Max:255.0 Min:0.0 2012-07-18 11:53:06,223 INFO [ ome.services.pixeldata.PixelDataHandler] (2-thread-1) Added StatsInfo:9552 for ome.model.core.Channel:Id_9581 - C:2 Max:255.0 Min:0.0 2012-07-18 11:53:06,223 INFO [ ome.services.pixeldata.PixelDataHandler] (2-thread-1) HANDLED EventLog:482795(entityId=6633) [342205 ms.] 2012-07-18 11:53:06,244 ERROR [ ome.services.util.ServiceHandler] (2-thread-1) Method interface ome.services.util.Executor$Work.doWork invocation took 342230
comment:4 Changed 12 years ago by rleigh
2012-07-18 11:07:53,161 ERROR [ ome.services.pixeldata.PixelDataHandler] (2-thread-1) Failed to handle pixels 6645 java.lang.IllegalArgumentException: Negative position at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:610) at loci.common.NIOByteBufferProvider.allocateDirect(NIOByteBufferProvider.java:133) at loci.common.NIOByteBufferProvider.allocate(NIOByteBufferProvider.java:118) at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:532) at loci.common.NIOFileHandle.seek(NIOFileHandle.java:254) at loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:129) at loci.formats.in.MinimalTiffReader.initFile(MinimalTiffReader.java:430) at loci.formats.in.BaseTiffReader.initFile(BaseTiffReader.java:518) at loci.formats.FormatReader.setId(FormatReader.java:1105) at ome.io.bioformats.OmeroPixelsPyramidWriter.postProcess(OmeroPixelsPyramidWriter.java:85) at ome.io.bioformats.OmeroPixelsPyramidWriter.close(OmeroPixelsPyramidWriter.java:52) at ome.io.bioformats.BfPyramidPixelBuffer.closeWriter(BfPyramidPixelBuffer.java:415) at ome.io.bioformats.BfPyramidPixelBuffer.close(BfPyramidPixelBuffer.java:650) at ome.io.nio.PixelsService.makePyramid(PixelsService.java:210) at ome.services.pixeldata.PixelDataHandler.process(PixelDataHandler.java:137) at ome.services.pixeldata.PixelDataHandler.doWork(PixelDataHandler.java:88) at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:508) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at ome.security.basic.NullEventHandler.invoke(NullEventHandler.java:39) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) at $Proxy66.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:406) at ome.services.util.Executor$Impl.execute(Executor.java:350) at ome.services.pixeldata.PixelDataThread$1.call(PixelDataThread.java:146) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) 2012-07-18 11:07:53,172 ERROR [ ome.services.util.ServiceHandler] (2-thread-1) Method interface ome.services.util.Executor$Work.doWork invocation took 1590321
comment:5 Changed 12 years ago by jmoore
My best guess is that this is a result of the int_vs_long_srv branch:
commit 99864f17cc312c59cbd35f3e62105198078711bb Merge: 53a32a5 379605e Author: Chris Allan <callan@blackcat.ca> Date: Fri Jun 22 13:23:00 2012 Merge remote-tracking branch 'origin/develop' into int_vs_long_srv Conflicts: components/insight/LIB/ome-io.jar components/romio/ivy.xml components/server/src/ome/services/RawPixelsBean.java etc/omero.properties lib/repository/ome-io-17db504.jar lib/repository/ome-io-362b9a2.jar lib/repository/poi-loci-362b9a2.jar
comment:6 Changed 12 years ago by rleigh
Reverting 99864f17cc312c59cbd35f3e62105198078711bb (git revert -m 2), imports still fail as above.
comment:7 Changed 12 years ago by mlinkert
comment:8 Changed 12 years ago by rleigh
All imports and pyramid generation using test_images_good/svs appear to work just fine. All the big images are viewable at all zoom levels. There's a single exception in test_images_good/svs/alexandra/UMD001_ORO.svs. One of the series imports as:
UMD001_ORO.svs [UMD001_ORO.svs]
Rather than [Series n]. The PixelData?-0 log shows:
2012-07-19 13:04:04,051 INFO [ ome.io.nio.FilePathResolver] (2-thread-1) Metadata only file, resulting path: /OMERO/Files/Dir-005/5465
which I think corresponds to this image. I'm not sure if it's a related issue, or a bug in the SVS reader. In insight, it claims to be 577x368 pixels, 4 timepoints, 3 channels. This looks a bit suspect though (particularly the timepoints), and having the filename as the series name is not common to any of the other SVS files.
Perhaps not related directly, but
% ls ~/images/test_images_good/svs/*.svs /Users/rleigh/images/test_images_good/svs/5794_001.svs /Users/rleigh/images/test_images_good/svs/UMD001_ORO.svs
These two files don't show up in the Insight importer file browser. But these do:
% ls ~/images/test_images_good/svs/alexandra/*.svs /Users/rleigh/images/test_images_good/svs/alexandra/UMD001_ORO.svs /Users/rleigh/images/test_images_good/svs/alexandra/UMD009_HE.svs /Users/rleigh/images/test_images_good/svs/alexandra/UMD019_Tri.svs
comment:9 Changed 12 years ago by mlinkert
Extra SVS series issue confirmed and PR opened: https://github.com/openmicroscopy/bioformats/pull/107
comment:10 Changed 12 years ago by mlinkert
- Remaining Time changed from 2.0 to 0
- Resolution set to fixed
- Status changed from new to closed
PRs merged and 4.4.1 released as a result.
comment:11 Changed 12 years ago by jmoore
- Milestone changed from OMERO-4.4.3 to OMERO-4.4.1
- Sprint 2012-07-31 (1) deleted
This was released as 4.4.1. Moving back.
Error from Carlos: