BUG: Disk space hanging on Nightshade

[rkferguson]

Trying to do some screenshots on Nightshade using Web client.
If access Username > User Settings get my details OK.
If I click on the Drivespace tab Safari hangs with beach ball.
Occurred at 1200 hrs logged in as rkferguson to see logs.

Screenshot attached.

[rkferguson]

Not hung, but eventually comes up after about 10 minutes. Browser unusable while waiting.

[jamoore]

Will / Ola: does this fall under an existing ticket? Or should this be moved to 5.0.2?

[wmoore]

I don't remember exactly how we're making these queries for an individual user's usage, but certainly Admin's disk usage chart fails on nightshade because we query 1000 images at a time and assign usage to the various users and there are simply too many images on nightshade.
Also, with OMERO 5 we should really query the size of the files themselves, instead of trying to calculate usage by pixel count and pixel type!

So, we need to rewrite all the disk usage queries in some way and test on nightshade.

[wmoore]

Josh - is there any way of querying the disk usage of /Pixels/ other than using size X, Y, Z, C, T and pixels type from the DB? If not, then I don't know of a way to make this work. Any ideas?

[khgillen]

Silly question, but can't the OMERO server service run e.g. du -h and du -sh on the Managed Repository for this?

[jamoore]

To Will:

-- "Estimated" /OMERO/Pixels size
ome2=> select pg_size_pretty(sum(sizex::int8 * sizey * sizez * sizet * sizec * bitsize/8)::int8) from pixels, pixelstype where pixels.pixelstype = pixelstype.id;
 pg_size_pretty 
----------------
 72 GB
(1 row)

-- Filter out OMERO5 data	
ome2=> select pg_size_pretty(sum(sizex::int8 * sizey * sizez * sizet * sizec * bitsize/8)::int8) from pixels, pixelstype, image where pixels.pixelstype = pixelstype.id and pixels.image = image.id and image.fileset is null;
 pg_size_pretty 
----------------
 2064 byte

Kenny: we've had that previously (ask Carlos). The server can quickly die under the load of calculating it. (Also doesn't work on Windows)

[wmoore]

That won't tell us how much each user in OMERO is using for OMERO 4 images (nothing in Managed Repository). I think we do use df or something like that to get the free space.

[wmoore]

I'm struggling to convert to iQuery:

After banging my head against the wall for half an hour, the closest I can get is:

query = "select p.sizeX * p.sizeY from Pixels as p"
s = conn.getQueryService().projection(query, p, ctx)

if I add anything to that, like:

query = "select sum(p.sizeX * p.sizeY) from Pixels as p"

I get various exceptions.

[jamoore]

omero@gretzky:~/OMERO-CURRENT$ bin/omero hql -q --admin "select sum(cast( p.sizeX as double ) * p.sizeY * p.sizeZ * p.sizeT * p.sizeC * pt.bitSize / 8) from Pixels p join p.pixelsType as pt" 
Using session 8df7dbca-328a-40a7-b4d8-2e4310907bb7 (root@localhost:4064). Idle timeout: 10.0 min. Current group: read-only-1
 # | Col1           
---+----------------
 0 | 658006364493.0 
(1 row)
omero@gretzky:~/OMERO-CURRENT$ bin/omero hql -q --admin "select sum(cast( p.sizeX as double ) * p.sizeY * p.sizeZ * p.sizeT * p.sizeC * pt.bitSize / 8) from Pixels p join p.pixelsType as pt join p.image i left outer join i.fileset f where f is null" 
Using session 8df7dbca-328a-40a7-b4d8-2e4310907bb7 (root@localhost:4064). Idle timeout: 10.0 min. Current group: read-only-1
 # | Col1           
---+----------------
 0 | 390249741645.0 
(1 row)

[wmoore]

Thanks Josh - that's great!
Just one more question - That query seems to be giving me a number that's double what the filesystem (and our previous 'manual' query) are giving.
E.g. when limiting to a single Pixels:

    p = omero.sys.ParametersI()
    p.add("pid", omero.rtypes.rlong(57))
    query = "select sum(cast( p.sizeX as double ) * p.sizeY * p.sizeZ * p.sizeT * p.sizeC * pt.bitSize / 8) " \
        "from Pixels p join p.pixelsType as pt join p.image i left outer join i.fileset f where f is null and p.id = (:pid)"
    s = conn.getQueryService().projection(query, p, ctx)
    print "Sum query:", s[0][0].val / 1000000, "MB"

This prints Sum query: 55.9872 MB
but the size of the file on disk is 28 MB.

If I don't limit it by ID, I get a total of Sum query: 562.755584 MB
but my Mac tells me that the /Pixels/ folder is 281 GB.

Cheers,

Will.

[jamoore]

I'm thinking this is due to the signange of the bits:

omero-c6100-dec-5-2013-valuable=> select * from pixelstype where id = 6;
-[ RECORD 1 ]-------
id          | 6
bitsize     | 32
permissions | -52
value       | uint16
external_id | 

omero-c6100-dec-5-2013-valuable=> \q
omero@gretzky:~/OMERO-CURRENT$ ls -ltra /nightshade/OMERO/omero-c6100-dec-5-2013-valuable/OMERO-merge-develop-FS/Pixels/Dir-034/34974
-rw-r--r-- 1 omero omero 6193152 2014-03-28 12:46 /nightshade/OMERO/omero-c6100-dec-5-2013-valuable/OMERO-merge-develop-FS/Pixels/Dir-034/34974


omero@gretzky:~/OMERO-CURRENT$ bin/omero hql -q --admin "select p.id, (cast( p.sizeX as double ) * p.sizeY * p.sizeZ * p.sizeT * p.sizeC * pt.bitSize / 8) from Pixels p join p.pixelsType as pt join p.image i left outer join i.fileset f where f is null"  | grep 34974
Using session 2c95d8c7-f0ba-4ef5-92af-b551b0946325 (root@localhost:4064). Idle timeout: 10.0 min. Current group: read-only-1
 20 | 34974 | 12386304.0 

Adding Melissa for confirmation.

We don't yet have this in the model, so we'd like need to do a UNION.

[mlinkert]

uint16 data is definitely going to have 2 bytes per pixel on the filesystem, so if it's 32 bits in the db (which is a little odd, but I guess makes sense) then you'll need to divide by 2.

[jamoore]

Melissa: would you change it to 16 then? ​https://github.com/openmicroscopy/openmicroscopy/blob/develop/sql/psql/OMERO5.0__0/psql-footer.sql#L2017

[mlinkert]

PR opened: ​https://github.com/openmicroscopy/openmicroscopy/pull/2261

[wmoore]

​https://github.com/openmicroscopy/openmicroscopy/pull/2270

[wmoore]

Actually - I need to also look at the User's / Group Owner's page - so, not done yet.

[jamoore]

Will: do you expect the rest of this to be in 5.0.2?

[wmoore]

Sorry - As I understand it, this actually got covered by the original PR, so we can close. ​https://github.com/openmicroscopy/openmicroscopy/pull/2270 That right Gus?

[rkferguson]

Ola did the testing so if she is happy I am happy.

[jamoore]

Thanks all, closing.