In-place FS import

[jamoore]

As requested during the Paris 2013 Users' meeting as well as periodically on the mailing lists, it would be useful and in some cases necessary to have OMERO 'import' (or 'register') files where they sit in order to prevent any copy or move action.

This would skip the upload step of ImportLibrary and would instead just symlink files into place.

The primary caveat of this strategy is that the files should be considered to belong to OMERO once this has taken place. Modifications may invalidate the DB state.

See:

• ​http://lists.openmicroscopy.org.uk/pipermail/ome-users/2013-July/003873.html
• ​http://lists.openmicroscopy.org.uk/pipermail/ome-devel/2013-September/002523.html
• ​https://www.openmicroscopy.org/community/viewtopic.php?f=4&t=7341&p=13051#p13051

*Placing in beta2 for evaluation*

[cblackburn]

Also requested verbally at !EuBIAS2013.

This could be very useful for avoiding the duplication inherent in DropBox under FS. Although unrelated if this evaluation is positive it would be worth extending DropBox to handle import into projects, datasets and screens (and groups).

[jamoore]

Initial exploratory PR opened via a script: ​https://github.com/openmicroscopy/openmicroscopy/pull/1929
Java-based CLI PR: ​https://github.com/openmicroscopy/openmicroscopy/pull/1989

[jamoore]

Summary of today's discussion:

• Blazej: this could lead to a paradigm shift in which people stop storing their data. We should be careful.
• Josh: users could be surprised when they delete images and no space is freed
• J-m/Gus: the users need a representation of the "in-placeness" of a fileset in the UI
• Mark: there's a potential maintenance burden, e.g. if a sysadmin moves a mount point, symlinks will need to be updated.
• Josh: we've said to this point that this is an advanced feature and there is risk involved. Sysadmins may have to get involved in writing update scripts.

5.0.0 TODOS (tickets on this story)

• Remove the commands from the basic help (DONE)
• Store the FileTransfer mechanism used per fileset (ticket to come)
• Integrate with Dropbox if possible
• Write documentation available upon request
• Offer for initial testing (post-release)

5.0.1 (trello card)

• Include a representation of the mechanism in the UI
• Integrate documentation
• Provide scenarios & integration tests (earlier than 5.0.1 if possible)

[agilo]

Updated status, related task in progress

[Josh Moore <josh@…>]

(In [8ab2c730d0e213922e2b88507ee4794fdc7459e9/ome.git] on branch develop) Merge pull request #2020 from joshmoore/rebased/develop/file-transfer

Inplace Import via SymlinkFileTransfer? (See #11573) (rebased onto develop)

[jamoore]

With #11931, all the 5.0.0 TODOs have been taken care of. Based on ​https://trello.com/c/fOGdQQCN/64-inplace-import-gui, we will likely need to create a new story and tickets in 5.0.1, etc.