Disallow unparsable timestamp

[omero-qa]

Create a new OME timestamp type that is a tighter restriction on the standard XSD timestamp. This will limit the object to what the code libraries we have available can actually process. Mostly this is limiting seconds to three decimal places.

The acceptable formats will follow these patterns

1111-11-11T11:11:11.111
1111-11-11T11:11:11.111Z
1111-11-11T11:11:11.111+11:11
1111-11-11T11:11:11.111-11:11
i.e.
YYYY-MM-DDThh:mm:ss.sss+HH:zz
Year
Month
Day
hour
minute 
second (with up to three decimal places)
Hour of time zone
z - minute of time zone

---

Was: Bug: Model supports only second resolution

​https://www.openmicroscopy.org/qa2/qa2/qa/feedback/7443/

Comment: The little coloured squares at the bottom of an image reporting the acquisition time for each channel don't work for images less than one second. So in a 3 channel image I only get 2 values (for green and red) for the first 6 z-sections with the last value (blue) appearing after that. I guess this is because of the values being rounded down to the nearest second, but it does look a bit odd.

AcquisitionTime? and TimeIncrement? have only second resolution; it's not possible for higher resolution timing to be stored at present.

[ajpatterson]

Schema does not need changed.

xsd:dateTime already has arbitrary precision for seconds.

see: ​http://www.w3.org/TR/xmlschema-2/#isoformats

Functions in the OME Model API will need updated.

Display code in OMERO.insight will need updated.

[jamoore]

This isn't a problem with the specification per-se, but there are no libraries for supporting all of the variants:

• In 4.4, we could put in a documentation change, not necessarily.
• In 5.x, we should disallow timestamps that aren't parse-able in our coding languages.

[rleigh]

Just to document what's parseable at present, this is what Bio-Formats is currently using (from ome-xml Timestamp.java):

Input:
  /** ISO 8601 date input formatter. */
  DateTimeFormatter ISO8601_PARSER =   ISODateTimeFormat.dateTimeParser().withZone(DateTimeZone.UTC);

Output:
  /** ISO 8601 date output format with milliseconds. */
  String ISO8601_FORMAT_MS = "yyyy-MM-dd'T'HH:mm:ss.SSS";

  /** ISO 8601 date output format without milliseconds. */
  String ISO8601_FORMAT_S = "yyyy-MM-dd'T'HH:mm:ss";

  /** ISO 8601 date output formatter with milliseconds. */
  DateTimeFormatter ISO8601_FORMATTER_MS = DateTimeFormat.forPattern(ISO8601_FORMAT_MS);

  /** ISO 8601 date output formatter without milliseconds. */
  DateTimeFormatter ISO8601_FORMATTER_S = DateTimeFormat.forPattern(ISO8601_FORMAT_S);

For input, we nominally accept any ISO8601 date. However, the reality is that while this is flexible, it's only supporting a fraction of the total spec and with limited precision. It /should/ support xsd:dateTime fairly comprehensively. However, precision is limited to milliseconds internally. It should handle input with greater than ms precision (untested), but will lose the extra precision.

For output, if the date has a millisecond fraction which is not .000s we output in ISO8601 with ms precision, or else we output ISO8601 without a subsecond part. The degrading to second resolution is to keep output smaller and compatible with existing data where the millisecond component is not used.

So for java the documented input and output is the format string "yyyy-MM-dd'T'HH:mm:ss.SSS" with some tolerance for variations on the input side and degrading to second resolution on output where possible. However, other languages won't necessarily have the ability to handle such flexible input (and the java code didn't until last week--the input was the second-accuracy format string with no timezone), so using the restricted format here in the specification means that implementations get guaranteed parseable input and it gives us the option to relax it and add e.g. more precision in a future update. With this restricted format, it should be parseable by hand in any language, or using library facilities where available. NB. we aren't specifying an internal format, so this can be platform/language specific so long as it's lossless.

Regarding negative dates, this is probably acceptable, so long as the implementation can accept it. On UNIX time_t is signed, so can store dates prior to 1970. On 64-bit systems this can go back a long way (note: date(1) lacks -d on the Mac, works on Linux):

$ date +%s -d "Mon Dec 26 09:46:24 1900"
-2177936016

$ date +%s -d "Mon Dec 26 09:46:24 0000"
-62136079941

$ date -d @-2438373434543
Wed Jan  9 05:56:22 LMT -75299

However, this does /not/ mean that the year parsing can handle negative dates (this one does not directly even though I can enter the time manually and get a negative year), or that this works across different platforms and languages. In this case this would only work on a 64-bit system with library support and time database to handle it.

In short, I think we /could/ support it, but this support would never be universal, and would break some implementations. So long as the limitations are clearly documented, I don't think it's unreasonable to allow it--it's not like this is going to be seen except in very special cases. But we probably do want tests for it to make sure it's working correctly; your examples look pretty good for that.

[ajpatterson]

Referencing ticket #10268 has changed sprint.

[ajpatterson]

Only schema changes were to add more samples, see: ​https://trello.com/c/P8Ob73SO/19-timestamp