Exiv2

Good to hear from you. Alan's our expert for both XMP and Date data and will say something about this.

When you set Exif.Image.DateTime, you're setting data in to the Exif part of the JPG. Then we have some conversion magic that converts that date into something for XMP.

I know that Exif specification defines dates that do not include TimeZone information. XMP uses an XML date spec which can include TZ information. Perhaps you should be writing Exif dates followed by Xmp dates and you'll end up with something like this:

601 rmills@rmillsmbp:~/temp $ exiv2 -pa Stonehenge.jpg | grep -i Date
Exif.Image.DateTime                          Ascii      20  2015:07:16 20:25:28         <--- Exif date (ISO Standard, no TZ)
Exif.Photo.DateTimeOriginal                  Ascii      20  2015:07:16 15:38:54
Exif.Photo.DateTimeDigitized                 Ascii      20  2015:07:16 15:38:54
Exif.NikonWt.DateDisplayFormat               Byte        1  Y/M/D
Exif.GPSInfo.GPSDateStamp                    Ascii      11  2015:07:16
Xmp.xmp.ModifyDate                           XmpText    25  2015-07-16T20:25:28+01:00   <-- Xmp date (XML standard and includes TZ)
602 rmills@rmillsmbp:~/temp $ 

XMP (with TZ ) -> Exif (TZ lost) -> XMP (no TZ) [ -> means "is converted to" ].

I fixed that by a storing the XMP Dates before setting Exif dates. Then I restored the XMP dates. You can see the fix in SVN in the issue report.

Of course, your circumstances might be a little different. Let's wait for Alan to chime into this and we can discuss this further if you wish.

Ah yes, right up my alley! Give me some moments to parse the xmp sidecar.

Mike, if you could run the command:

 exiv2 -g Date 06068-20150411-LaurenBrad-Lutrus-wed-bc-AlienSkinX.jpg 

As always, it also helps to review the MWG guidelines on metadata DateTime fields.

http://www.metadataworkinggroup.org/pdf/mwg_guidance.pdf#page=37

I have attached a text file that is the output of exiv2 -pa for the jpeg that we export

also, the -g Date command on the jpeg gives me:

Exif.Image.DateTime                          Ascii      20  2015:10:30 17:12:57
Exif.Photo.DateTimeDigitized                 Ascii      20  2015:04:11 11:39:23
Iptc.Application2.DateCreated                Date        8  2015-04-11
Xmp.xmp.CreateDate                           XmpText    22  2015-04-11T11:39:23.44
Xmp.xmp.ModifyDate                           XmpText    25  2015-10-30T17:12:57-04:00
Xmp.xmp.MetadataDate                         XmpText    20  2015-10-30T21:14:15Z
Xmp.tiff.DateTime                            XmpText    19  2015-10-30T17:12:57
Xmp.exif.DateTimeDigitized                   XmpText    22  2015-04-11T11:39:23.44
Xmp.photoshop.DateCreated                    XmpText    10  2015-04-11

Strange thing is that Exif.Photo.DateTimeOriginal is missing from the image. That field is always written by digital cameras, if it wasn't, it's a bug in the camera firmware! Otherwise, how did it get deleted?

MWG guidelines (and exiv2) map Exif.Photo.DateTimeOriginal to Xmp.photoshop.DateCreated which IS present, but only has Date information, not Time. Something funky is happening to the metadata of that "original" image file before exiv2 gets it's hands on it!

That's the issue. The field is not missing from the source image. It is only missing because it doesn't get into our sidecar, and we use the sidecar as a repository for the metadata that goes in the image when we export.

Anyway I figured it out. Exiv seems hostile to the "belt-and-suspenders" ( "belt-and-braces" over there?) approach.

If I do this:

image->xmpData()["Xmp.exif.DateTimeOriginal"] = item->captureTime().toString( Qt::ISODate ).toStdString();
image->xmpData()["Xmp.exif.DateTimeDigitized"] = item->captureTime().toString( Qt::ISODate ).toStdString();
image->xmpData()["Xmp.tiff.DateTime"] = item->captureTime().toString( Qt::ISODate ).toStdString();


followed by this:
image->exifData()["Exif.Image.DateTime"] = item.captureTime().toString( Qt::ISODate ).toStdString();
image->exifData()["Exif.Photo.DateTimeOriginal"] = item.captureTime().toString( Qt::ISODate ).toStdString();
image->exifData()["Exif.Photo.DateTimeDigitized"] = item.captureTime().toString( Qt::ISODate ).toStdString();

none of the exiv fields get written to the sidecar. This seems to be because of the way the converter functions are written, but I haven't dived in too deeply there. This is what the exported jpeg looks like with exiv -g Date when both sets of keys are set

Iptc.Application2.DateCreated Date 8 2015-04-11
Xmp.xmp.CreateDate XmpText 22 2015-04-11T11:39:23.44
Xmp.xmp.ModifyDate XmpText 25 2015-10-30T17:12:57-04:00
Xmp.xmp.MetadataDate XmpText 20 2015-11-02T17:27:40Z
Xmp.photoshop.DateCreated XmpText 10 2015-04-11

if I just set the exif versions of the keys it is the same thing. The capture time info is omitted from the sidecar.
When I just use the pure xmp versions of the keys, all is well. The exported jpg looks like this with exiv -g Date

Exif.Image.DateTime Ascii 20 2015:04:11 11:39:23
Exif.Photo.DateTimeOriginal Ascii 20 2015:04:11 11:39:23
Exif.Photo.DateTimeDigitized Ascii 20 2015:04:11 11:39:23
Iptc.Application2.DateCreated Date 8 2015-04-11
Xmp.xmp.CreateDate XmpText 22 2015-04-11T11:39:23.44
Xmp.xmp.ModifyDate XmpText 25 2015-10-30T17:12:57-04:00
Xmp.xmp.MetadataDate XmpText 20 2015-11-02T17:18:48Z
Xmp.exif.DateTimeDigitized XmpText 20 2015-04-11T15:39:23Z
Xmp.exif.DateTimeOriginal XmpText 19 2015:04:11 11:39:23
Xmp.tiff.DateTime XmpText 20 2015-04-11T15:39:23Z
Xmp.photoshop.DateCreated XmpText 10 2015-04-11

So, word to anyone using exiv with sidecars: The library is rather pedantic about what create date keys you use when writing to / reading from sidecars. Only use the pure xmp versions of the keys. This may apply to lots of other keys that get converted, too. I am going to be checking that very closely.

I see, you provided the exported image data, not the original.

Yes, sorry about that. exiv -g Date on the original looks like:

Exif.Image.DateTime Ascii 20 2015:10:30 17:12:57
Exif.Photo.DateTimeOriginal Ascii 20 2015:04:11 11:39:23
Exif.Photo.DateTimeDigitized Ascii 20 2015:04:11 11:39:23
Iptc.Application2.DateCreated Date 8 2015-04-11
Iptc.Application2.DigitizationDate Date 8 2015-04-11
Xmp.xmp.CreateDate XmpText 22 2015-04-11T11:39:23.44
Xmp.xmp.ModifyDate XmpText 25 2015-10-30T17:12:57-04:00
Xmp.xmp.MetadataDate XmpText 25 2015-10-30T17:12:57-04:00
Xmp.photoshop.DateCreated XmpText 25 2015-04-11T11:39:23+00:00

I don't think you should be writing to Xmp.exif.DateTimeOriginal, Xmp.exif.DateTimeDigitized & Xmp.tiff.DateTime.

According to MWG guidelines, the correct XMP properties for those are, in the same order: Xmp.photoshop.DateCreated, Xmp.xmp.CreateDate & Xmp.xmp.ModifyDate.

exiv2 runs a convert function when writing and reading xmp sidecar files. The conversion table for exiv2 version 0.25 is here: http://www.exiv2.org/conversion.html

Earlier versions had a different conversion table, but it was updated to agree with MWG guidelines.

We are still on 0.24 because 0.25 came too close to release.
According to the convert tables in convert.cpp:

line 341 maps Xmp.exif.DateTimeOriginal to Exif.Photo.DateTimeOriginal
line 342 maps Xmp.exif.DateTimeDigitized to Exif.Photo.DateTimeDigitized
line 325 maps Xmp.tiff.DateTime to Exif.Image.DateTime

I don't know what MWG says, but the other editors that exist in the real world say that DateTimeOriginal in the exif metadata of an image absolutely needs to be filled in. I can't get back in a situation where these fields are blank in images exported by us. It is quite easy using exiv 0.24 to find ourselves in that situation. I think I am going to stick with the current mappings because they actually work in real world code. Maybe we will revisit it if / when we go to 0.25.

Mike Erickson wrote:

We are still on 0.24 because 0.25 came too close to release.
According to the convert tables in convert.cpp:

line 341 maps Xmp.exif.DateTimeOriginal to Exif.Photo.DateTimeOriginal
line 342 maps Xmp.exif.DateTimeDigitized to Exif.Photo.DateTimeDigitized
line 325 maps Xmp.tiff.DateTime to Exif.Image.DateTime

Yes, correct, those where the old mappings, but had been obsolete for some time.

http://dev.exiv2.org/issues/864

I don't know what MWG says,

http://www.metadataworkinggroup.org/pdf/mwg_guidance.pdf#page=37

but the other editors that exist in the real world say that DateTimeOriginal in the exif metadata of an image absolutely needs to be filled in.

100% correct and that is the way exiv2 should always work. My command line testing on these conversions go back to 0.23 and I have not come across the issue you are seeing before.

I can't get back in a situation where these fields are blank in images exported by us. It is quite easy using exiv 0.24 to find ourselves in that situation.

So this behaviour started with exiv2 0.24?

I think I am going to stick with the current mappings because they actually work in real world code. Maybe we will revisit it if / when we go to 0.25.

I'd be curious if the same issues occur with 0.25. Testing that might help track down where the problem is occurring.

Not sure when it started as we've only ever used 0.24. We should be switching over to 0.25 after the holidays. I'll let you know. This could be related to how I coerce exiv into writing a pure xmp sidecar. I may be doing it in a hacky way.