Bug #1064
exiv2 -iX <file.jpg> deletes Makernote
100%
Description
exiv2 -iX <file.jpg> is used to import metadata from an XMP sidecar file which was typically created with the command: exiv2 -eX <file.jpg>.
If the jpg file contains a makernote, -iX deletes that makernote. This was tested on images with both Nikon makernotes and on images with Olympus makernotes.
The XMP sidecar file created by -eX does not contain any makernote information. The XMP specification does not say anything about storing makernotes in XMP sidecar files, so that behaviour is normal and correct.
Importing metadata from XMP sidecar file should not delete makernotes.
Associated revisions
#1064 Fix -iX to merge (not replace) metadata from source to destination.
History
Updated by Robin Mills over 6 years ago
- Status changed from New to Assigned
- Target version set to 0.26
I'm going to push this one to v0.26. If this is a bug, it's been there a long time and nobody has mentioned it. If this a feature request to preserve the Makernote in the image being edited, I want to defer it to 0.26.
I feel that exiv2 -eX should export the Makernote as the attribute exif.Makernote="..base64.." and exiv2 -iX should recreate the Makernote. There are at least three reasons:
- It's symmetrical
- It's consistent with exiv2 -e{tgt} and exiv2 -i{tgt} which create and read .exv "sidekick files"
- It's clean. Preserving the Exif.Makernote when updating other Exif takes is ugly. It might even be difficult to implement.
Updated by Alan Pater over 6 years ago
Lot's of people will be happy if we can figure out a reliable way to store makernotes in XMP. But also, some say no, not possible, don't go there. So not everyone will be happy ...
Phil Harvey, exiftool: "... the MakerNote tag existed in the early Adobe XMP specification, they never defined an encoding method for the data, so the tag is essentially useless .." http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=1878.0
Leonard Rosenthol, Adobe Systems: "I've discussed this the XMP team and they do NOT recommend the inclusion of a "binary blob" of data inside of XMP, and ESPECIALLY NOT the MakerNote data." http://lists.freedesktop.org/archives/create/2007-June/000835.html
Perhaps a -eMX option to export the makernote to a separate "binary blob" file alongside the xmp sidecar file? Then -iMX would re-import them both.
Updated by Robin Mills over 6 years ago
Thanks for digging into this, Alan. We mustn't put the makernote into the XMP. Adobe, undoubtably for very good reasons, have said "No. Don't go there!". We don't need to add an M flag to say "Do a Microsoft and ignore the standard" We already have the .exv file that preserves the makernote. We don't need a sidecar to the sidecar!
So that brings us back to the original topic which is to retain the image's makernote when reading in a sidecar. What happens if the side car contradicts the makernote. We don't have an code to rewrite/update the makernote.
We could compromise and retain the makernote if it doesn't contradict the sidecar. However when there's conflict, we remove the makernote. That's a mess. Somebody will report a bug saying, sometimes the makernote is deleted and sometimes it isn't.
So, how about an m import tgt to say "retain the makernote". We can put out a warning if we think it's conflicted.
We could have import tgt m or M:
m = retain makernote when not conflicted
M = retain makernote unconditionally
Updated by Alan Pater over 6 years ago
Robin Mills wrote:
So that brings us back to the original topic which is to retain the image's makernote when reading in a sidecar. What happens if the side car contradicts the makernote. We don't have an code to rewrite/update the makernote.
I don't think there are any conflicts between information in the
makernote and in XMP, IPTC or Exif. I thought makernotes are completely
separate tags in a completly different location inside the file.
For instance, all the Nikon makernote tags are under Exif.Nikon...
http://www.exiv2.org/tags-nikon.htmlAll the Exif tags are under Exif.Image or Exif.Photo or Exif.GPSInfo.
There is no overlap.
Updated by Robin Mills over 6 years ago
Oh, the makernote tag is safely stored in the file. If information for the XMP is derived from the MakerNotes, it could be edited in XMP. When we read the XMP back, it could conflict. Example:
725 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa -g eed ~/R.jpg Exif.Photo.ISOSpeedRatings Short 1 400 Exif.Nikon3.ISOSpeed Short 2 400 Exif.Nikon3.CropHiSpeed Short 7 0 6016 4016 6016 4016 0 0 726 rmills@rmillsmbp:~/gnu/exiv2/trunk $The XMP looks like this:
727 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -eX -f ~/R.jpg
728 rmills@rmillsmbp:~/gnu/exiv2/trunk $ grep -i -B 2 -A 2 speed ~/R.xmp
</rdf:Alt>
</exif:UserComment>
<exif:ISOSpeedRatings>
<rdf:Seq>
<rdf:li>400</rdf:li>
--
--
<rdf:li>400</rdf:li>
</rdf:Seq>
</exif:ISOSpeedRatings>
<exif:Flash
exif:Fired="False"
731 rmills@rmillsmbp:~/gnu/exiv2/trunk $So if we modify the 400 to 200 and read it back into the image, we have 2 values for the ISOSpeedRatings. I think a warning is all that required.Updated by Alan Pater over 6 years ago
Agreed. ISO speed is not something anyone should be modifying after the fact.
Updated by Alan Pater over 6 years ago
Besides, we probably will not be able to edit Xmp.exif.ISOSpeedRatings.
"Editing existing XMP properties in XMP sidecars does not work"
http://dev.exiv2.org/issues/639
Updated by Robin Mills almost 6 years ago
- % Done changed from 0 to 10
- Estimated time set to 10.00 h
I'm adding a SWAG to my outstanding issues for v0.26 to get a more accurate assessment of the status of v0.26. I can't remember anything about this - so I'll guess about a day's work.
Updated by Robin Mills about 5 years ago
- % Done changed from 10 to 30
Here's my view of this issue. -ie replaces the Makernote, -iX does not. It feels like a bug in -iX. However -iX reads from foo.xmp (not foo.exv). The Sidecar (foo.xmp) does not have the makernote. I think the fix for -iX to update the Exif data from the Sidecar and not to replace it. Here's my analysis:
1 Setup the test data:
810 rmills@rmillsmbp:~/gnu/exiv2/trunk $ curl -O --silent http://clanmills.com/Stonehenge.jpg
811 rmills@rmillsmbp:~/gnu/exiv2/trunk $ mv Stonehenge.jpg S.jpg
812 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pR S.jpg | grep -i makernot
416 | 0x927c MakerNote | UNDEFINED | 3152 | 914 | Nikon.....II*.....9.........0211 ...
813 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep MakerNote S.jpg
Exif.Photo.MakerNote Undefined 3152 (Binary value suppressed)
Exif.MakerNote.Offset Long 1 914
Exif.MakerNote.ByteOrder Ascii 3 II
814 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -ea --verbose --force S.jpg
File 1/1: S.jpg
Writing Exif data from S.jpg to ./S.exv
Writing IPTC data from S.jpg to ./S.exv
Writing XMP data from S.jpg to ./S.exv
815 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pR S.jpg | grep -i makernot
416 | 0x927c MakerNote | UNDEFINED | 3152 | 914 | Nikon.....II*.....9.........0211 ...
816 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep MakerNote S.exv
Exif.Photo.MakerNote Undefined 3152 (Binary value suppressed)
Exif.MakerNote.Offset Long 1 820
Exif.MakerNote.ByteOrder Ascii 3 II
817 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 2 -ie works820 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pR S.jpg | grep -i makernote
416 | 0x927c MakerNote | UNDEFINED | 3152 | 914 | Nikon.....II*.....9.........0211 ...
821 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa S.jpg | grep -i makernote
Exif.Photo.MakerNote Undefined 3152 (Binary value suppressed)
Exif.MakerNote.Offset Long 1 914
Exif.MakerNote.ByteOrder Ascii 3 II
822 rmills@rmillsmbp:~/gnu/exiv2/trunk $
3 -iX requires S.xmp829 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -iX --verbose --force S.jpg File 1/1: S.jpg ./S.xmp: Failed to open the file 830 rmills@rmillsmbp:~/gnu/exiv2/trunk $4 The man page states
-i tgt Insert target(s) for the 'insert' action. Possible targets are
the same as those for the -d option, plus an optional modifier:
X : Insert metadata from an XMP sidecar file <file>.xmp. The
remaining insert targets determine what metadata to insert from
the sidecar file. Possible are Exif, IPTC and XMP and the
default is all of these. Note that the inserted XMP properties
include those converted to Exif and IPTC.5 Create S.xmp with -eX
This is xmp and does not contain the makernote (as explained above).
831 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -eX --verbose --force S.jpg File 1/1: S.jpg Writing Exif data from S.jpg to ./S.xmp Writing XMP data from S.jpg to ./S.xmp 832 rmills@rmillsmbp:~/gnu/exiv2/trunk $ grep -i maker S.xmp 833 rmills@rmillsmbp:~/gnu/exiv2/trunk $6 -iX destroys the makernote
833 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -iX --verbose --force S.jpg File 1/1: S.jpg Writing Exif data from ./S.xmp to S.jpg Writing IPTC data from ./S.xmp to S.jpg Writing XMP data from ./S.xmp to S.jpg 834 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa S.jpg | grep -i makernote 835 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pR S.jpg | grep -i makernote 835 rmills@rmillsmbp:~/gnu/exiv2/trunk $7 Conclusion
I think -iX should "merge" the metadata from S.xmp into S.jpg. It currently replaces it. I'm going to investigate how to achieve that.
If a user wishes to retain the existing behaviour, it can be easily achieved by deleting the Exif metadata with -de before using -iX. For example:
$ exiv2 -de -iX S.jpg
Updated by Robin Mills about 5 years ago
- Status changed from Assigned to Closed
- % Done changed from 30 to 100
- Estimated time changed from 10.00 h to 5.00 h
Fix submitted: r4427
Brilliant analysis, Robin. The code in src/actions.cpp+1993 did not handle the preserve flag correctly.
int metacopy(const std::string& source,
const std::string& tgt,
int targetType,
bool preserve);
#1057, #1064, #922, #1148. Work in progress. This is a composite patch of several matters in development. None are totally complete at this time.