Bug #1284
Possible exiv2 0.26-svn bug
100%
Description
Hello,
Here is a sample image:
https://drive.google.com/open?id=0B5_iknSPeSNBcGFZcUduTzJsZzQ
The XMP packet in the image start with :
<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:xmp="http://ns.adobe.com/xap/1.0/"><xmp:CreatorTool>Microsoft Photo Gallery 16.4.3528.331</xmp:CreatorTool></rdf:Description><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-29/"><prefix0:LocationCreated><rdf:Bag xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:li><rdf:Description xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><prefix0:CountryName xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">Canada</prefix0:CountryName><prefix1:ProvinceState xmlns:prefix1="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">AB</prefix1:ProvinceState><prefix2:City xmlns:prefix2="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">Alberta</prefix2:City></rdf:Description>
Exiv2 CLI tool said that a XMP namespace do not match.
I thought maybe the XMP definition is be fully implemented in Exiv2. Can you investigate when you have a chance please?
Related issues
History
Updated by Robin Mills over 4 years ago
- Status changed from New to Assigned
- Assignee set to Robin Mills
- % Done changed from 0 to 30
- Estimated time set to 3.00 h
Wil:
Thank you for reporting this and providing the test file. Is that photo in New Zealand?
I've reproduced the fault as follows:
520 rmills@rmillsmbp:~ $ exiv2 -px ~/Downloads/20160911_132121.jpg Error: XMP Toolkit error 101: Schema namespace URI and prefix mismatch Warning: Failed to decode XMP metadata. 521 rmills@rmillsmbp:~ $This is an error being thrown by the XMPsdk embedded in Exiv2. This error isn't being thrown by the latest Adobe XMPsdk:
526 rmills@rmillsmbp:~/gnu/xmpsdk $ XMP-Toolkit-SDK-CC201607/samples/target/macintosh/intel_64/Release/DumpFile ~/Downloads/20160911_132121.jpg Dumping JPEG file [size 2499072 (0x262200)] JPEG:FFD8 [offset 0 (0x0),SOI] ... OK 527 rmills@rmillsmbp:~/gnu/xmpsdk $Exiv2 can perform as similar file analysis:
528 rmills@rmillsmbp:~/gnu/xmpsdk $ exiv2 -pS ~/Downloads/20160911_132121.jpg STRUCTURE OF JPEG FILE: /Users/rmills/Downloads/20160911_132121.jpg address | marker | length | data 0 | 0xffd8 SOI 2 | 0xffe0 APP0 | 16 | JFIF.....H.H.... 20 | 0xffe1 APP1 | 14936 | Exif..MM.*...................... 14958 | 0xffe4 APP4 | 13144 | ............................... 28104 | 0xffe5 APP5 | 26 | ....2016:09:11 19:21:21... 28132 | 0xffe1 APP1 | 13429 | http://ns.adobe.com/xap/1.0/.<?x 41563 | 0xffdb DQT | 67 41632 | 0xffdb DQT | 67 41701 | 0xffc0 SOF0 | 17 41720 | 0xffc4 DHT | 31 41753 | 0xffc4 DHT | 181 41936 | 0xffc4 DHT | 31 41969 | 0xffc4 DHT | 181 42152 | 0xffda SOS 529 rmills@rmillsmbp:~/gnu/xmpsdk $And exiv2 can extract the "raw" XMP.
521 rmills@rmillsmbp:~ $ exiv2 -pX ~/Downloads/20160911_132121.jpg | xmllint --format - <?xml version="1.0"?> <?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?> <x:xmpmeta xmlns:x="adobe:ns:meta/"> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:Description xmlns:xmp="http://ns.adobe.com/xap/1.0/" rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b"> <xmp:CreatorTool>Microsoft Photo Gallery 16.4.3528.331</xmp:CreatorTool> </rdf:Description> <rdf:Description xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-29/" rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b"> <prefix0:LocationCreated> <rdf:Bag xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:li> <rdf:Description xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <prefix0:CountryName xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">Canada</prefix0:CountryName> <prefix1:ProvinceState xmlns:prefix1="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">AB</prefix1:ProvinceState> <prefix2:City xmlns:prefix2="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">Alberta</prefix2:City> </rdf:Description> </rdf:li> </rdf:Bag> </prefix0:LocationCreated> </rdf:Description> </rdf:RDF> </x:xmpmeta> <?xpacket end='w'?> 522 rmills@rmillsmbp:~ $I don't know the meaning of the XMPsdk exception: Error: XMP Toolkit error 101: Schema namespace URI and prefix mismatch. It's clearly legal XML, however there something about the namespace declarations that is upsetting XMPsdk. The pattern of XMP/namespaces looks similar to test/data/exiv2-1112.xmp
535 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa test/data/exiv2-bug1112.xmp Exif.Photo.DateTimeDigitized Ascii 20 2012:02:01 14:28:00 Iptc.Application2.DigitizationDate Date 8 2012-02-01 Iptc.Envelope.CharacterSet String 3 G Xmp.xmp.CreateDate XmpText 25 2012-02-01T16:28:00+02:00 536 rmills@rmillsmbp:~/gnu/exiv2/trunk $ cat test/data/exiv2-bug1112.xmp <?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?> <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2"> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:Description rdf:about="" xmlns:xmp="http://ns.adobe.com/xap/1.0/" xmp:CreateDate="2012-02-01T16:28:00+02:00"/> </rdf:RDF> </x:xmpmeta> <?xpacket end="w"?>537 rmills@rmillsmbp:~/gnu/exiv2/trunk $I think there's something in prefix0 (or prefix1 or prefix2) causing XMPsdk's code
<exiv2-dir>/xmpsdk/src/ParseRDF.cpp
to stop parsing the file.
I think you will have to register the namespaces prefix0/1/2 as they are not registered by default with XMPsdk. You can determine the pre-registered namespaces with the command:
$ exiv2 --verbose --version --grep xmlns ... xmlns=xmpNote:http://ns.adobe.com/xmp/note/ xmlns=xmpRights:http://ns.adobe.com/xap/1.0/rights/ xmlns=xmpT:http://ns.adobe.com/xap/1.0/t/ xmlns=xmpTPg:http://ns.adobe.com/xap/1.0/t/pg/ xmlns=xmpidq:http://ns.adobe.com/xmp/Identifier/qual/1.0/ 541 rmills@rmillsmbp:~/gnu/exiv2/trunk $I've tried to register prefix0, prefix1 and prefix2:
565 rmills@rmillsmbp:~/gnu/exiv2 $ exiv2 \ -M'reg prefix0 http://iptc.org/std/Iptc4xmpExt/2008-02-29/' \ -M'reg prefix1 http://iptc.org/std/Iptc4xmpExt/2008-02-29/' \ -M'reg prefix2 http://iptc.org/std/Iptc4xmpExt/2008-02-29/' \ ~/Downloads/20160911_132121-2.jpg Error: XMP Toolkit error 101: Schema namespace URI and prefix mismatch Warning: Failed to decode XMP metadata. 566 rmills@rmillsmbp:~/gnu/exiv2 $There is something suspicious about the declaration of prefix0, as it's declared twice. However rdf is also declared twice without obvious panic. I also notice that the URI for prefix0, prefix1 and prefix2 are identical. I don't know if that's significant.
567 rmills@rmillsmbp:~/gnu/exiv2 $ exiv2 -pX ~/Downloads/20160911_132121-3.jpg | xmllint --format - | grep prefix <rdf:Description xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-29/" rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b"> <prefix0:LocationCreated> <prefix0:CountryName xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">Canada</prefix0:CountryName> <prefix1:ProvinceState xmlns:prefix1="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">AB</prefix1:ProvinceState> <prefix2:City xmlns:prefix2="http://iptc.org/std/Iptc4xmpExt/2008-02-29/">Alberta</prefix2:City> </prefix0:LocationCreated> 568 rmills@rmillsmbp:~/gnu/exiv2 $
Incidentally, it's possible in exiv2 to extract/edit/insert RAW XML/XMP as follows:
$ exiv2 -pX foo.jpg | xmllint bla bla | exiv2 -iX- foo.jpgI'll run your file through the debugger in the next 24 hours to understand more. If the analysis above leads you to understand more about this issue, please share your thoughts.
Updated by Robin Mills over 4 years ago
I see this is Alberta and not NZ. I've been in Glacier National Park and never crossed into Canada. One day. We were in the Southern Alps of New Zealand in January and thought "Alpine Snow in September. Must be NZ!".
I've run your file in the debugger. It's throwing the exception here in VerifyXPathRoot in XMPCoreImpl.cpp:
// The propName is qualified. Make sure the prefix is legit. Use the associated URI and qualified name. size_t prefixLen = colonPos - propName + 1; // ! Include the colon. VerifySimpleXMLName ( colonPos+1, colonPos+strlen(colonPos) ); XMP_VarString prefix ( propName, prefixLen ); XMP_StringMapPos prefixPos = sNamespacePrefixToURIMap->find ( prefix ); if ( prefixPos == sNamespacePrefixToURIMap->end() ) { XMP_Throw ( "Unknown schema namespace prefix", kXMPErr_BadSchema ); // <--- trouble here } if ( prefix != uriPos->second ) { XMP_Throw ( "Schema namespace URI and prefix mismatch", kXMPErr_BadSchema ); }The value of
prefix = prefix0
. It's trying to parse prefix0:LocationCreated
. Why? I don't know. I think I'll add prefix0,1,2 to the predefined namespaces in XMPsdk and run it in the debugger. I can't do that until Tuesday.
Incidentally, I don't know if XMPsdk DumpFile actually parses the XMP - it might just dump the structure of the file. Similar to exiv2 -pS. However for certain, ReadingXMP does parse XMP. For sure, XMPsdk 201607 is happy with that XMP.
607 rmills@rmillsmbp:~/gnu/xmpsdk/XMP-Toolkit-SDK-CC201607/samples/target/macintosh/intel_64/Release $ ./ReadingXMP ~/Downloads/20160911_132121.jpg /Users/rmills/Downloads/20160911_132121.jpg is opened successfully CreatorTool = G925W8VLU4CPG2 dc:title in Englisuh = dc:title in French = Flash Used = False XMP dumped to XMPDump.txt 608 rmills@rmillsmbp:~/gnu/xmpsdk/XMP-Toolkit-SDK-CC201607/samples/target/macintosh/intel_64/Release $ cat XMPDump.txt Dumping XMPMeta object "" (0x0) xmp: http://ns.adobe.com/xap/1.0/ (0x80000000 : schema) xmp:CreatorTool = "G925W8VLU4CPG2" xmp:ModifyDate = "2016-11-02T14:41:56" xmp:CreateDate = "2016-09-11T13:21:21" Iptc4xmpExt: http://iptc.org/std/Iptc4xmpExt/2008-02-29/ (0x80000000 : schema) Iptc4xmpExt:LocationCreated (0x200 : isArray) [1] (0x100 : isStruct) Iptc4xmpExt:CountryName = "Canada" Iptc4xmpExt:ProvinceState = "AB" Iptc4xmpExt:City = "Alberta" xmpMM: http://ns.adobe.com/xap/1.0/mm/ (0x80000000 : schema) xmpMM:InstanceID = "uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" tiff: http://ns.adobe.com/tiff/1.0/ (0x80000000 : schema) tiff:ImageWidth = "5312" tiff:ImageLength = "2988" tiff:Orientation = "1" tiff:XResolution = "72/1" tiff:YResolution = "72/1" tiff:ResolutionUnit = "2" tiff:YCbCrPositioning = "1" tiff:Make = "samsung" tiff:Model = "SM-G925W8" exif: http://ns.adobe.com/exif/1.0/ (0x80000000 : schema) exif:ColorSpace = "1" exif:PixelXDimension = "5312" exif:PixelYDimension = "2988" exif:ImageUniqueID = "A16LLIC08SM A16LLIL02GM " exif:ExposureTime = "1/386" exif:FNumber = "19/10" exif:ExposureProgram = "2" exif:ShutterSpeedValue = "859/100" exif:ApertureValue = "185/100" exif:BrightnessValue = "685/100" exif:ExposureBiasValue = "0/10" exif:MaxApertureValue = "185/100" exif:MeteringMode = "2" exif:FocalLength = "430/100" exif:ExposureMode = "0" exif:WhiteBalance = "0" exif:FocalLengthIn35mmFilm = "28" exif:SceneCaptureType = "0" exif:DateTimeOriginal = "2016-09-11T13:21:21" exif:ISOSpeedRatings (0x600 : isOrdered isArray) [1] = "40" exif:ExifVersion = "0220" exif:FlashpixVersion = "0100" exif:Flash (0x100 : isStruct) exif:Fired = "False" exif:Return = "0" exif:Mode = "0" exif:Function = "False" exif:RedEyeMode = "False" photoshop: http://ns.adobe.com/photoshop/1.0/ (0x80000000 : schema) photoshop:DateCreated = "2016-09-11T13:21:21" 609 rmills@rmillsmbp:~/gnu/xmpsdk/XMP-Toolkit-SDK-CC201607/samples/target/macintosh/intel_64/Release $
Updated by Robin Mills over 4 years ago
- Status changed from Assigned to Resolved
- % Done changed from 30 to 90
There is no doubt about what is causing this. The version of XMPsdk embedded in Exiv2 enforces the condition that every namespace prefix (such as prefix0 and prefix1) a different URI. I've modified your file with a Hex Editor (HexFiend on the Mac) so that xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-20/
and similar for prefix1 and prefix2.
649 rmills@rmillsmbp:~/Downloads $ exiv2 -pX 20160911_132121.jpg | xmllint --format - <?xml version="1.0"?> <?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?> <x:xmpmeta xmlns:x="adobe:ns:meta/"> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:Description xmlns:xmp="http://ns.adobe.com/xap/1.0/" rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b"> <xmp:CreatorTool>Microsoft Photo Gallery 16.4.3528.331</xmp:CreatorTool> </rdf:Description> <rdf:Description xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-20/" rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b"> <prefix0:LocationCreated> <rdf:Bag xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:li> <rdf:Description xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <prefix0:CountryName xmlns:prefix0="http://iptc.org/std/Iptc4xmpExt/2008-02-20/">Canada</prefix0:CountryName> <prefix1:ProvinceState xmlns:prefix1="http://iptc.org/std/Iptc4xmpExt/2008-02-21/">AB</prefix1:ProvinceState> <prefix2:City xmlns:prefix2="http://iptc.org/std/Iptc4xmpExt/2008-02-22/">Alberta</prefix2:City> </rdf:Description> </rdf:li> </rdf:Bag> </prefix0:LocationCreated> </rdf:Description> </rdf:RDF> </x:xmpmeta> <?xpacket end='w'?>
650 rmills@rmillsmbp:~/Downloads $ exiv2 -px 20160911_132121.jpg Xmp.xmp.CreatorTool XmpText 37 Microsoft Photo Gallery 16.4.3528.331 Xmp.prefix0.LocationCreated XmpText 0 type="Bag" Xmp.prefix0.LocationCreated[1] XmpText 0 type="Struct" Xmp.prefix0.LocationCreated[1]/prefix0:CountryName XmpText 6 Canada Xmp.prefix0.LocationCreated[1]/prefix1:ProvinceState XmpText 2 AB Xmp.prefix0.LocationCreated[1]/prefix2:City XmpText 7 Alberta Xmp.xmpMM.InstanceID XmpText 41 uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b 651 rmills@rmillsmbp:~/Downloads $I've seen something similar recently #1276 in which the exception is thrown in RDFParse.cpp hence my earlier guess about the cause of the fault. It's possible that the unique URI condition was specified in an earlier edition of the XMP Spec and subsequently relaxed.
I'm very reluctant to change code in the embedded copy of XMPsdk in Exiv2 because:
1) It has worked well for years with very few issues.
2) This issue has been in Exiv2 for years and has not been introduced by changes for v0.26.
3) We have a project already in progress to update to the latest XMPsdk in Exiv2 v0.27 (#941)
4) I'm in the last few days of Exiv2 v0.26 and very reluctant to change XMPsdk code as the consequences are unpredictable and unknowable.
I've associated this issue with #941 so that when #941 is serviced, this issue will be easily visible. Based on my investigation into XMPdump, I expect this issue to disappear when we upgrade XMPsdk for v0.27.
I feel I've pursued this matter as far as I can at the moment and this issue should be closed. Do you agree?
Updated by Wil Cowb over 4 years ago
Hello Robin,
Thank you very much for taking the time to investigate this.
The mountains on the picture are Canadian Rockies by Calgary, Alberta, Canada.
Yes I agree 100% with your plan. There is no need to introduce any major changes in Exiv2 code right now.
The only reason I even noticed that is because Exiv2 actually crashes on Linux Mint instead of throwing an C++ exception.
Looks like there is some issue with libc++ in Mint because after I moved to a different distro (Manjaro 17) Exiv2 does not crash anymore.
On a separate note, Do you think if I got rid of these fields the issue will dissapear?
I stopped using Microsoft Photo Gallery because it will be discontiuned soon so can delete all the mess it made and forget about it.
[XMP-rdf] About : uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b
[XMP-xmp] CreatorTool : Microsoft Photo Gallery 16.4.3528.331
[XMP-iptcExt] LocationCreatedCountryName : Canada
[XMP-iptcExt] LocationCreatedProvinceState : AB
[XMP-iptcExt] LocationCreatedCity : Alberta
Updated by Robin Mills over 4 years ago
- Status changed from Resolved to Closed
- % Done changed from 90 to 100
I'm very surprised to hear that Exiv2 is crashing on Mint. Can you raise a report about that and I'll create a Mint VM and investigate (after v0.26 ships). Do you know if it's crashing generally, or only on this file? Our buildserver uses Kubuntu and I have VMs for Fedora and FreeBSD which were created to investigate platform issues raised by users.
The quick fix is to delete the XMP from your images with:
$ exiv2 -dx image ...As always:
1 be sure to backup your images before you try this.
2 This will throw out the XMP in your image. If somebody (apart from Microsoft Photo Gallery) has been editing the XMP, you'll also throw away anything they have stored in XMP.
Thanks for getting back to me about. I'm going to close this one.
Updated by Wil Cowb over 4 years ago
I just found out that it crashes on Manjaro too.
I narrowed the issue down and looks like it only crashes when:
1. You have the following packages installed:
gdb (7.12.1-1)
gdb-common (7.12.1-1)
2. Run application in "debug" mode, e.g. if I use digiKam I would run 'digikam.appimage debug' in terminal.
Too bad I actually moved from Mint to Manjaro because of this...
Updated by Robin Mills over 4 years ago
Is it crashing from the exiv2 command-line, or from digiKam? Gilles has reported numerous crashes which turned out to be build issues with digiKam.
I'm really only tooled up to deal with issues which can be reproduced from the command-line using the exiv2 samples - especially exiv2(.exe). The primary development platform in MacOS-X. The code is very portable. Platform dependent errors are rather unusual.
Updated by Wil Cowb over 4 years ago
It is crashing from digiKam. Gilles is aware of the issue now. Initially he thought Exiv2 is the one not creating C++ exceptions but now he knows that the issue is somewhere else.
I thought the issue was with lib++ library in Linux Mint so I tried another Linux distribution but it turned out that the debugger is the one causing problems.
I just deleted it and keep working on my images. Got too many of them piled up waiting to be processed.
Updated by Robin Mills over 4 years ago
Right. Thanks for the update. digiKam is a big beast and I've no idea how to use it, built it or debug it. I try to reduce complexity and focus purely on Exiv2 code. There's 100k lines of code of which I probably wrote about 10-20k and have modified another 10-20k. Plenty of puzzles to keep me busy.
You work on your images, I'll work on Exiv2 v0.26 and Gilles can worry about digiKam. We'll all survive - well we've always survived until now!
Updated by Gilles Caulier over 4 years ago
The C++ exception crash is NOT a compilation issue, for digiKam or Exiv2. I'm sure, else we will receive a lots of report about...
But, i suspect a settings somewhere on Linux env. to turn on/off C++ exception at run time. I suspect that your Linux distro do something in background when you install GDB.
This is a weird side effect, and for me it's a bug in Linux Distro. The C++ exception handling must be always enabled to prevent unwanted crash. Here i tested under Suse, Mageia, Centos, all work perfectly with GDB (or not)...