Project

General

Profile

Support #1331

exiv2 fails to delete bulk of metadata in jpeg

Added by Denis-Carl Robidoux almost 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Robin Mills
Category:
not-a-bug
Target version:
0.27
Start date:
19 Dec 2017
Due date:
% Done:

100%

Estimated time:
2.00 h

Description

I got this file which I made available here for you to test:
http://www.deniscarl.com/overweightGeorgeMichael.jpg
It's a 3 Megabytes jpeg with a lot of metadata from some obscure photoshop session.
If I delete the metadata with exiftool

exiftool -all="" overweightGeorgeMichael.jpg

it gives out a jpeg of 564K

But if I try to do it with exiv2:

exiv2 rm overweightGeorgeMichael.jpg

it gives out a jpeg of 2.8Megs

That a lot of undeleted metadata.

Related issues

Related to Exiv2 - Feature #941: Upgrade xmpsdk source to Adobe's current versionClosed27 Dec 2013

Actions

History

#1

Updated by Robin Mills almost 4 years ago

  • Tracker changed from Bug to Support
  • Category set to metadata
  • Status changed from New to Assigned
  • Assignee set to Robin Mills
  • Target version set to 0.27
  • % Done changed from 0 to 10
  • Estimated time set to 2.00 h

Thanks. I'll investigate and give up an update.

#2

Updated by Robin Mills almost 4 years ago

I know what's the matter, however I don't have a fix. The obvious work-around is to use exiftool.

Your explanation It's a 3 Megabytes jpeg with a lot of metadata from some obscure photoshop session seems to be correct. PhotoShop is using "Extended XMP" which isn't currently supported by Exiv2.

If we examine the structure of your file, I see: 

592 rmills@rmillsmbp:~/Downloads $ curl -O http://www.deniscarl.com/overweightGeorgeMichael.jpg ; ls -alt over*
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2986k  100 2986k    0     0  1838k      0  0:00:01  0:00:01 --:--:-- 1842k
-rw-r--r--+ 1 rmills  staff  3058367 19 Dec 17:08 overweightGeorgeMichael.jpg
593 rmills@rmillsmbp:~/Downloads $ exiv2 -pS overweightGeorgeMichael.jpg 
STRUCTURE OF JPEG FILE: overweightGeorgeMichael.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe0 APP0  |      16 | JFIF.....H.H....
      20 | 0xffed APP13 |    8292 | Photoshop 3.0.8BIM.......'..Z...
    8314 | 0xffe1 APP1  |    2657 | Exif..MM.*...............V......
   10973 | 0xffe2 APP2  |    3160 | ICC_PROFILE......HLino....mntrRG chunk 1/1
   14135 | 0xffe1 APP1  |   65535 | http://ns.adobe.com/xap/1.0/.<?x
   79672 | 0xffe1 APP1  |   65535 | CC7DFB7A8ED5E120CA9</rdf:li> <rd
...
 2439004 | 0xffe1 APP1  |   42980 | 19F46DDB1CA0F98ED</rdf:li> <rdf:
 2481986 | 0xffdb DQT   |      67 
 2482055 | 0xffdb DQT   |      67 
 2482124 | 0xffc0 SOF0  |      17 
 2482143 | 0xffc4 DHT   |      31 
 2482176 | 0xffc4 DHT   |      79 
 2482257 | 0xffc4 DHT   |      20 
 2482279 | 0xffc4 DHT   |      20 
 2482301 | 0xffda SOS  
594 rmills@rmillsmbp:~/Downloads $
A couple of years ago, I added command options -pX (print XMP) and -dX (delete XMP) which understand Extended XMP. Neither option is effective on your file. So, I'm wondering if there's something strange about how the XMP has been added by PhotoShop. I have stepped the code and the "XMP Extended Metadata" flag isn't set. I think PhotoShop is using a method to extend the XMP which isn't supported by Exiv2.

There are about 40 APP1 segments of 64k. That's about 2.6mb. That's the data that is not being deleted.

I think the fix is to see if building with the latest XMPsdk fixes this. Updating Exiv2 to build with XMPsdk (2016,2014 or 2013) is work-in-progress at the moment.

#3

Updated by Robin Mills almost 4 years ago

  • Category changed from metadata to not-a-bug
  • % Done changed from 10 to 30
  • Estimated time changed from 2.00 h to 6.00 h

I've made another discovery about this. I thought "Adobe distribute DumpFile" as part of the XMPsdk. Let's try that". Interesting:

628 rmills@rmillsmbp:~/gnu/xmpsdk/XMP-Toolkit-SDK-CC201607 $ samples/target/macintosh/intel_64/Release/DumpFile ~/Downloads/over*.jpg 
Abort trap: 6
629 rmills@rmillsmbp:~/gnu/xmpsdk/XMP-Toolkit-SDK-CC201607 $

Here's the traceback:

0x104301000 - 0x10433cfff +DumpFile (0) ... /Users/USER/*/DumpFile
0x10435c000 - 0x104497fff +com.xmp.XMPCore (XMP Core 5.6.0 - 0) ... /Users/USER/*/XMPCore.framework/Versions/A/XMPCore
0x1045dc000 - 0x104711ff7 +com.xmp.XMPFiles (XMP Files 5.7.0 - 0) ... /Users/USER/*/XMPFiles.framework/Versions/A/XMPFiles
...
Adobe is crashing in XMPCore when called by XMPFiles. Conclusion: Your file has illegal XMP.

It's not impossible to do something about this. About 3 years ago I added option -dI (delete everything that appears to be IPTC). There was an existing option -di to delete IPTC metadata. According to the spec, there should only be one IPTC segment. I added -dI to help a user with files containing multiple IPTC segments.

I'm willing to consider extending the scope of -dX to say delete everything that appears to be XMP. I'm rather overloaded at the moment and don't want to be side-tracked into this issue. I will consider it next year when time permits.

#4

Updated by Robin Mills almost 4 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 30 to 100
  • Estimated time changed from 6.00 h to 2.00 h

Also available in: Atom PDF