Feature #1232

Support JPEGs encoding and decoding in which the Exif data exceeds 64k bytes

Added by Robin Mills 5 months ago. Updated 5 months ago.

Status:NewStart date:26 Sep 2016
Priority:NormalDue date:
Assignee:-% Done:


Target version:1.0

XmpPart3-2016-page13.png (142 KB) Robin Mills, 26 Sep 2016 19:44


#1 Updated by Robin Mills 5 months ago

Adobe published a new Edition of the XMPsdk on 30 July 2016. The latest version of this document XMP-Toolkit-SDK-CC201607/docs/XMPSpecificationPart3.pdf contains this statement on page 13. This statement was not in the 2010 Edition of that document.

The JPEG standard does not prescribe ordering among APPn segments, but some related standards do. For example, Exif requires that Exif APP1 segment be immediately after the SOI. Also, some applications improperly assume that the segments are in a particular order. For compatibility, it is best to put the Exif APP1 first, the XMP APP1 next, the PSIR APP13 next, followed by all other marker segments.

NOTE If the size of the Exif APP1 marker or the PSIR APP13 marker exceeds 64KB, the marker is split into multiple blocks of 64KB size each.

If this is true, we can indeed store huge blobs of Exif metadata. If the metadata exceeds 64k, we can simply divide it into multiple segments. However, I would want to see a reliable example of such a file. By "reliable", I mean it has a blue-chip pedigree (such as PhotoShop). And I'd like to see a specification which is focused on this matter. The statement in this spec doesn't feel definitive and doesn't address APP1 continuation block signatures.

A typical JPEG structure is as follow:

696 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pS test/data/Reagan.jpg 
STRUCTURE OF JPEG FILE: test/data/Reagan.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe1 APP1  |    5718 | Exif..MM.*......................     <---- This is the Exif data (Tiff Encoded)
    5722 | 0xffed APP13 |    3038 | Photoshop 3.0.8BIM..........Z...
    8762 | 0xffe1 APP1  |    5329 | http://ns.adobe.com/xap/1.0/.<?x
   14093 | 0xffe2 APP2  |     576 | ICC_PROFILE......0ADBE....mntrRG chunk 1/1
   14671 | 0xffee APP14 |      14 | Adobe.d@......
   14687 | 0xffdb DQT   |     132 
   14821 | 0xffc0 SOF0  |      17 
   14840 | 0xffdd DRI   |       4 
   14846 | 0xffc4 DHT   |     418 
   15266 | 0xffda SOS  
697 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
I think Adobe are saying that if you follow an APP1 segment with one or more APP1 segments of exactly 64k bytes you should consider it to be a "segmented" continuation of its predecessor. That seems reasonable to me. However it's quite different from the "chunk" mechanism used to allow ICC profiles to exceed a single APP2 segment. Additionally, APP1 segments begin with a signature such as "Exif\0\0" to identify them to readers.

I've hunted through Phil (of ExifTool fames) repository of 7000 JPG files. Not one has Exif data encoded in this way.

AGFA have files with multiple segments of length 65535. However they are not APP1 segment. Here's the evidence:

702 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pS /mmHD/Users/rmills/Jenkins/testfiles/Phils/Agfa/AgfaOPTIMA1338mT.jpg 
STRUCTURE OF JPEG FILE: /mmHD/Users/rmills/Jenkins/testfiles/Phils/Agfa/AgfaOPTIMA1338mT.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe1 APP1  |   46459 | Exif..II*......................
   46463 | 0xffe3 APP3  |   65535 | ................................   <--- what is this?
  112000 | 0xffe4 APP4  |   65535 | ..Hc..w .8<...z..M.77.h...{.....   <--- or this?
  177537 | 0xffe5 APP5  |    7243 | .U......K..u=).pl.W.F...B.$.3...
  184782 | 0xffdb DQT   |     132 
  184916 | 0xffc0 SOF0  |      17 
  184935 | 0xffc4 DHT   |      75 
  185012 | 0xffda SOS  
703 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
I'm willing to implement "Exif data exceeds 64k" feature in the future provided we have a more substantial specification and good pedigree test files.

Also available in: Atom PDF

Redmine Appliance - Powered by TurnKey Linux