Feature #1074

ICC Profile in APP2 segment.

Added by Tim Zaman about 2 years ago. Updated 10 months ago.

Status:ClosedStart date:07 May 2015
Priority:NormalDue date:
Assignee:Robin Mills% Done:

100%

Category:metadataEstimated time:50.00 hours
Target version:0.26

Description

See thread on forum [[http://dev.exiv2.org/boards/3/topics/2059?r=2065#message-2065]].

Currently, exiv2 cannot store an ICC profile (custom) in a way that is readable by most viewers (f.e. previewers or Adobe). They want the ICC profile to be in the APP2 segment. As a feature, i would like to see this possibility in exiv2. To my opinion, exiv2 would be the perfect c++ metadata/xmp/iptc/exif page in zie world.

Maybe nice to pursue for 0.27?

Reagan.jpg (713 KB) Robin Mills, 26 Aug 2015 14:35

Reagan.tiff (8.23 MB) Robin Mills, 26 Aug 2015 16:51

scan.png (34.1 KB) Max Pozdeev, 17 Mar 2016 17:59


Related issues

Related to Exiv2 - Feature #756: Access to ICC Profile (TAG: 0x8773) data in Exif as unint... Closed 04 Feb 2011
Related to Exiv2 - Patch #676: Patch for reading ICC color profiles Closed 26 Jan 2010
Related to Exiv2 - Bug #1126: Extracting Exif from PNG in Exiv2-0.25 Closed 08 Oct 2015
Related to Exiv2 - Feature #1057: Implement target/modifier - (stdin/stdout) for exiv2 opti... Closed 17 Apr 2015

Associated revisions

Revision 3898
Added by Robin Mills almost 2 years ago

#1074. Implemented exiv2 options -eC and -pC for JPEG.

Revision 4166
Added by Robin Mills over 1 year ago

#1108 and #1074 Correction to r4165 to fix MSVC build breaker and to document: exiv2 -eC (extract ICC profile).

Revision 4168
Added by Robin Mills over 1 year ago

#1108 and #1074 -pC and -pR (print Color Profile, print Recursively) now work on png.

Revision 4169
Added by Robin Mills over 1 year ago

#1074 -pC now works on tiff files. Added test convenience targets to Makefile

Revision 4170
Added by Robin Mills over 1 year ago

#1074 Correction to r4169. Changes to config/Makefile.in to match Makefile

Revision 4171
Added by Robin Mills over 1 year ago

#1108 and #1074 Correction to r4168 to fix MSVC build breaker.

Revision 4172
Added by Robin Mills over 1 year ago

#1074 Correction to r4169 to fix test suite for platform/wc dependencies on Linux.

Revision 4173
Added by Robin Mills over 1 year ago

#1074 Correction to r4169. Use cksum (better than wc) to test extracted profile correctness. wc is platform dependant.

Revision 4193
Added by Robin Mills over 1 year ago

#1074 test files

Revision 4229
Added by Robin Mills over 1 year ago

#1074. Thanks to Max for reporting this and providing the test file.

Revision 4230
Added by Robin Mills over 1 year ago

#1074 Added Max's file to the test suite.

Revision 4234
Added by Robin Mills over 1 year ago

#1074 Fixed 'overwrite?' prompt in command: exiv2 -eC image

Revision 4241
Added by Robin Mills over 1 year ago

#1074 #1108 Added ICC profile to test/data/Reagan.jpg

Revision 4242
Added by Robin Mills over 1 year ago

#1074. jpgimage.cpp ICC support (Work in Progress)

Revision 4455
Added by Robin Mills 11 months ago

#1074 Work in progress for ICC/jpg - changes to readMetadata() and printStructure()

Revision 4456
Added by Robin Mills 11 months ago

#1074 Correction to r4455 where in updated test/bugfixes-test.sh instead of test/data/bugfixes-test.out

Revision 4457
Added by Robin Mills 11 months ago

#1074 ICC support in jpgimage.cpp. Add test/icc-test.sh

Revision 4458
Added by Robin Mills 11 months ago

#1074 Correction to r4457 to fix msvc build-breaker.

Revision 4461
Added by Robin Mills 11 months ago

#1220 and #1074 Work in progress. Improved -pS for JPEG. #1220 is due to a bug in writing the ICC profile in jpgimage.cpp/doWriteMetadata()

Revision 4469
Added by Robin Mills 11 months ago

#1074 Correction to r4466 to fix MinGW build breaker on include <sys/select.h>

Revision 4470
Added by Robin Mills 11 months ago

#1074 and #1057 Fixing build breakers on Cygwin and msvc MinGW to be further investigaged. add checkSum to test functions

Revision 4471
Added by Robin Mills 11 months ago

#1074 Omitted from previous commit.

Revision 4479
Added by Robin Mills 11 months ago

#1074 Added ICC support to tiffimage.writeMetadata(). pngimage.readMetadata() is not complete.

Revision 4481
Added by Robin Mills 10 months ago

#1074. Rollback r4479 which has inflicted a lot harm on the test harness.

Revision 4485
Added by Robin Mills 10 months ago

#1074 Work in progress. ICC Jpeg/Png/Tiff exiv2 -eC foo.xxx writes good foo.icc profiles. icc-test.sh is broken and to be investigated.

Revision 4490
Added by Robin Mills 10 months ago

#1074. Fix bug in jpgimage.cpp/readMetadata() concerning ICC profile

Revision 4493
Added by Robin Mills 10 months ago

#1074 Work in progress. eC -pC -eC work for JPG/PNG/TIFF/WEBP. pngimage::doWriteMetadata() does not update iCCP chunk.

Revision 4495
Added by Robin Mills 10 months ago

#1074 Work in progress. fixed pngimage.cpp doWriteMetadata().

Revision 4499
Added by Robin Mills 10 months ago

#1074 Tests for webp, png, tiff and jpg

Revision 4540
Added by Robin Mills 10 months ago

#1074 Fix bug in TiffImage printStructure() for large icc profiles.

Revision 4541
Added by Robin Mills 10 months ago

#1074 Correction to r4540. Rollback changes to actions.cpp which should not have been submitted. Resubmitted test/data/icc-test.out

History

#1 Updated by Robin Mills about 2 years ago

  • Category changed from not-a-bug to metadata
  • Priority changed from Low to Normal

Thanks, Tim. I think Exiv2 is already the best metadata library in the world and feature would make it best++.

Can we clarify the scope of this. Remember, the more you want, the less likely you are to get it AND the longer it will take!

  1. Read AND Write ICC Profiles?
  2. Most formats: JPEG, TIFF, PNG (raw, dng etc)?
  3. Can XMP sidecars deal with color profiles?
  4. Do you want to treat a Profile as a metadata container?

I've set the target to v0.26 because the other choice is 1.0. This is unlikely to be ready for v0.26 due to contention with existing commitments.

#2 Updated by Tim Zaman about 2 years ago

Reading the APP2 profile data is usually not necessary, as it is often also, duplicated, (in my experience) in the normal section. Formats, most common where one would use this, TIFF and JPG. If you really care about color you often use JPG and TIFF.
So writing for JPEG/TIFF would be a good start. I think profile could indeed be seen as a separate container im, should be straightforward without many bells and whistles. In my experience you can almost copy the .icc file to the header verbatim.

#3 Updated by Robin Mills about 2 years ago

Tim

To keep this simple, I'd like to limit this to JPEG, TIFF and PNG:

1) Extract ICC profile
add the option -eC extract color profile. So foo.jpg -> foo.icc
add the option -pC to write the color profile to stdout
I think the code that handles -pS (print Structure) and -pX (print XMP) can do this will minor modification

2) Replace/embed the profile:
add the option -iC import color profile. So foo.icc -> foo.{tiff|jpg|png}

I'm not offering to delve inside the data of the profile. There is all manner of color magic (LUTs and what not), however I am only offering treat the profile as a blob of binary data.

#4 Updated by Tim Zaman about 2 years ago

Yeah making color profiles is as much art as it is science. In terms of options in a command line program, i was more interested in the c++ api, or is the cli derived from this?

#5 Updated by Robin Mills about 2 years ago

I'll have to think about the API. At the moment, the interface on -pS and -pX is in image.hpp:

image.hpp:    typedef enum { kpsNone, kpsBasic, kpsXMP } printStructureOption_e ;
image.hpp:    virtual void printStructure(std::ostream& out,printStructureOption_e option=kpsNone);
So you open the image and call printStructure with an option. printStructureOption_e can easily extended with kpsProfile.

I'll have to think about the API to embed/replacing an object in the image.

#6 Updated by Robin Mills about 2 years ago

  • Assignee deleted (Robin Mills)

#7 Updated by Robin Mills almost 2 years ago

  • File Reagan.jpg added
  • Status changed from New to Assigned
  • Assignee set to Robin Mills
  • % Done changed from 0 to 30
  • Estimated time set to 20.00

I'm going to implement the following for jpeg/png/tiff files. Of course, only jpeg have an APP2 block, however I'll also extract the profile from PNG and TIFF. All other file formats will throw an exception.

exiv2 -pC file # print profile to std::out
exiv2 -eC file # extract profile to file.icc
exiv2 -iC profile file # replace/add profile to an image

Regrettably the ICC sample application iccDumpProfile does not read from stdin and can't be used as a pipe.

794 rmills@rmillsmbp:~ $ exiv2 --verbose -pS http://dev.exiv2.org/attachments/download/820/Reagan.jpg
File 1/1: http://dev.exiv2.org/attachments/download/820/Reagan.jpg
STRUCTURE OF JPEG FILE: http://dev.exiv2.org/attachments/download/820/Reagan.jpg
 address | marker     | length  | data
       2 | 0xd8 SOI   |       0 
       4 | 0xe0 APP0  |      16 | JFIF.....,.,......Exif..MM.*....
      22 | 0xe1 APP1  |    5642 | Exif..MM.*......................
    5666 | 0xe1 APP1  |    3672 | http://ns.adobe.com/xap/1.0/.<?x
    9340 | 0xe2 APP2  |    3160 | ICC_PROFILE......HLino....mntrRG
   12502 | 0xed APP13 |   13986 | Photoshop 3.0.8BIM.............
   26490 | 0xee APP14 |      14 | Adobe.d@........................
   26506 | 0xdb DQT   |     132 
   26640 | 0xc0 SOF0  |      17 
   26659 | 0xdd DRI   |       4 
   26665 | 0xc4 DHT   |     418 
   27085 | 0xda SOS   |      12 
795 rmills@rmillsmbp:~ $ exiv2 --verbose -pC http://dev.exiv2.org/attachments/download/820/Reagan.jpg > Reagan.iccProfile
796 rmills@rmillsmbp:~ $ exiv2 --verbose -eC http://dev.exiv2.org/attachments/download/820/Reagan.jpg 
File 1/1: http://dev.exiv2.org/attachments/download/820/Reagan.jpg
Writing iccProfile: Reagan.icc
797 rmills@rmillsmbp:~ $ ls -alt Reag*
-rw-r--r--  1 rmills  staff     9 26 Aug 17:40 Reagan.exv
-rw-r--r--  1 rmills  staff  3144 26 Aug 17:40 Reagan.icc
-rw-r--r--  1 rmills  staff  3211 26 Aug 17:40 Reagan.iccProfile
798 rmills@rmillsmbp:~ $ diff Reagan.icc Reagan.iccProfile 
Binary files Reagan.icc and Reagan.iccProfile differ
799 rmills@rmillsmbp:~ $ iccDumpProfile Reagan.icc
Profile:          'Reagan.icc'
Profile ID:       Profile ID not calculated.
Size:             3144(0xc48) bytes

Header
------
Attributes:       Reflective | Glossy
Cmm:              Unknown 'Lino' = 4C696E6F
Creation Date:    2/9/1998  06:49:00
Creator:          'HP  ' = 48502020
Data Color Space: RgbData
Flags             EmbeddedProfileFalse | UseAnywhere
PCS Color Space:  XYZData
Platform:         Microsoft
Rendering Intent: Perceptual
Type:             DisplayClass
Version:          2.10
Illuminant:       X=0.9642, Y=1.0000, Z=0.8249

Profile Tags
------------
                      Tag    ID      Offset        Size
                     ----  ------    ------        ----
             copyrightTag  'cprt'       336          51
    profileDescriptionTag  'desc'       388         108
       mediaWhitePointTag  'wtpt'       496          20
       mediaBlackPointTag  'bkpt'       516          20
           redColorantTag  'rXYZ'       536          20
         greenColorantTag  'gXYZ'       556          20
          blueColorantTag  'bXYZ'       576          20
         deviceMfgDescTag  'dmnd'       596         112
       deviceModelDescTag  'dmdd'       708         136
       viewingCondDescTag  'vued'       844         134
     viewingConditionsTag  'view'       980          36
             luminanceTag  'lumi'      1016          20
           measurementTag  'meas'      1036          36
            technologyTag  'tech'      1072          12
                redTRCTag  'rTRC'      1084        2060
              greenTRCTag  'gTRC'      1084        2060
               blueTRCTag  'bTRC'      1084        2060
800 rmills@rmillsmbp:~ $ 

#8 Updated by Robin Mills almost 2 years ago

Adding Reagan.tiff with an embedded ICC profile.

Neither PhotoShop Elements nor Apple/Preview appear to be able to embed ICC profiles in PNG files.

#9 Updated by Tim Zaman almost 2 years ago

Redmine is not the place for compliments, but: awesome! <3 Robin.

#10 Updated by Robin Mills over 1 year ago

  • % Done changed from 30 to 50

r4165 I've done more work on option -pC for PNG files. The location of the profile is documented here: http://www.w3.org/TR/PNG/#11iCCP I've can easily find the profile - however zlib refuses to decompress it! How odd. The code is submitted.

I've put in effort on -pR (recursive dump), so I now have the code available to locate profiles which are embedded in tiff-encoded records which can appear in tiff files, Exif datablocks, ExifTag records, or tiff-encoded MakerNotes.

Exiv2 man page updated to include -pC (print ICC profile to stdout) and -eC (extract ICC profile to file.icc)

#11 Updated by Robin Mills over 1 year ago

State of play:
Done:
-pC extract the ICC profile from png, tiff and jpeg

TODO to complete ICC Profile support for v0.26:
1 Extract ICC profile to .exv file (-eC) (Import is done, as a .exv file is a JPEG)
2 Extract ICC profile from foo.jpg to foo.icc (-ec)
3 Import ICC profile -iC
4 Update metacopy (and other samples) to respect ICC profiles

There is a related issue #1057 to add tgt modifier - for stdin/stdout to -etgt and -itgt. This will enable a useful work flow to pipe an ICC profile from one image to another:

$ exiv2 -eC- foo.tiff | exiv2 -iC- foo.jpg foo.png

#12 Updated by Max Pozdeev over 1 year ago

Hi Robin,
I found an png image where icc profile is not displayed by exiv2. Image was produced by Apple Image Capture from scanner in grayscale.
Here the structure. Seems the profile is present.

STRUCTURE OF PNG FILE: /Pictures/scan.png
 address | index | chunk_type |  length | data
       8 |     0 |       IHDR |      13 | ............
      33 |     1 |       iCCP |    3294 | ICC Profile..X..W.XS...[...z..:.
    3339 |     2 |       pHYs |       9 | .........
    3360 |     3 |       iTXt |     596 | XML:com.adobe.xmp.....<x:xmpmeta
    3968 |     4 |       IDAT |   16384 | x.....%Y.......UY.==.0.p...w..".
   20364 |     5 |       IDAT |   14548 | y.oCU`......C7.....9.1N#......1
   34924 |     6 |       IEND |       0 | 

#13 Updated by Robin Mills over 1 year ago

Ah, that's very helpful. I'll have a look at that. Thanks for letting me know.

#14 Updated by Robin Mills over 1 year ago

Fixed. Good find, Max. Fix submitted r4229

I've modified the code to do case-independant check on the "ICC...".

$ exiv2 -pR ~/Downloads/scan.png 
STRUCTURE OF PNG FILE: /Users/rmills/Downloads/scan.png
 address | index | chunk_type |  length | data
       8 |     0 |       IHDR |      13 | ............
      33 |     1 |       iCCP |    3294 | ICC Profile..X..W.XS...[...z..:.
    3339 |     2 |       pHYs |       9 | .........
    3360 |     3 |       iTXt |     596 | XML:com.adobe.xmp.....<x:xmpmeta
    3968 |     4 |       IDAT |   16384 | x.....%Y.......UY.==.0.p...w..".
   20364 |     5 |       IDAT |   14548 | y.oCU`......C7.....9.1N#......1
   34924 |     6 |       IEND |       0 | 
$ exiv2 -pC ~/Downloads/scan.png > /tmp/scan.icc ; ls -alt /tmp/scan.icc 
-rw-r--r--  1 rmills  wheel  6562 17 Mar 22:29 /tmp/scan.icc
$ iccDumpProfile /tmp/scan.icc
Profile:          '/tmp/scan.icc'
Profile ID:       Profile ID not calculated.
Size:             4456(0x1168) bytes

Header
------
Attributes:       Reflective | Glossy
Cmm:              Apple
Creation Date:    8/23/2012  15:46:15
Creator:          'appl' = 6170706C
Data Color Space: GrayData
Flags             EmbeddedProfileFalse | UseAnywhere
PCS Color Space:  XYZData
Platform:         Macintosh
Rendering Intent: Perceptual
Type:             DisplayClass
Version:          2.00
Illuminant:       X=0.9642, Y=1.0000, Z=0.8249

Profile Tags
------------
                      Tag    ID      Offset        Size
                     ----  ------    ------        ----
    profileDescriptionTag  'desc'       192         121
Unknown 'dscm' = 6473636D  'dscm'       316        2024
             copyrightTag  'cprt'      2340          35
       mediaWhitePointTag  'wtpt'      2376          20
               grayTRCTag  'kTRC'      2396        2060
$

#15 Updated by Robin Mills over 1 year ago

I've updated the test suite to include Max's file. r4230. I assume you are happy to donate your file, please say otherwise.

#16 Updated by Max Pozdeev over 1 year ago

Robin, thank you for the patch. The file was made specially for tests, use as you wish.

#17 Updated by Robin Mills over 1 year ago

  • % Done changed from 50 to 70

#18 Updated by Robin Mills 11 months ago

  • % Done changed from 70 to 80

Mostly done. I'll have to test this, update the test suite appropriately and fix things that have still to be implemented.

#19 Updated by Robin Mills 11 months ago

  • % Done changed from 80 to 60
  • Estimated time changed from 20.00 to 60.00

#20 Updated by Robin Mills 11 months ago

  • % Done changed from 60 to 70
  • Estimated time changed from 60.00 to 40.00

r4462 Fixed #1220 (JPEG/ICC corrupting files). r4463 implemented -iC-.

Work in progress. Still have to deal with ICC in PNG and Tiff images. Reduced time estimate to 40 hours.

#21 Updated by Robin Mills 11 months ago

  • % Done changed from 70 to 80

#22 Updated by Robin Mills 11 months ago

  • % Done changed from 80 to 90

r4479 Fix submitted for tiffImage::doWriteMetadata().

image::findIccProfile() is closed the FILE* when called from pngImage::readMetadata().

#23 Updated by Robin Mills 10 months ago

  • % Done changed from 90 to 80
  • Estimated time changed from 40.00 to 50.00

r4492 Many fixes.

Thanks to Phil for some advice about the jpeg/embedding. http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,7586.0.html

And Thanks to Ben for using GIMP to to create test/data/ReaganLargeXxx.xxx files.

Stuff remaining:
1) $ exiv2 -iC foo.png (in pngimage.cpp/doWriteMetadata())
2) tests for tiff/jpeg/png/webp in test/icc-test.sh
3) $ exiv2 -iC- to read profile from stdin is doubtful
4) $ exiv2 -eC- to write profile to stdout is doubtful

#24 Updated by Robin Mills 10 months ago

  • % Done changed from 80 to 90

r4495 Fix writing PNG files. Into the end game. More testing and this will be finished.

#25 Updated by Robin Mills 10 months ago

  • Status changed from Assigned to Closed
  • % Done changed from 90 to 100

r4499 Tests for JPG/WEBP/TIFF/PNG

Also available in: Atom PDF

Redmine Appliance - Powered by TurnKey Linux