Project

General

Profile

Bug #1182

Exiv2 is unable to update any Exif.SubImageN.xxx tag such as Exif.SubImage1.DefaultScale in a DNG

Added by Robin Mills over 5 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Robin Mills
Category:
tiff parser
Target version:
0.26
Start date:
02 May 2016
Due date:
% Done:

100%

Estimated time:
10.00 h

Description

See discussion/test files: http://dev.exiv2.org/boards/3/topics/2546

Files

exiv2 (2.26 MB) exiv2 Robin Mills, 07 May 2016 10:48
May8InEngland.jpg (393 KB) May8InEngland.jpg Robin Mills, 08 May 2016 14:57

Related issues

Related to Exiv2 - Feature #668: "TIFF-safe" setExifData variantClosed03 Jan 2010

Actions
Related to Exiv2 - Feature #992: Better raw file support and testAssigned18 Sep 2014

Actions

History

#1

Updated by Robin Mills over 5 years ago

  • Assignee changed from Robin Mills to Andreas Huggel

Andreas

Can you look at this for me, please? It's down inside the tiffvisitor code. exiv2 does rewrite the dng, however the changes are not in the file. For example: 

$ curl -O http://dev.exiv2.org/attachments/download/992/M28-1507_1_2016-04-25_0001_C0000_000001.dng
$ exiv2 -pa --grep DefaultScale/i M28-1507_1_2016-04-25_0001_C0000_000001.dng 
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
$ exiv2 --verbose -M'set Exif.SubImage1.DefaultScale 3/1' M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "3/1" (Rational)
$ exiv2 -pa --grep DefaultScale/i M28-1507_1_2016-04-25_0001_C0000_000001.dng 
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
If you don't have time to investigate, please assign it to me. I'll probably defer it for a future release.

#2

Updated by Robin Mills over 5 years ago

  • % Done changed from 20 to 60
  • Estimated time changed from 5.00 h to 10.00 h

I've spent most of a day looking at this. Gosh, it's an amazing trick to write the SubImageN tags in-line with the other metadata. There is something from the original file which 'eclipses' TiffEncoder::exifData_ and causes TiffSubIfd to encode the data from the original file into the output destination file. There's a very complicated network of classes involved in this conspiracy:

  1. TiffParserWorker
  2. TiffCopier
  3. TiffEncoder
  4. TiffSubIfd
  5. TiffVisitor

If Andreas doesn't fix this for v0.26, I will defer this for a future version.

#3

Updated by D Anderson over 5 years ago

That complicated huh?
Maybe Andreas can find some code from 2014 laying around ;).

#4

Updated by Robin Mills over 5 years ago

  • Estimated time changed from 10.00 h to 12.00 h

I can't find any evidence that this worked during 2013/2014. I've done 20 builds with this script:

#!/bin/bash

# 526 rmills@rmillsmbp:~/gnu/exiv2/trunk $ svn log src/tiffvisitor.cpp | grep ^r | grep -e 2013 -e 2014
# r3479 | robinwmills | 2014-12-25 13:55:57 +0000 (Thu, 25 Dec 2014) | 1 line
# r3233 | ahuggel | 2014-01-17 10:23:30 +0000 (Fri, 17 Jan 2014) | 1 line
# r3214 | robinwmills | 2013-12-28 03:41:01 +0000 (Sat, 28 Dec 2013) | 1 line
# r3201 | ahuggel | 2013-12-01 12:13:42 +0000 (Sun, 01 Dec 2013) | 1 line
# r3190 | ahuggel | 2013-10-06 15:04:48 +0100 (Sun, 06 Oct 2013) | 1 line
# r3090 | robinwmills | 2013-07-24 06:05:52 +0100 (Wed, 24 Jul 2013) | 1 line
# r3089 | robinwmills | 2013-07-24 04:12:38 +0100 (Wed, 24 Jul 2013) | 1 line
# r3056 | nkbj | 2013-06-19 05:02:05 +0100 (Wed, 19 Jun 2013) | 1 line
# r2991 | robinwmills | 2013-02-20 05:30:14 +0000 (Wed, 20 Feb 2013) | 1 line
# r2976 | robinwmills | 2013-01-16 02:23:10 +0000 (Wed, 16 Jan 2013) | 1 line
# r2966 | robinwmills | 2013-01-13 01:31:18 +0000 (Sun, 13 Jan 2013) | 1 line
# 527 rmills@rmillsmbp:~/gnu/exiv2/trunk $

for r in 4298 2966 2976 2991 3056 3089 3190 3201 3214 3233 3280 3300 3320 3340 3360 3380 3400 3420 3440 3460 3479 4298 ; do
    echo ------------- $r begin --------------
    if [ -e $r ]; then rm -rf $r ; fi
    svn checkout svn://dev.exiv2.org/svn/trunk --revision $r $r | wc
    cd $r
    make config 2>/dev/null | wc
    ./configure             | wc
    make -j  2>&1           | wc
    sudo make install       | wc
    echo SVN = $(exiv2 -vV | grep svn)
    curl --silent -O http://dev.exiv2.org/attachments/download/992/M28-1507_1_2016-04-25_0001_C0000_000001.dng
    exiv2 -pa M28-1507_1_2016-04-25_0001_C0000_000001.dng | grep SubImage1 | grep DefaultScale
    exiv2 -v -M'set Exif.SubImage1.DefaultScale 10/9 8/7' M28-1507_1_2016-04-25_0001_C0000_000001.dng
    exiv2 -pa M28-1507_1_2016-04-25_0001_C0000_000001.dng | grep SubImage1 | grep DefaultScale
    echo ------------- $r end --------------
    echo ''
    cd ..
done

# That's all Folks!
##
And here's the output: 
547 rmills@rmillsmbp:~/gnu/exiv2 $ ./build_them.sh 
------------- 4298 begin --------------
    1281    2564   51584
       2      10     116
     197    1156    9388
     197    4649   64793
     106     589    8088
SVN = svn=4298
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 4298 end --------------

------------- 2966 begin --------------
    1286    2574   60296
       2      10     116
     190    1120    9108
     260    4813   64230
      97     551    6797
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 2966 end --------------

------------- 2976 begin --------------
    1286    2574   60296
       2      10     116
     190    1120    9108
     242    4734   63438
      97     551    6797
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 2976 end --------------

------------- 2991 begin --------------
    1290    2582   60437
       2      10     116
     190    1120    9108
     242    4734   63438
      97     551    6797
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 2991 end --------------

------------- 3056 begin --------------
    1412    2826   65431
       2      10     116
     190    1120    9108
     615    6385   81155
basicio.cpp:29:1: warning: variable 'rcsId' is uninitialized when used within its own initialization [-Wuninitialized]

____ died a horrible death ____

make[1]: *** [basicio.o] Error 1
make: *** [install] Error 2
       4      50     595
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3056 end --------------

------------- 3089 begin --------------
    1518    3038   70600
       2      10     116
     190    1120    9108
   11578   65680  942138

____ died a horrible death ____

make[1]: *** [asfvideo.o] Error 1
make: *** [install] Error 2
       4      50     597
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3089 end --------------

------------- 3190 begin --------------
    1518    3038   70600
       2      10     116
     190    1120    9108
     242    4732   63436
      95     528    6545
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3190 end --------------

------------- 3201 begin --------------
    1518    3038   70600
       2      10     116
     190    1120    9108
     242    4732   63436
      95     528    6545
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3201 end --------------

------------- 3214 begin --------------
    1521    3044   70693
       2      10     116
     190    1120    9108
     198    4427   60936
      95     528    6545
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3214 end --------------

------------- 3233 begin --------------
    1521    3044   70693
       2      10     116
     190    1120    9108
     198    4427   60936
      95     528    6545
SVN =
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3233 end --------------

------------- 3280 begin --------------
    1525    3052   70849
       2      10     116
     190    1120    9108
     199    4429   60961
      96     534    6632
SVN = svn=3280
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3280 end --------------

------------- 3300 begin --------------
    1527    3056   70904
       2      10     116
     190    1120    9108
     199    4429   60961
      96     534    6632
SVN = svn=3300
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3300 end --------------

------------- 3320 begin --------------
    1527    3056   70904
       2      10     116
     190    1120    9108
     199    4429   60961
      96     534    6632
SVN = svn=3320
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3320 end --------------

------------- 3340 begin --------------
    1527    3056   70904
       2      10     116
     190    1120    9108
     199    4429   60961
      96     534    6632
SVN = svn=3340
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3340 end --------------

------------- 3360 begin --------------
    1534    3070   71145
       2      10     116
     190    1120    9108
     201    4484   61746
      97     540    6721
SVN = svn=3360
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3360 end --------------

------------- 3380 begin --------------
    1536    3074   71211
       2      10     116
     190    1120    9108
     201    4484   61746
      97     540    6721
SVN = svn=3380
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3380 end --------------

------------- 3400 begin --------------
    1536    3074   71211
       2      10     116
     190    1120    9108
     201    4484   61746
      97     540    6721
SVN = svn=3400
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3400 end --------------

------------- 3420 begin --------------
    1536    3074   71211
       2      10     116
     190    1120    9108
     201    4484   61746
      97     540    6721
SVN = svn=3420
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3420 end --------------

------------- 3440 begin --------------
    1536    3074   71211
       2      10     116
     190    1120    9108
     201    4484   61746
      97     540    6721
SVN = svn=3440
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3440 end --------------

------------- 3460 begin --------------
    1540    3082   71349
       2      10     116
     190    1120    9108
     201    4484   61746
      97     540    6721
SVN = svn=3460
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3460 end --------------

------------- 3479 begin --------------
    1092    2186   44193
       2      10     116
     200    1166    9494
     224    5104   71717
     102     587    8058
SVN = svn=3479
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3479 end --------------

------------- 4298 begin --------------
    1281    2564   51584
       2      10     116
     197    1156    9388
     197    4649   64793
     106     589    8088
SVN = svn=4298
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 4298 end --------------

548 rmills@rmillsmbp:~/gnu/exiv2 $

#5

Updated by D Anderson over 5 years ago

Very strange. Highly unlikely but did this guy solve the issue in his static build?
http://dev.exiv2.org/issues/1180#note-23
I could try and see if it,s possible to get some answers from him.

#6

Updated by D Anderson over 5 years ago

Exiv2 doesn,t allow deleting default scale tag either it seems. DEL works with the mysterious build from here http://dev.exiv2.org/issues/1180#note-23

#7

Updated by Robin Mills over 5 years ago

DEL doesn't work for the same reason that SET and ADD do not work. The metadata in the original file is 'eclipsing' SubImage1 changes and being rewritten into the file.

Can you contact whoever made your build as they seem to have fixed this. Perhaps Andreas will look this weekend. If he doesn't, I might have another session in the debugger next week. I always fix things, however it takes time to understand other folk's magic. It's possible that Andreas didn't write this either.

I've rerun the builds with --enable-static and --disable-shared and tweaked the script:

    ./configure --disable-shared --enable-static | wc
    make -j  2>&1           | wc
    sudo make install       | wc
    echo build = $(exiv2 -vV | grep -e svn -e dll) $(ls -alt $(which exiv2))
The results are the same:

.... lots of similar stuff ....

------------- 3460 begin --------------
    1540    3082   71349
       2      10     116
     190    1120    9106
     127    2370   31848
      95     509    6447
build = dll=0 svn=3460 -rwxr-xr-x 1 root admin 2782712 6 May 10:05 /usr/local/bin/exiv2
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3460 end --------------

------------- 3479 begin --------------
    1092    2186   44193
       2      10     116
     200    1166    9492
     148    2788   38309
     100     557    7800
build = dll=0 svn=3479 -rwxr-xr-x 1 root admin 2905640 6 May 10:06 /usr/local/bin/exiv2
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 3479 end --------------

------------- 4298 begin --------------
    1281    2564   51584
       2      10     116
     197    1156    9386
     126    2488   33627
     104     559    7830
build = dll=0 svn=4298 -rwxr-xr-x 1 root admin 2757260 6 May 10:07 /usr/local/bin/exiv2
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
------------- 4298 end --------------
#8

Updated by Robin Mills over 5 years ago

  • File exiv2 exiv2 added
  • % Done changed from 60 to 70

I attach the static version of exiv2 which Daniel provided with #1180. Daniel is going to contact the person who made this as we believe she/he has the fix. 

517 rmills@rmillsmbp:~/temp $ ./exiv2 -vV
exiv2 0.24 001800 (64 bit build)
Copyright (C) 2004-2013 Andreas Huggel.

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free
Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
Boston, MA 02110-1301 USA
exiv2=0.24.0
platform=apple
compiler=Clang
bits=64
dll=0
debug=0
version=4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)
date=May 12 2014
time=11:21:35
executable=/Users/rmills/temp/./exiv2
library=/usr/lib/libiconv.2.dylib
library=/usr/lib/libz.1.dylib
library=/usr/lib/libexpat.1.dylib
library=/usr/lib/libSystem.B.dylib
library=/usr/lib/libc++.1.dylib
library=/usr/lib/system/libcache.dylib
library=/usr/lib/system/libcommonCrypto.dylib
library=/usr/lib/system/libcompiler_rt.dylib
library=/usr/lib/system/libcopyfile.dylib
library=/usr/lib/system/libcorecrypto.dylib
library=/usr/lib/system/libdispatch.dylib
library=/usr/lib/system/libdyld.dylib
library=/usr/lib/system/libkeymgr.dylib
library=/usr/lib/system/liblaunch.dylib
library=/usr/lib/system/libmacho.dylib
library=/usr/lib/system/libquarantine.dylib
library=/usr/lib/system/libremovefile.dylib
library=/usr/lib/system/libsystem_asl.dylib
library=/usr/lib/system/libsystem_blocks.dylib
library=/usr/lib/system/libsystem_c.dylib
library=/usr/lib/system/libsystem_configuration.dylib
library=/usr/lib/system/libsystem_coreservices.dylib
library=/usr/lib/system/libsystem_coretls.dylib
library=/usr/lib/system/libsystem_dnssd.dylib
library=/usr/lib/system/libsystem_info.dylib
library=/usr/lib/system/libsystem_kernel.dylib
library=/usr/lib/system/libsystem_m.dylib
library=/usr/lib/system/libsystem_malloc.dylib
library=/usr/lib/system/libsystem_network.dylib
library=/usr/lib/system/libsystem_networkextension.dylib
library=/usr/lib/system/libsystem_notify.dylib
library=/usr/lib/system/libsystem_platform.dylib
library=/usr/lib/system/libsystem_pthread.dylib
library=/usr/lib/system/libsystem_sandbox.dylib
library=/usr/lib/system/libsystem_secinit.dylib
library=/usr/lib/system/libsystem_trace.dylib
library=/usr/lib/system/libunc.dylib
library=/usr/lib/system/libunwind.dylib
library=/usr/lib/system/libxpc.dylib
library=/usr/lib/libobjc.A.dylib
library=/usr/lib/libauto.dylib
library=/usr/lib/libc++abi.dylib
library=/usr/lib/libDiagnosticMessagesClient.dylib
518 rmills@rmillsmbp:~/temp $  curl --silent -O http://dev.exiv2.org/attachments/download/992/M28-1507_1_2016-04-25_0001_C0000_000001.dng
519 rmills@rmillsmbp:~/temp $ ./exiv2 -pa M28-1507_1_2016-04-25_0001_C0000_000001.dng | grep SubImage1 | grep DefaultScale
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
520 rmills@rmillsmbp:~/temp $ ./exiv2 -v -M'set Exif.SubImage1.DefaultScale 10/9 8/7' M28-1507_1_2016-04-25_0001_C0000_000001.dng
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
521 rmills@rmillsmbp:~/temp $ ./exiv2 -pa M28-1507_1_2016-04-25_0001_C0000_000001.dng | grep SubImage1 | grep DefaultScale
Exif.SubImage1.DefaultScale                  Rational    2  10/9 8/7
522 rmills@rmillsmbp:~/temp $
To be certain about this situation, I have built a static version of vanilla v0.24. This build has the fault.
524 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ makes
make
       6      41     364
sudo make install
      93     497    6271
525 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ bin/exiv2 -vV
exiv2 0.24 001800 (64 bit build)
Copyright (C) 2004-2013 Andreas Huggel.

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free
Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
Boston, MA 02110-1301 USA
exiv2=0.24.0
platform=apple
compiler=Clang
bits=64
dll=0
debug=0
version=4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)
date=May  7 2016
time=09:42:40
executable=/Users/rmills/gnu/exiv2/0.24/bin/exiv2
library=/usr/local/lib/libintl.8.dylib
library=/usr/lib/libSystem.B.dylib
library=/usr/lib/libiconv.2.dylib
library=/usr/local/lib/libz.1.dylib
library=/usr/lib/libc++.1.dylib
library=/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
library=/usr/lib/system/libcache.dylib
library=/usr/lib/system/libcommonCrypto.dylib
library=/usr/lib/system/libcompiler_rt.dylib
library=/usr/lib/system/libcopyfile.dylib
library=/usr/lib/system/libcorecrypto.dylib
library=/usr/lib/system/libdispatch.dylib
library=/usr/lib/system/libdyld.dylib
library=/usr/lib/system/libkeymgr.dylib
library=/usr/lib/system/liblaunch.dylib
library=/usr/lib/system/libmacho.dylib
library=/usr/lib/system/libquarantine.dylib
library=/usr/lib/system/libremovefile.dylib
library=/usr/lib/system/libsystem_asl.dylib
library=/usr/lib/system/libsystem_blocks.dylib
library=/usr/lib/system/libsystem_c.dylib
library=/usr/lib/system/libsystem_configuration.dylib
library=/usr/lib/system/libsystem_coreservices.dylib
library=/usr/lib/system/libsystem_coretls.dylib
library=/usr/lib/system/libsystem_dnssd.dylib
library=/usr/lib/system/libsystem_info.dylib
library=/usr/lib/system/libsystem_kernel.dylib
library=/usr/lib/system/libsystem_m.dylib
library=/usr/lib/system/libsystem_malloc.dylib
library=/usr/lib/system/libsystem_network.dylib
library=/usr/lib/system/libsystem_networkextension.dylib
library=/usr/lib/system/libsystem_notify.dylib
library=/usr/lib/system/libsystem_platform.dylib
library=/usr/lib/system/libsystem_pthread.dylib
library=/usr/lib/system/libsystem_sandbox.dylib
library=/usr/lib/system/libsystem_secinit.dylib
library=/usr/lib/system/libsystem_trace.dylib
library=/usr/lib/system/libunc.dylib
library=/usr/lib/system/libunwind.dylib
library=/usr/lib/system/libxpc.dylib
library=/usr/lib/libobjc.A.dylib
library=/usr/lib/libauto.dylib
library=/usr/lib/libc++abi.dylib
library=/usr/lib/libDiagnosticMessagesClient.dylib
library=/usr/lib/libicucore.A.dylib
library=/usr/lib/libz.1.dylib
526 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ curl --silent -O http://dev.exiv2.org/attachments/download/992/M28-1507_1_2016-04-25_0001_C0000_000001.dng
528 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ ./bin/exiv2 -pa M28-1507_1_2016-04-25_0001_C0000_000001.dng | grep SubImage1 | grep DefaultScale
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
529 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ ./bin/exiv2 -v -M'set Exif.SubImage1.DefaultScale 10/9 8/7' M28-1507_1_2016-04-25_0001_C0000_000001.dng
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7" (Rational)
530 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ ./bin/exiv2 -pa M28-1507_1_2016-04-25_0001_C0000_000001.dng | grep SubImage1 | grep DefaultScale
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
531 rmills@rmillsmbp:~/gnu/exiv2/0.24 $
There are more forensics I can perform on static version of exiv2 attached to this issue (and issue #1180). However, let's see if Daniel can find out more before I continue exploring the code with the debugger.

#9

Updated by Andreas Huggel over 5 years ago

Sorry, I just only managed to look into this.
The issue is that in TIFF and TIFF-like images, we need to be careful, which tags to modify (in particular, delete), as both, the actual image information and the metadata is part of the same TIFF structure: When you, e.g., ask exiv2 to delete all metadata, you expect it not to delete the actual image(s). For that purpose there is the notion of "image tags", which exiv2 will not touch, even if you ask it to, and the SubImage tags are such untouchable "image tags" by definition.
See this forum post and the related issues and discussions linked from there for more detailed explanations. The code to look at is TiffHeader::isImageTag and the real solution would be to split out proper DNG and/or NEF parser(s) instead of using the TIFF parser for these.

#10

Updated by Robin Mills over 5 years ago

Thanks for talking about this, Andreas. TiffImage::writeMetadata() is an ingenious recursive encoder. When I was cutting the grass today, I was thinking "I may have to rewrite TiffImage::writeMetadata() to fix this". Then I thought that's tricky because the image and the metadata are all mixed together in one big pot. I'm rather hoping that Daniel will get the fix from the person who created the static exiv2 attached to this issue.

I recently refactored class OrfImage : public TiffImage { ... }; maybe it's easy to add: class DngImage : public TiffImage { ... }; which would have my new function DngImage::writeMetadata() { ... }; .

I'm going to reassign this for my attention and hope to either fix or defer it this week. On target to be code-complete v0.26 early June.

#11

Updated by Robin Mills over 5 years ago

As always, Andreas, you know the answer. I've brutalised the code as discussed here: http://dev.exiv2.org/boards/3/topics/1650?r=1651#message-1651 And it works: 

709 rmills@rmillsmbp:~/gnu/exiv2/ttt $ curl --silent -O http://dev.exiv2.org/attachments/download/992/M28-1507_1_2016-04-25_0001_C0000_000001.dng
710 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pa --grep DefaultScale/i M28-1507_1_2016-04-25_0001_C0000_000001.dng
Exif.SubImage1.DefaultScale                  Rational    2  1/1 5/3
711 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 --verbose -M'set Exif.SubImage1.DefaultScale 10/9 8/7 6/5 4/3' M28-1507_1_2016-04-25_0001_C0000_000001.dng
File 1/1: M28-1507_1_2016-04-25_0001_C0000_000001.dng
Set Exif.SubImage1.DefaultScale "10/9 8/7 6/5 4/3" (Rational)
712 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pa --grep DefaultScale/i M28-1507_1_2016-04-25_0001_C0000_000001.dng
Exif.SubImage1.DefaultScale                  Rational    4  10/9 8/7 6/5 4/3
713 rmills@rmillsmbp:~/gnu/exiv2/ttt $
Here's the patch:
717 rmills@rmillsmbp:~/gnu/exiv2/ttt $ svn diff .
Index: src/tiffimage.cpp
===================================================================
--- src/tiffimage.cpp    (revision 4302)
+++ src/tiffimage.cpp    (working copy)
@@ -2517,6 +2517,7 @@
                                       IfdId          group,
                                 const PrimaryGroups* pPrimaryGroups) const
     {
+        if ( hasImageTags_) return false;
         if (!hasImageTags_) {
 #ifdef DEBUG
             std::cerr << "No image tags in this image\n";
718 rmills@rmillsmbp:~/gnu/exiv2/ttt $
I'm not going to submit this code as it breaks a sensible defence mechanism. I'll have to think a little more before deciding how to proceed.

#12

Updated by D Anderson over 5 years ago

No word still from the person I pm:d regarding this issue.
I suspected there was a "defence mechanism" involved here. Exiftool allows for modifying this tag, however I had some email correspondance with Dave Coffin(dcraw) once about dcraw not reading the defaultscale tag. He responded that usually this tag isn,t supposed to be modified and that was that :).
I,ll see if I get the patch working. Since It,s only supposed to be used on a specific type of Magic lantern produced dng files I don,t see myself in any serious danger ;).
Thanks a lot for checking into this.

#13

Updated by Robin Mills over 5 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 70 to 100
  • Estimated time changed from 12.00 h to 10.00 h

I think your build was made by Thomas Davrai, who was given advice by Andreas!

When that was discussed two years ago, I didn't know what a SubIFD was! Although I've worked on Exiv2 for 8 years now, it's only in the last couple of years that I've started to really understand how the metadata is stored. Before that, I mostly focused on the build.

We have an open v1.0 issue #992 to do more work on our raw image support (which includes DNG). I thinking about recruit a student next year to work on that. http://dev.exiv2.org/boards/3/topics/1650?r=2570#message-2570

I've been thinking about adding global options which can be switched on/off at will. For example, outputting debug information. An option to say "allow editing of tags in sub images" (default false) is another possibility. I'm in the final month before code-complete for v0.26 and unwilling to extend the scope of the project. So, I'm going to close this as you have a work-around. I'll update #992 to reference this conversation and this will be considered again when #992 is serviced.

#14

Updated by D Anderson over 5 years ago

Going for a static build

sudo make distclean 
make config 
./configure --disable-shared --enable-static ; make -j ; sudo make install ; make -j samples

I,m trying to replace this 

                                      IfdId          group,
                                const PrimaryGroups* pPrimaryGroups) const
    {
        if (!hasImageTags_) {
#ifdef DEBUG
            std::cerr << "No image tags in this image\n";

With this

                                       IfdId          group,
                                 const PrimaryGroups* pPrimaryGroups) const
     {
+        if ( hasImageTags_) return false;
         if (!hasImageTags_) {
 #ifdef DEBUG
             std::cerr << "No image tags in this image\n";

But it seems to miss an expression somewhere? I,m a more than a little lost when it comes to C++ :). Any hints?
error message

tiffimage.cpp:2458:10: error: expected expression
+        if ( hasImageTags_) { return false;

#15

Updated by D Anderson over 5 years ago

Solved the issue myself! Erased the +. ANd it works! Great! That is awesomeness.


                                       IfdId          group,
                                 const PrimaryGroups* pPrimaryGroups) const
     {
         if ( hasImageTags_) return false;
         if (!hasImageTags_) {
 #ifdef DEBUG
             std::cerr << "No image tags in this image\n";

#16

Updated by D Anderson over 5 years ago

Solved the issue myself! Erased the +. ANd it works! Great! That is awesomeness.


                                       IfdId          group,
                                 const PrimaryGroups* pPrimaryGroups) const
     {
         if ( hasImageTags_) return false;
         if (!hasImageTags_) {
 #ifdef DEBUG
             std::cerr << "No image tags in this image\n";

#17

Updated by Andreas Huggel over 5 years ago

Some additional comments:

class OrfImage : public TiffImage { ... };

All image classes used to be derived from Image, and maybe use a different parser and their own header. I'm not sure of the implications when the image class is now derived from TiffImage.

[..] add: class DngImage [...]

Yes, the architecture is such that it should be easy to add new classes for different image formats. The question here is whether to split out DngImage or NefImage or both. The latter is the one with the issue, DngImage and TiffImage may be able to remain happy twins and share the same functionality (i.e., more fine-grained control of which subimage tags can be modified).

Then of course, the main question is whether it is legitimate to allow modifying (and thus deleting) subimage tags (all, some, which ones?): If you ask exiv2 to delete all metadata, it will delete all tags it considers metadata (i.e., not image tags). Is the result what you expected, or does it cripple the actual image(s)? It may be an improvement, some research is required, which for TIFF and DNG shouldn't be too difficult, as unlike NEF, these are open formats.

In the meantime, I'm quite happy if a few users, who figured out that exiv2 can actually be used to write TIFF-like images (including the image data), make the small changes to the code that are required for that for their own use. That's very different from us trying to support modifications of the actual image data in TIFFs, in addition to just the metadata. (Did you know you can write a TIFF image from scratch with exiv2 commands? - I think it's done in one of the test cases.)

#18

Updated by Andreas Huggel over 5 years ago

[...] global options which can be switched on/off at will. For example, outputting debug information

That already exists. See the -Q option of the exiv2 tool and class Exiv2::LogMsg which also explains how to output stuff from the library.

The difficulty with "outputting debug info" is deciding what debug info is useful. If we simply translate all the #ifdef DEBUG blocks to such debug logs and then turn it on, the output will be one overwhelming mess. Because I had difficulties defining what useful debug data is, there is currently not much. Personally, I still prefer the #ifdef DEBUGs, which can be commented selectively when I want to point my torchlight in one particular corner to see what is happening there, but that may be just my old-fashioned way.

I don't remember if info messages are used at all, for the same reason as debug. For errors and warnings on the other hand it is very clear what should be written out.

#19

Updated by D Anderson over 5 years ago

Very interesting.

TIFF image from scratch with exiv2 commands?

One aim I had with exiftool when starting using it was to be able to rebuild all metadata from dng files and rebuid it to cdng spec. Not sure if that would be possible with exiv2. Probably should learn how to code better and create the wanted cdng file from scratch ;).

Thanks for explaining. I just recently learnt how to work with the basic command in exiv2.
Regarding modifying the defaultscale tag it is only happening with a certain film mode in 720p filming raw material with canon eos cameras. The footage comes out stretched so a "unstretch" factor of 1.66 is very usable. The tag is not respected in Premiere or Davinci resolve but ACR and also dcraw reads this tag.

So my own statement below was wrong. I actually referred to another tag now when I rechecked my emails(I think it was about the Baseline Exposure tag)

Dave Coffin(dcraw) once about dcraw not reading the defaultscale tag. He responded that usually this tag isn,t supposed to be modified and that was that :).

Thanks for sharing a great tool.

#20

Updated by Robin Mills over 5 years ago

Daniel:
Your code for src/tiffimage.cpp#2200 should be:

    bool TiffHeader::isImageTag(      uint16_t       tag,
                                      IfdId          group,
                                const PrimaryGroups* pPrimaryGroups) const
    {
        if ( hasImageTags ) return false; // #1182
        if (!hasImageTags_) {
#ifdef DEBUG
            std::cerr << "No image tags in this image\n";
#endif
            return false;
        }
        //// .... //// don't change anything
    } // TiffHeader::isImageTag
Andreas:
Thanks for your comments about global options and deriving Image classes. I'm not going to change any of that code before v0.26. These subjects can be revisited at some time in the future.

Both:
This conversation has been very productive. I hope to recruit and mentor a student in GSoC 2017 to work on raw image support (#992). So the major features of v0.27 could be:

  1. Improved raw image code and test (#992)
  2. Update the version of Adobe XMPsdk in our code base (#941)
  3. Coverity Scan (#883)
  4. Harden the video code (#1068)
  5. Additional image (webp/webm) formats (#1048)
  6. Mobile (iOS and Android) build and test support

Lots of work ahead.

Also available in: Atom PDF