Bug #1189
Updating time changes Makernote
100%
Description
I have a bunch of Nikon D90 photos (jpg+nef) that I need to update timezone for (actually, both timezone and date, since, you know...)
I used command:
exiv2 -a -07 adjust test.nef
Then I compared original tag with the updated one - unfortunately, Makernotes were changes as well as time/date tags!
Attaching extracted metadata files.
I compared the files manually, it seems that the last 28 bytes of original Makernote were truncated.
Files
Related issues
Associated revisions
#1189 Use in-memory file to dump Nikon MakerNote tag 0x927c
#1189 Code change provided by private email from Taras. Thank You Taras.
History
Updated by Robin Mills over 5 years ago
- Category set to makernote
- Status changed from New to Assigned
- Assignee set to Robin Mills
- Target version set to 0.26
- % Done changed from 0 to 10
- Estimated time set to 3.00 h
What version of exiv2 are you using?
$ exiv2 -vV # verbose versionYou're going to have to help me to get onto "the same page" as you. I have a D5300. I've run the following commands:
$ curl -O --silent http://clanmills.com/files/DSC_0001.NEF $ exiv2 -pa DSC_0001.NEF | sort > before.txt $ exiv2 -a -07 DSC_0001.NEF $ exiv2 -pa DSC_0001.NEF | sort > after.txt $ diff before.txt after.txtThe output seems fine to me! Sure the makernote has been rewritten and is a slightly different length. The location of the image has moved and so on. However the metadata seems preserved and OK (evidence below):
665 rmills@rmillsmbp:~/temp/foo $ diff before.txt after.txt 4c4 < Exif.Image.DateTime Ascii 20 2016:04:20 08:00:13 --- > Exif.Image.DateTime Ascii 20 2016:04:20 01:00:13 6,7c6,7 < Exif.Image.ExifTag Long 1 2544 < Exif.Image.GPSTag Long 1 177664 --- > Exif.Image.ExifTag Long 1 3328 > Exif.Image.GPSTag Long 1 177846 22,23c22,23 < Exif.Image.StripOffsets Long 1 178176 < Exif.Image.SubIFDs Long 3 236032 236152 236380 --- > Exif.Image.StripOffsets Long 1 177864 > Exif.Image.SubIFDs Long 3 2866 2984 3210 25c25 < Exif.Image.XMLPacket Byte 2048 (Binary value suppressed) --- > Exif.Image.XMLPacket Byte 2378 (Binary value suppressed) 29c29 < Exif.MakerNote.Offset Long 1 3088 --- > Exif.MakerNote.Offset Long 1 3810 55c55 < Exif.Nikon3.Preview Long 1 23766 --- > Exif.Nikon3.Preview Long 1 23740 122c122 < Exif.NikonPreview.JPEGInterchangeFormat Long 1 24038 --- > Exif.NikonPreview.JPEGInterchangeFormat Long 1 23846 145,146c145,146 < Exif.Photo.DateTimeDigitized Ascii 20 2016:04:20 08:00:13 < Exif.Photo.DateTimeOriginal Ascii 20 2016:04:20 08:00:13 --- > Exif.Photo.DateTimeDigitized Ascii 20 2016:04:20 01:00:13 > Exif.Photo.DateTimeOriginal Ascii 20 2016:04:20 01:00:13 160c160 < Exif.Photo.MakerNote Undefined 174576 (Binary value suppressed) --- > Exif.Photo.MakerNote Undefined 173976 (Binary value suppressed) 176c176 < Exif.SubImage1.JPEGInterchangeFormat Long 1 1239552 --- > Exif.SubImage1.JPEGInterchangeFormat Long 1 235464 197c197 < Exif.SubImage2.StripOffsets Long 1 4171776 --- > Exif.SubImage2.StripOffsets Long 1 3167648 201c201 < Exif.SubImage3.JPEGInterchangeFormat Long 1 236544 --- > Exif.SubImage3.JPEGInterchangeFormat Long 1 21326300 666 rmills@rmillsmbp:~/temp/foo $The maker note is decoded into tags Exif.NikonN.XXXX. There are 112 of them and they seem to have been transferred safely.
Exif.NikonFi.FileNumber Short 1 1
669 rmills@rmillsmbp:~/temp/foo $ exiv2 -pa --grep Nikon/i ~/Pictures/2016/Raw/DSC_0001.NEF > orig.txt
670 rmills@rmillsmbp:~/temp/foo $ exiv2 -pa --grep Nikon/i DSC_0001.NEF > adju.txt
671 rmills@rmillsmbp:~/temp/foo $ diff orig.txt adju.txt
12c12
< Exif.Nikon3.Preview Long 1 23766
---
> Exif.Nikon3.Preview Long 1 23740
17c17
< Exif.NikonPreview.JPEGInterchangeFormat Long 1 24038
---
> Exif.NikonPreview.JPEGInterchangeFormat Long 1 23846
672 rmills@rmillsmbp:~/temp/foo $ wc orig.txt adju.txt
112 550 7688 orig.txt
112 550 7688 adju.txt
224 1100 15376 total
673 rmills@rmillsmbp:~/temp/foo $ It appears that Exiv2 has "shortened" the preview. My hunch is that Exiv2 enforces the rule that there should be no metadata in a preview. You can extract the previews on the original and adjusted files and inspect the extracted previews for metadata. I haven't bothered to do this because I'm on vacation at the moment!!Updated by Stan Kaminski over 5 years ago
Version is current Arch for ARMv5tel:
$ exiv2 -vV exiv2 0.25 001900 (32 bit build) Copyright (C) 2004-2015 Andreas Huggel. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA exiv2=0.25.0 platform=linux compiler=G++ bits=32 dll=1 debug=0 version=5.3.0 date=Dec 11 2015 time=11:41:06 svn=0 ssh=0 curl=0 id=$Id: version.cpp 3800 2015-05-08 22:26:36Z robinwmills $ executable=/usr/bin/exiv2 library=/usr/lib/libstdc++.so.6 library=/usr/lib/libgcc_s.so.1 library=/usr/lib/libc.so.6 library=/lib/ld-linux.so.3 library=/usr/lib/libz.so.1 library=/usr/lib/libexpat.so.1 library=/usr/lib/libdl.so.2 library=/usr/lib/libm.so.6 have_regex=1 have_strerror_r=1 have_gmtime_r=1 have_inttypes=1 have_libintl=1 have_lensdata=1 have_iconv=1 have_memory=1 have_memset=1 have_lstat=1 have_stdbool=1 have_stdint=1 have_stdlib=1 have_strlib=0 have_strchr=1 have_strerror=1 have_strerror_r=1 have_strings_h=0 have_strtol=1 have_mmap=1 have_munmap=1 have_sys_stat=1 have_timegm=1 have_unistd_h=0 have_sys_mman=1 have_libz=0 have_xmptoolkit=1 have_bool=1 have_strings=1 have_sys_types=1 have_unistd=1 have_unicode_path=0 enable_video=1 enable_webready=0
It seems I missed that exiv2 decodes the "binary blob" as proper tags? I thought that exiv2 says "binary blob" because it's unable to decode proprietary "blobs", I'm happy that I was mistaken.
Can we somehow check what metadata is lost? But you are, of course, right - matter of a quick script to restore this.
Just a quick question before I file another unnecessary bug: exiv2 does not modify "Exif.Image.DateTimeOriginal" on purpose, so I'll have to update it manually, right?
PS. I'm very impressed with your speed - thanks for taking care of such a useful tool!
Updated by Robin Mills over 5 years ago
- % Done changed from 80 to 100
In all honesty, I don't believe any metadata is being lost. However, I invite you to prove me wrong!
In general, exiv2 will allow you to modify almost any item of metadata. However, it is not a metadata policeman. It doesn't enforce the standard. It gives you enough power to hang yourself. There are some items of metadata that cannot be changed. For example, you can't change the image sizes. However tags such as Exif.Image.DateTimeOriginal are no protected.
747 rmills@rmillsmbp:~/temp/foo $ exiv2 -pa --grep Dimen http://clanmills.com/Stonehenge.jpg Exif.Photo.PixelXDimension Short 1 6000 Exif.Photo.PixelYDimension Short 1 4000
I believe when Andreas started writing exiv2, it was simply intended to modify some dates and options such as -t appeared at that time. So, -t might update more than one tag at a time. In general this approach is no longer part of our philosophy, although -t continues to be supported.
So, to answer your question, "Yes, you should update Exif.Image.DateTimeOriginal manually". Easy to sync this in bash:
783 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ file=Stonehenge.jpg 784 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ cp ~/$file . 785 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ DateTime=$(exiv2 -PEv -K Exif.Image.DateTime $file) 786 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ echo $DateTime 2015:07:16 20:25:28 787 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ exiv2 -pa --grep DateTime $file Exif.Image.DateTime Ascii 20 2015:07:16 20:25:28 Exif.Photo.DateTimeOriginal Ascii 20 2015:07:16 15:38:54 Exif.Photo.DateTimeDigitized Ascii 20 2015:07:16 15:38:54 788 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ exiv2 -M"set Exif.Photo.DateTimeOriginal $DateTime" $file 789 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ exiv2 -pa --grep DateTime $file Exif.Image.DateTime Ascii 20 2015:07:16 20:25:28 Exif.Photo.DateTimeOriginal Ascii 20 2015:07:16 20:25:28 Exif.Photo.DateTimeDigitized Ascii 20 2015:07:16 15:38:54 790 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $
Updated by Robin Mills over 5 years ago
- Status changed from Assigned to Closed
- Estimated time changed from 3.00 h to 2.00 h
Stan
I'm going to mark this "Closed". You are welcome to update this issue and I will respond. I will re-open the issue if we discover further work has to be done.
Robin
Updated by Stan Kaminski over 5 years ago
Hi Robin,
I'm afraid this either necessitates more clarification, or it's something to be fixed...
I tried doing Makernote transplant between files, but it still loses about 100 bytes. Sequence:
$ exiv2 ad -a +7 test.nef $ exiv2 -b -Pkv test2.nef -g Exif.Photo.MakerNote > origin.exif $ exiv2 -b -Pkv test.nef -g Exif.Photo.MakerNote > timezone.exif $ ls -la -rw-r--r-- 1 stan stan 428344 May 31 12:30 origin.exif -rw-r--r-- 1 stan stan 12913969 May 29 21:25 test2.nef -rw-r--r-- 1 stan stan 6568533 May 29 22:09 test.jpg -rw-r--r-- 1 stan stan 12913969 May 31 12:30 test.nef -rw-r--r-- 1 stan stan 428243 May 31 12:30 timezone.exif
Interesting (perhaps?) note: sizes of test.nef and test2.nef match, but size of extracted metadata differs.
$ cat origin.exif | sed -E -e "s/^/set /" > foo.cmd $ exiv2 -mfoo.cmd test.nef $ exiv2 -b -Pkv test.nef -g Exif.Photo.MakerNote > timezone2.exif $ ls -la -rw-r--r-- 1 stan stan 428348 May 31 12:30 foo.cmd -rw-r--r-- 1 stan stan 428344 May 31 12:30 origin.exif -rw-r--r-- 1 stan stan 12913969 May 29 21:25 test2.nef -rw-r--r-- 1 stan stan 6568533 May 29 22:09 test.jpg -rw-r--r-- 1 stan stan 12913969 May 31 12:30 test.nef -rw-r--r-- 1 stan stan 428243 May 31 12:30 timezone2.exif -rw-r--r-- 1 stan stan 428243 May 31 12:30 timezone.exif
And here you can see that the size of "timezone.exif" and "timezone2.exif" is the same, and different from the size of "origin.exif" (which I tried to import).
Based on what you stated above, does exiv2 parser silently strips some data there? Can we do something about it? (import Makernote 1:1)
Updated by Robin Mills over 5 years ago
I didn't write Exiv2, so I can't say off-hand exactly why we rewrite the file with a slightly different makernote lengths.
More important is to see if metadata has been lost in the rewrite. You'll see in the note#1 above, I compared orig.txt and adju.txt http://dev.exiv2.org/issues/1189#note-1
Can you repeat that investigation on your files. You can also extract the previews and compare them (pixel by pixel) before and after. You'll need solid evidence to convince me that there is something wrong here. Just saying "the lengths are different" isn't evidence that the data has been been corrupted. When the bytes of the graphics are written to file, the offsets will be different and that can/could easily explain the difference.
I'm on vacation, so my time is limited at present.
Updated by Robin Mills over 5 years ago
I've had another little look at this using my file DSC_0001.NEF
Previews (Thumbnails)
The preview images have not been disturbed by the -t option. For sure the physical location of the preview/subImages has changed, however their content has not.
698 rmills@rmillsmbp:~/temp $ del thumbs *preview* 699 rmills@rmillsmbp:~/temp $ file=DSC_0001.NEF 700 rmills@rmillsmbp:~/temp $ cp ~/Pictures/2016/Raw/$file . 701 rmills@rmillsmbp:~/temp $ exiv2 -pa $file | sort > orig.txt 702 rmills@rmillsmbp:~/temp $ exiv2 --verbose -ep $file File 1/1: DSC_0001.NEF Writing preview 1 (image/tiff, 160x120 pixels, 57852 bytes) to file ./DSC_0001-preview1.tif Writing preview 2 (image/jpeg, 640x424 pixels, 150119 bytes) to file ./DSC_0001-preview2.jpg Writing preview 3 (image/jpeg, 1620x1080 pixels, 1002500 bytes) to file ./DSC_0001-preview3.jpg Writing preview 4 (image/jpeg, 6000x4000 pixels, 2932184 bytes) to file ./DSC_0001-preview4.jpg 703 rmills@rmillsmbp:~/temp $ mkdir thumbs 704 rmills@rmillsmbp:~/temp $ mv *preview* thumbs 705 rmills@rmillsmbp:~/temp $ exiv2 -a -07 $file 706 rmills@rmillsmbp:~/temp $ exiv2 -pa $file | sort > adju.txt 707 rmills@rmillsmbp:~/temp $ exiv2 --verbose -ep $file File 1/1: DSC_0001.NEF Writing preview 1 (image/tiff, 160x120 pixels, 57852 bytes) to file ./DSC_0001-preview1.tif Writing preview 2 (image/jpeg, 640x424 pixels, 150119 bytes) to file ./DSC_0001-preview2.jpg Writing preview 3 (image/jpeg, 1620x1080 pixels, 1002500 bytes) to file ./DSC_0001-preview3.jpg Writing preview 4 (image/jpeg, 6000x4000 pixels, 2932184 bytes) to file ./DSC_0001-preview4.jpg 708 rmills@rmillsmbp:~/temp $ ls -lt thumbs/* -rw-r--r--+ 1 rmills staff 57852 31 May 21:08 thumbs/DSC_0001-preview1.tif -rw-r--r--+ 1 rmills staff 150119 31 May 21:08 thumbs/DSC_0001-preview2.jpg -rw-r--r--+ 1 rmills staff 1002500 31 May 21:08 thumbs/DSC_0001-preview3.jpg -rw-r--r--+ 1 rmills staff 2932184 31 May 21:08 thumbs/DSC_0001-preview4.jpg 709 rmills@rmillsmbp:~/temp $ ls -lt *preview* -rw-r--r--+ 1 rmills staff 57852 31 May 21:10 DSC_0001-preview1.tif -rw-r--r--+ 1 rmills staff 150119 31 May 21:10 DSC_0001-preview2.jpg -rw-r--r--+ 1 rmills staff 1002500 31 May 21:10 DSC_0001-preview3.jpg -rw-r--r--+ 1 rmills staff 2932184 31 May 21:10 DSC_0001-preview4.jpg 710 rmills@rmillsmbp:~/temp $ diff orig.txt adju.txt 4c4 < Exif.Image.DateTime Ascii 20 2016:04:20 08:00:13 --- > Exif.Image.DateTime Ascii 20 2016:04:20 01:00:13 6,7c6,7 < Exif.Image.ExifTag Long 1 2544 < Exif.Image.GPSTag Long 1 177664 --- > Exif.Image.ExifTag Long 1 3328 > Exif.Image.GPSTag Long 1 177846 22,23c22,23 < Exif.Image.StripOffsets Long 1 178176 < Exif.Image.SubIFDs Long 3 236032 236152 236380 --- > Exif.Image.StripOffsets Long 1 177864 > Exif.Image.SubIFDs Long 3 2866 2984 3210 25c25 < Exif.Image.XMLPacket Byte 2048 (Binary value suppressed) --- > Exif.Image.XMLPacket Byte 2378 (Binary value suppressed) 29c29 < Exif.MakerNote.Offset Long 1 3088 --- > Exif.MakerNote.Offset Long 1 3810 55c55 < Exif.Nikon3.Preview Long 1 23766 --- > Exif.Nikon3.Preview Long 1 23740 122c122 < Exif.NikonPreview.JPEGInterchangeFormat Long 1 24038 --- > Exif.NikonPreview.JPEGInterchangeFormat Long 1 23846 145,146c145,146 < Exif.Photo.DateTimeDigitized Ascii 20 2016:04:20 08:00:13 < Exif.Photo.DateTimeOriginal Ascii 20 2016:04:20 08:00:13 --- > Exif.Photo.DateTimeDigitized Ascii 20 2016:04:20 01:00:13 > Exif.Photo.DateTimeOriginal Ascii 20 2016:04:20 01:00:13 160c160 < Exif.Photo.MakerNote Undefined 174576 (Binary value suppressed) --- > Exif.Photo.MakerNote Undefined 173976 (Binary value suppressed) 176c176 < Exif.SubImage1.JPEGInterchangeFormat Long 1 1239552 --- > Exif.SubImage1.JPEGInterchangeFormat Long 1 235464 197c197 < Exif.SubImage2.StripOffsets Long 1 4171776 --- > Exif.SubImage2.StripOffsets Long 1 3167648 201c201 < Exif.SubImage3.JPEGInterchangeFormat Long 1 236544 --- > Exif.SubImage3.JPEGInterchangeFormat Long 1 21326300 711 rmills@rmillsmbp:~/temp $
200 Keys are unchanged. No keys lost
734 rmills@rmillsmbp:~/temp $ wc orig.txt adju.txt
208 991 14386 orig.txt
208 991 14381 adju.txt
416 1982 28767 total
735 rmills@rmillsmbp:~/temp $
XMLPacket
The tag Exif.Image.XMLPacket has a different length before and after the use of -t. However, the XML (xmp) is identical. It's formatted differently for reasons I have not investigated.
720 rmills@rmillsmbp:~/temp $ orig=orig.nef
721 rmills@rmillsmbp:~/temp $ adju=adju.nef
722 rmills@rmillsmbp:~/temp $ file=DSC_0001.NEF
723 rmills@rmillsmbp:~/temp $ cp ~/Pictures/2016/Raw/$file $orig
724 rmills@rmillsmbp:~/temp $ cp ~/Pictures/2016/Raw/$file $adju
725 rmills@rmillsmbp:~/temp $ exiv2 -a -07 $adju
726 rmills@rmillsmbp:~/temp $ exiv2 -pa --grep XMLPacket $orig $adju
orig.nef Exif.Image.XMLPacket Byte 2048 (Binary value suppressed)
adju.nef Exif.Image.XMLPacket Byte 2378 (Binary value suppressed)
727 rmills@rmillsmbp:~/temp $ exiv2 -pX $orig | xmllint --pretty 1 - > orig.xmp
728 rmills@rmillsmbp:~/temp $ exiv2 -pX $adju | xmllint --pretty 1 - > adju.xmp
729 rmills@rmillsmbp:~/temp $ ls -alt *.xmp
-rw-r--r--+ 1 rmills staff 350 31 May 21:23 adju.xmp
-rw-r--r--+ 1 rmills staff 359 31 May 21:22 orig.xmp
-rw-r--r--+ 1 rmills staff 3867 18 Jan 07:32 ttt.xmp
-rw-r--r--+ 1 rmills staff 3867 18 Jan 07:32 trunk.xmp
-rw-r--r--+ 1 rmills staff 3332 8 Jan 11:15 exv.xmp
-rw-r--r--+ 1 rmills staff 4854 8 Jan 11:09 eX.xmp
-rw-r--r--+ 1 rmills staff 4178 8 Jan 10:56 raw.xmp
730 rmills@rmillsmbp:~/temp $ del ttt.xmp trunk.xmp exv.xmp eX.xmp raw.xmp
731 rmills@rmillsmbp:~/temp $ cat adju.xmp
<?xml version="1.0"?>
<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description xmlns:xmp="http://ns.adobe.com/xap/1.0/" rdf:about="" xmp:Rating="0"/>
</rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>
732 rmills@rmillsmbp:~/temp $
Exif.Image.Exif
Don't know what this is. Changed? Yes. Why? Don't know.
< Exif.Image.ExifTag Long 1 2544 --- > Exif.Image.ExifTag Long 1 3328
Exif.Image.GPS
Don't know what this is. Changed? Yes. Why? Don't know.
< Exif.Image.GPSTag Long 1 177664 --- > Exif.Image.GPSTag Long 1 177846
Right. I'm in Largs in Scotland. I was born here. I haven't been "home" for 3 years. The weather is perfect. About 24ºC today and not a cloud in the sky.
Updated by Robin Mills over 5 years ago
Stan
Another beautiful sunny morning in Scotland. I've taken a careful look at your note #6 above: http://dev.exiv2.org/issues/1189#note-6
I don't know that copying a makernote from one file to another is too wise, however I believe it will work. I'm a little lost about the difference between test.nef and test2.nef. Perhaps you can use Dropbox or something to send me the files for investigation.
Updated by Stan Kaminski over 5 years ago
You know, this issue is probably not important enough to warrant looking at the screen instead of looking at this incredible vista :-)
I've been doing some digging myself, but it's taking me time to get up-to-speed with how EXIF information works.
I did some tests with exiftool, just to compare results, and man, this one changes even more stuff. At least I could confirm that the preview has not changed at all. So back to exiv2 :-)
I found that in the Makernote, the 100-byte difference is just 28 characters (28 bytes), in my case: "71 14 168 208 168 96 49 156 66 149 217 195 50 159 182 148 37 210 59 252 209 131 130 82 185 255 217 0". If that'd be all 0's then I don't care, but clearly here is some information that is lost. Original size is 120624 bytes, after modification it's 120597 bytes.
Now the interesting part is that apart from losing 28 bytes at the end, the rest is identical between versions. Is it possible that exiv2 does not know the tags at the end and quietly deletes them? Looking at the code, it should report even unknown tags (that's also something I checked, there's no difference in unknown tags between versions).
Also, there's no xmp data in my NEFs, exiv2 even throws relevant warning.
When I do "adjust" it actually changes only times, does nothing to Exif.Image.GPSTag and Exif.Image.ExifTag.
Here's the photo I'm doing my tests on (12 MB): https://www.dropbox.com/s/whpww59cc1kjkns/2016-05-20%2008.28.54.nef?dl=0 - let me know when you get it, so I can remove it from public folder.
Updated by Stan Kaminski over 5 years ago
Can't edit previous update: I just wanted to state the obvious, it looks like exiv2 is prematurely finishing writing Makernote. Either that, or it's reading too much? (I mean, dump = read -> too long, or update = write -> too short).
Updated by Robin Mills over 5 years ago
- Status changed from Closed to Assigned
- % Done changed from 100 to 50
- Estimated time changed from 2.00 h to 4.00 h
Thanks, Stan. I've got you file.
I don't know what's in those 28 bytes. Will have to step the code in the debugger to find out what's going on.
You might be interested to see the output from exiv2 -pS (v0.25) and even more interested to see exiv2 -pR (trunk). Here's what I see in your file:
506 rmills@rmillsmbp:~ $ exiv2 -pR ~/Downloads/2016-05-20\ 08.28.54.nef
STRUCTURE OF TIFF FILE (MM): /Users/rmills/Downloads/2016-05-20 08.28.54.nef
address | tag | type | count | offset | value
10 | 0x00fe NewSubfileType | LONG | 1 | 1 | 1
22 | 0x0100 ImageWidth | LONG | 1 | 160 | 160
34 | 0x0101 ImageLength | LONG | 1 | 120 | 120
46 | 0x0102 BitsPerSample | SHORT | 3 | 316 | 8 8 8
58 | 0x0103 Compression | SHORT | 1 | 65536 | 1
70 | 0x0106 PhotometricInterpretation | SHORT | 1 | 131072 | 2
82 | 0x010f Make | ASCII | 18 | 324 | NIKON CORPORATION
94 | 0x0110 Model | ASCII | 10 | 344 | NIKON D90
106 | 0x0111 StripOffsets | LONG | 1 | 121656 | 121656
118 | 0x0112 Orientation | SHORT | 1 | 524288 | 8
130 | 0x0115 SamplesPerPixel | SHORT | 1 | 196608 | 3
142 | 0x0116 RowsPerStrip | LONG | 1 | 120 | 120
154 | 0x0117 StripByteCounts | LONG | 1 | 57600 | 57600
166 | 0x011a XResolution | RATIONAL | 1 | 356 | 356/0
178 | 0x011b YResolution | RATIONAL | 1 | 364 | 364/0
190 | 0x011c PlanarConfiguration | SHORT | 1 | 65536 | 1
202 | 0x0128 ResolutionUnit | SHORT | 1 | 131072 | 2
214 | 0x0131 Software | ASCII | 10 | 372 | Ver.1.00
226 | 0x0132 DateTime | ASCII | 20 | 384 | 2016:05:20 08:28:54
238 | 0x014a SubIFDs | LONG | 2 | 404 | 179256 179376
STRUCTURE OF TIFF FILE (MM): /Users/rmills/Downloads/2016-05-20 08.28.54.nef
address | tag | type | count | offset | value
179258 | 0x00fe NewSubfileType | LONG | 1 | 1 | 1
179270 | 0x0103 Compression | SHORT | 1 | 393216 | 6
179282 | 0x011a XResolution | RATIONAL | 1 | 179360 | 48288/2
179294 | 0x011b YResolution | RATIONAL | 1 | 179368 | 48296/2
179306 | 0x0128 ResolutionUnit | SHORT | 1 | 131072 | 2
179318 | 0x0201 JPEGInterchangeFormat | LONG | 1 | 179648 | 179648
179330 | 0x0202 JPEGInterchangeFormatLeng | LONG | 1 | 1547046 | 1547046
179342 | 0x0213 YCbCrPositioning | SHORT | 1 | 131072 | 2
END /Users/rmills/Downloads/2016-05-20 08.28.54.nef
250 | 0x0214 ReferenceBlackWhite | RATIONAL | 6 | 412 | 0/0 1/0 255/0 1/0 0/0 ...
262 | 0x8769 ExifTag | LONG | 1 | 480 | 480
STRUCTURE OF TIFF FILE (MM): /Users/rmills/Downloads/2016-05-20 08.28.54.nef
address | tag | type | count | offset | value
482 | 0x829a ExposureTime | RATIONAL | 1 | 872 | 872/0
494 | 0x829d FNumber | RATIONAL | 1 | 880 | 880/0
506 | 0x8822 ExposureProgram | SHORT | 1 | 0 | 0
518 | 0x8827 ISOSpeedRatings | SHORT | 1 | 18350080 | 280
530 | 0x9003 DateTimeOriginal | ASCII | 20 | 888 | 2016:05:20 08:28:54
542 | 0x9004 DateTimeDigitized | ASCII | 20 | 908 | 2016:05:20 08:28:54
554 | 0x9204 ExposureBiasValue | SRATIONAL | 1 | 928 | 928/0
566 | 0x9205 MaxApertureValue | RATIONAL | 1 | 936 | 936/0
578 | 0x9207 MeteringMode | SHORT | 1 | 327680 | 5
590 | 0x9208 LightSource | SHORT | 1 | 0 | 0
602 | 0x9209 Flash | SHORT | 1 | 1572864 | 24
614 | 0x920a FocalLength | RATIONAL | 1 | 944 | 944/0
626 | 0x927c MakerNote | UNDEFINED | 120624 | 1012 | Nikon.....MM.*.....6........0210 ...
638 | 0x9286 UserComment | UNDEFINED | 44 | 952 | ASCII... ...
650 | 0x9290 SubSecTime | ASCII | 3 | 808452096 | 00
662 | 0x9291 SubSecTimeOriginal | ASCII | 3 | 808452096 | 00
674 | 0x9292 SubSecTimeDigitized | ASCII | 3 | 808452096 | 00
686 | 0xa217 SensingMethod | SHORT | 1 | 131072 | 2
698 | 0xa300 FileSource | UNDEFINED | 1 | 50331648 | .
710 | 0xa301 SceneType | UNDEFINED | 1 | 16777216 | .
722 | 0xa302 CFAPattern | UNDEFINED | 8 | 996 | ........
734 | 0xa401 CustomRendered | SHORT | 1 | 0 | 0
746 | 0xa402 ExposureMode | SHORT | 1 | 0 | 0
758 | 0xa403 WhiteBalance | SHORT | 1 | 0 | 0
770 | 0xa404 DigitalZoomRatio | RATIONAL | 1 | 1004 | 1004/0
782 | 0xa405 FocalLengthIn35mmFilm | SHORT | 1 | 2424832 | 37
794 | 0xa406 SceneCaptureType | SHORT | 1 | 0 | 0
806 | 0xa407 GainControl | SHORT | 1 | 0 | 0
818 | 0xa408 Contrast | SHORT | 1 | 0 | 0
830 | 0xa409 Saturation | SHORT | 1 | 0 | 0
842 | 0xa40a Sharpness | SHORT | 1 | 0 | 0
854 | 0xa40c SubjectDistanceRange | SHORT | 1 | 0 | 0
END /Users/rmills/Downloads/2016-05-20 08.28.54.nef
274 | 0x8825 GPSTag | LONG | 1 | 121636 | 121636
286 | 0x9003 DateTimeOriginal | ASCII | 20 | 460 | 2016:05:20 08:28:54
298 | 0x9216 TIFFEPStandardID | BYTE | 4 | 16777216 | ...
END /Users/rmills/Downloads/2016-05-20 08.28.54.nef
507 rmills@rmillsmbp:~ $ The interesting tag is:626 | 0x927c MakerNote | UNDEFINED | 120624 | 1012 | Nikon.....MM.*.....6........0210 ...Clearly the MakerNote is a TIFF. I thought I had code to recursively dump that tag, maybe not. There's fun ahead with this. But not today. The sun's shining and I'm "home" for the first time in 3 years.
Updated by Robin Mills over 5 years ago
- Status changed from Assigned to Closed
- % Done changed from 50 to 100
r4316. I've updated -pR to recursively dump the TIFF that's embedded in the MakerNote. I don't really believe the output. This will require more investigation which we'll factor into #992 (which is the top priority item for v0.27). I have no more time available to spend on this if I'm going to make the schedule to be code-complete on v0.26 in June 2016.
Updated by Robin Mills over 5 years ago
- Estimated time changed from 4.00 h to 6.00 h
I've spent another couple of hours on this. I've used an in-memory file (using the bytes in the 0x927c tag) to perform the recursive dump. The output of -pR now seems sensible.
STRUCTURE OF TIFF FILE (MM): /Users/rmills/Downloads/2016-05-20 08.28.54.nef
address | tag | type | count | offset | value
10 | 0x00fe NewSubfileType | LONG | 1 | 1 | 1
22 | 0x0100 ImageWidth | LONG | 1 | 160 | 160
34 | 0x0101 ImageLength | LONG | 1 | 120 | 120
46 | 0x0102 BitsPerSample | SHORT | 3 | 316 | 8 8 8
58 | 0x0103 Compression | SHORT | 1 | 65536 | 1
70 | 0x0106 PhotometricInterpretation | SHORT | 1 | 131072 | 2
82 | 0x010f Make | ASCII | 18 | 324 | NIKON CORPORATION
94 | 0x0110 Model | ASCII | 10 | 344 | NIKON D90
106 | 0x0111 StripOffsets | LONG | 1 | 121656 | 121656
118 | 0x0112 Orientation | SHORT | 1 | 524288 | 8
130 | 0x0115 SamplesPerPixel | SHORT | 1 | 196608 | 3
142 | 0x0116 RowsPerStrip | LONG | 1 | 120 | 120
154 | 0x0117 StripByteCounts | LONG | 1 | 57600 | 57600
166 | 0x011a XResolution | RATIONAL | 1 | 356 | 356/0
178 | 0x011b YResolution | RATIONAL | 1 | 364 | 364/0
190 | 0x011c PlanarConfiguration | SHORT | 1 | 65536 | 1
202 | 0x0128 ResolutionUnit | SHORT | 1 | 131072 | 2
214 | 0x0131 Software | ASCII | 10 | 372 | Ver.1.00
226 | 0x0132 DateTime | ASCII | 20 | 384 | 2016:05:20 08:28:54
238 | 0x014a SubIFDs | LONG | 2 | 404 | 179256 179376
STRUCTURE OF TIFF FILE (MM): /Users/rmills/Downloads/2016-05-20 08.28.54.nef
address | tag | type | count | offset | value
179258 | 0x00fe NewSubfileType | LONG | 1 | 1 | 1
179270 | 0x0103 Compression | SHORT | 1 | 393216 | 6
179282 | 0x011a XResolution | RATIONAL | 1 | 179360 | 48288/2
179294 | 0x011b YResolution | RATIONAL | 1 | 179368 | 48296/2
179306 | 0x0128 ResolutionUnit | SHORT | 1 | 131072 | 2
179318 | 0x0201 JPEGInterchangeFormat | LONG | 1 | 179648 | 179648
179330 | 0x0202 JPEGInterchangeFormatLeng | LONG | 1 | 1547046 | 1547046
179342 | 0x0213 YCbCrPositioning | SHORT | 1 | 131072 | 2
END /Users/rmills/Downloads/2016-05-20 08.28.54.nef
250 | 0x0214 ReferenceBlackWhite | RATIONAL | 6 | 412 | 0/0 1/0 255/0 1/0 0/0 ...
262 | 0x8769 ExifTag | LONG | 1 | 480 | 480
STRUCTURE OF TIFF FILE (MM): /Users/rmills/Downloads/2016-05-20 08.28.54.nef
address | tag | type | count | offset | value
482 | 0x829a ExposureTime | RATIONAL | 1 | 872 | 872/0
494 | 0x829d FNumber | RATIONAL | 1 | 880 | 880/0
506 | 0x8822 ExposureProgram | SHORT | 1 | 0 | 0
518 | 0x8827 ISOSpeedRatings | SHORT | 1 | 18350080 | 280
530 | 0x9003 DateTimeOriginal | ASCII | 20 | 888 | 2016:05:20 08:28:54
542 | 0x9004 DateTimeDigitized | ASCII | 20 | 908 | 2016:05:20 08:28:54
554 | 0x9204 ExposureBiasValue | SRATIONAL | 1 | 928 | 928/0
566 | 0x9205 MaxApertureValue | RATIONAL | 1 | 936 | 936/0
578 | 0x9207 MeteringMode | SHORT | 1 | 327680 | 5
590 | 0x9208 LightSource | SHORT | 1 | 0 | 0
602 | 0x9209 Flash | SHORT | 1 | 1572864 | 24
614 | 0x920a FocalLength | RATIONAL | 1 | 944 | 944/0
626 | 0x927c MakerNote | UNDEFINED | 120624 | 1012 | Nikon.....MM.*.....6........0210 ...
STRUCTURE OF TIFF FILE (MM): MemIo
address | tag | type | count | offset | value
10 | 0x0001 Version | UNDEFINED | 4 | 808595760 | 0210
22 | 0x0002 ISOSpeed | SHORT | 2 | 280 | 0 280
34 | 0x0004 Quality | ASCII | 8 | 662 | RAW
46 | 0x0005 WhiteBalance | ASCII | 13 | 670 | AUTO
58 | 0x0007 Focus | ASCII | 7 | 686 | AF-A
70 | 0x0008 FlashSetting | ASCII | 13 | 694 | NORMAL
82 | 0x0009 GPSStatus | ASCII | 20 | 710 |
94 | 0x000b ProcessingSoftware | SSHORT | 2 | 0 | 0 0
106 | 0x000c GPSSpeedRef | RATIONAL | 4 | 730 | 459/0 256/0 371/0 256/0
118 | 0x000d GPSSpeed | UNDEFINED | 4 | 67072 | ...
130 | 0x000e GPSTrackRef | UNDEFINED | 4 | 68608 | ...
142 | 0x0011 GPSImgDirection | LONG | 1 | 13954 | 13954
154 | 0x0012 GPSMapDatum | UNDEFINED | 4 | 67072 | ...
166 | 0x0013 GPSDestLatitudeRef | SHORT | 2 | 280 | 0 280
178 | 0x0017 GPSDestBearingRef | UNDEFINED | 4 | 67072 | ...
190 | 0x0018 GPSDestBearing | UNDEFINED | 4 | 67072 | ...
202 | 0x0019 GPSDestDistanceRef | SRATIONAL | 1 | 762 | 762/0
214 | 0x001b GPSProcessingMethod | SHORT | 7 | 770 | 0 4352 2868 4352 2868 ...
226 | 0x001c GPSAreaInformation | UNDEFINED | 3 | 67072 | ...
238 | 0x001d GPSDateStamp | ASCII | 8 | 786 | 7034901
250 | 0x001e GPSDifferential | SHORT | 1 | 131072 | 2
262 | 0x001f | UNDEFINED | 8 | 794 | 0100...
274 | 0x0022 | SHORT | 1 |4294901760 | 65535
286 | 0x0023 | UNDEFINED | 58 | 802 | 0100STANDARD............STANDARD ...
298 | 0x0024 | UNDEFINED | 4 | 3932160 | .<.
310 | 0x0025 | UNDEFINED | 14 | 862 | N.....N.......
322 | 0x002c | UNDEFINED | 94 | 878 | 0100...d....................... ...
334 | 0x0083 | BYTE | 1 | 234881024 | .
346 | 0x0084 | RATIONAL | 4 | 974 | 180/0 10/0 1050/0 10/0
358 | 0x0087 | BYTE | 1 | 0 |
370 | 0x0089 | SHORT | 1 | 0 | 0
382 | 0x008a | SHORT | 1 | 131072 | 2
394 | 0x008b | UNDEFINED | 4 |1073810432 | @..
406 | 0x008c | UNDEFINED | 578 | 1006 | I0............................. ...
418 | 0x0091 | UNDEFINED | 6600 | 1586 | 0213..........y"..+.....2.W2..*. ...
430 | 0x0093 | SHORT | 1 | 262144 | 4
442 | 0x0095 | ASCII | 5 | 8186 | OFF
454 | 0x0096 | UNDEFINED | 624 | 8194 | D .H.H.H.H....... .0.@.P.`.p.... ...
466 | 0x0097 | UNDEFINED | 1302 | 8818 | 0211........................... ...
478 | 0x0098 | UNDEFINED | 33 | 10122 | 02041...E..^GPH..........W10o.k ...
490 | 0x0099 | SHORT | 2 | 142607770 | 2176 1434
502 | 0x009e | SHORT | 10 | 10158 | 0 0 0 0 0 ...
514 | 0x00a3 | BYTE | 1 | 0 |
526 | 0x00a4 | UNDEFINED | 4 | 808595504 | 0200
538 | 0x00a7 | LONG | 1 | 4736 | 4736
550 | 0x00a8 | UNDEFINED | 22 | 10178 | 0103.................
562 | 0x00ab | ASCII | 16 | 10202 | AUTO
574 | 0x00b0 | UNDEFINED | 16 | 10218 | 0100...........
586 | 0x00b1 | SHORT | 1 | 0 | 0
598 | 0x00b6 | UNDEFINED | 8 | 10234 | .......
610 | 0x00b7 | UNDEFINED | 30 | 10242 | 0100.........................
622 | 0x00b8 | UNDEFINED | 172 | 10274 | 0100...d....................... ...
634 | 0x00bb | UNDEFINED | 6 | 10446 | 0100.
646 | 0x00bc | UNDEFINED | 3500 | 10454 | 0100.D."...........P..p..@...A.. ...
END MemIo
638 | 0x9286 UserComment | UNDEFINED | 44 | 952 | ASCII... ...
650 | 0x9290 SubSecTime | ASCII | 3 | 808452096 | 00
662 | 0x9291 SubSecTimeOriginal | ASCII | 3 | 808452096 | 00
674 | 0x9292 SubSecTimeDigitized | ASCII | 3 | 808452096 | 00
686 | 0xa217 SensingMethod | SHORT | 1 | 131072 | 2
698 | 0xa300 FileSource | UNDEFINED | 1 | 50331648 | .
710 | 0xa301 SceneType | UNDEFINED | 1 | 16777216 | .
722 | 0xa302 CFAPattern | UNDEFINED | 8 | 996 | ........
734 | 0xa401 CustomRendered | SHORT | 1 | 0 | 0
746 | 0xa402 ExposureMode | SHORT | 1 | 0 | 0
758 | 0xa403 WhiteBalance | SHORT | 1 | 0 | 0
770 | 0xa404 DigitalZoomRatio | RATIONAL | 1 | 1004 | 1004/0
782 | 0xa405 FocalLengthIn35mmFilm | SHORT | 1 | 2424832 | 37
794 | 0xa406 SceneCaptureType | SHORT | 1 | 0 | 0
806 | 0xa407 GainControl | SHORT | 1 | 0 | 0
818 | 0xa408 Contrast | SHORT | 1 | 0 | 0
830 | 0xa409 Saturation | SHORT | 1 | 0 | 0
842 | 0xa40a Sharpness | SHORT | 1 | 0 | 0
854 | 0xa40c SubjectDistanceRange | SHORT | 1 | 0 | 0
END /Users/rmills/Downloads/2016-05-20 08.28.54.nef
274 | 0x8825 GPSTag | LONG | 1 | 121636 | 121636
286 | 0x9003 DateTimeOriginal | ASCII | 20 | 460 | 2016:05:20 08:28:54
298 | 0x9216 TIFFEPStandardID | BYTE | 4 | 16777216 | ...
END /Users/rmills/Downloads/2016-05-20 08.28.54.nef
Many of the tags are not being identified by name. For example 0x0084 is Lens:
689 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ grep -H 0084 src/*.cpp
src/nikonmn.cpp: TagInfo(0x0084, "Lens", N_("Lens"), N_("Lens"), nikon3Id, makerTags, unsignedRational, -1, print0x0084),
src/nikonmn.cpp: std::ostream& Nikon3MakerNote::print0x0084(std::ostream& os,
src/quicktimevideo.cpp: { 0x2000084, "Xmp.video.LensModel" },
690 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ exiv2 -pR ~/Downloads/2016-05-20\ 08.28.54.nef | grep 0084
346 | 0x0084 | RATIONAL | 4 | 974 | 180/0 10/0 1050/0 10/0
691 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ exiv2 -pv --grep Lens$ ~/Downloads/2016-05-20\ 08.28.54.nef
0x0084 Nikon3 Lens Rational 4 180/10 1050/10 35/10 56/10
692 rmills@rmillsmbp:~/gnu/exiv2/bigtiff $ I don't have time to investigate further in v0.26.
#1189 See issue report for discussion.