Feature #1180

Add tags defined by Adobe in the Cinema DNG Specification

Added by D Anderson 12 months ago. Updated 12 months ago.

Status:ClosedStart date:25 Apr 2016
Priority:NormalDue date:
Assignee:Robin Mills% Done:

100%

Category:metadataEstimated time:4.00 hours
Target version:0.26

Description

Issue coming from here
http://dev.exiv2.org/boards/3/topics/2546

request adding following missing dng tags
TimeCodes Tag 0xc763
FrameRate Tag 0xc764

Pointed out from user Alan Pater

The FrameRate and TimeCodes tags are not yet included in exiv2.

According to exiftool, your sample file includes TimeCodes Tag 0xc763 and FrameRate Tag 0xc764.

Please open an New Issue here requesting that these tags and any others that you notice missing be added.

M28-1507_1_2016-04-27_0001_C0000_000002.dng - Missing default scale tag (1.48 MB) D Anderson, 27 Apr 2016 08:34

exiv2 - Latest compilation (2.38 MB) D Anderson, 27 Apr 2016 08:34

exiv2 (2.26 MB) D Anderson, 27 Apr 2016 14:17

Associated revisions

Revision 4288
Added by Robin Mills 12 months ago

#1180 Thank You to D Anderson for reporting this. Thank You to Alan for his analysis.

Revision 4289
Added by Robin Mills 12 months ago

#1180. Correction to r4288. Adding additional CDNG tags and tag descriptions. Thanks to Alan for bringing this to my attention.

Revision 4290
Added by Robin Mills 12 months ago

#1180. Corrections to r4289.

Revision 4291
Added by Robin Mills 12 months ago

#1180. Thanks to D Anderson for reporting this an providing the patch.

Revision 4292
Added by Robin Mills 12 months ago

#1180. Reverting r4291 as it is not required. DefaultScale was adding in r1546 by Andreas in 2008.

History

#1 Updated by Robin Mills 12 months ago

  • Category set to metadata
  • Status changed from New to Assigned
  • Assignee set to Robin Mills
  • Target version set to 0.26
  • % Done changed from 0 to 80
  • Estimated time set to 1.00

Thanks for reporting this. And thanks to Alan for his analysis. Fix submitted r4288.

Yes. I can see that the tags are there, however we don't have them registered in dngTags in src/tags.cpp

Output before the fix:

771 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pR ~/Downloads/M28-1507_1_2016-04-25_0001_C0000_000001.dng  | grep 76
     250 | 0x8769 ExifTag                   |      LONG |        1 |      1084 | 1084
     346 | 0xc628 AsShotNeutral             |  RATIONAL |        3 |       678 | 7681/0 13200/0 1/0
     406 | 0xc763                           |      BYTE |        8 |       734 | .......
     418 | 0xc764                           | SRATIONAL |        1 |       742 | 742/0
772 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
Output with fix:
799 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pR ~/Downloads/M28-1507_1_2016-04-25_0001_C0000_000001.dng  | grep c7
     406 | 0xc763 TimeCodes                 |      BYTE |        8 |       734 | .......
     418 | 0xc764 FrameRate                 | SRATIONAL |        1 |       742 | 742/0
800 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pa --grep TimeCodes --grep FrameRate ~/Downloads/M28-1507_1_2016-04-25_0001_C0000_000001.dng  
Exif.Image.TimeCodes                         Byte        8  1 0 0 0 0 0 0 0
Exif.Image.FrameRate                         SRational   1  2997/50
801 rmills@rmillsmbp:~/gnu/exiv2/ttt $ 

#2 Updated by Robin Mills 12 months ago

  • Status changed from Assigned to Resolved
  • % Done changed from 80 to 100

#3 Updated by Alan Pater 12 months ago

  • Status changed from Resolved to Assigned
  • % Done changed from 100 to 90

There are a few other tags in the CinemaDNG spec that didn't appear in the sample file:

TStop          0xc772
ReelName       0xc789
CameraLabel    0xc7a1

http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/cinemadng/pdfs/CinemaDNG_Format_Specification_v1_1.pdf#page=10

exiftool includes others as well:

http://owl.phy.queensu.ca/~phil/exiftool/TagNames/EXIF.html

#4 Updated by D Anderson 12 months ago

Thanks Robin Mills. Great response. That was fast. Now I just have to make exiv2 compile on my machine to test it out.
There are some other tags as pointed out by Alan Pater. I don,t use them personally but I,m sure they could be useful to somebody.

#5 Updated by D Anderson 12 months ago

Just tested right now after running automake with Robin Mills latest changes. Confirmed working.

Exif.Image.TimeCodes                         Byte        8  1 0 0 0 0 0 0 0
Exif.Image.FrameRate                         SRational   1  2997/50

#6 Updated by Robin Mills 12 months ago

  • % Done changed from 90 to 60
  • Estimated time changed from 1.00 to 2.00

Alan:
Thanks I'll investigate and add those three tags this evening. I'll have a sniff around the exiftool web site at the others.

D:
Glad that you have the code build. We have a build server and it builds (and tests) the code every night on all our supported platforms (about 20). We retain "daily" builds for 60 days, every Monday's build for 120 days, and the first of every month indefinietly. http://exiv2.dyndns.org:8080/userContent/builds/

The buildserver also builds the code on 5 platforms every time we submit code, however those builds are mainly for team members to be sure they haven't broken the build. We don't preserve/publish incremental builds.

#7 Updated by Robin Mills 12 months ago

  • Tracker changed from Bug to Feature
  • Subject changed from Add Frame Rate and Timecode tag to dng spec to Add tags defined by Adobe in the Cinema DNG Specification
  • % Done changed from 60 to 100
  • Estimated time changed from 2.00 to 3.00

Fix submitted: r4289. I've modified the title of this issue.

Alan:
Thanks for bring the Adobe CDNG spec to my attention. We now have all 5 tags and I've updated the tag descriptions from the spec.

I see a very long list of tags on Phil's web site. One day, in that mysterious land (Mars) where there are more than 24 hours in every day, that long list should be reconciled with src/tags.cpp. However that day won't be before v0.26 code-complete in June 2016. If you believe this should be done, please raise an issue and target it for v1.0. It will be considered for inclusion in v0.27, possibly as a GSoC project and combined with #992. And we should install all of Phil's test files on the build-server and have an extended test suite that is run occasionally on the build-server.

#8 Updated by D Anderson 12 months ago

Hi Robin! Been testing some more and somehow I cannot set the "default scale tag" in later exiv2 builds. Reason I didn,t see this before is I have been testing with a static build I found on the web from around 2014. My command looks like this. altering default scale tag will affect aspect ratio in ACR which is very useful for some dng files produced with Magic Lantern project.

exiv2 -v -M"add Exif.SubImage1.DefaultScale Rational 1/1 5/3" *.dng

From -verbose it says.

File   2/760: M28-1507_1_2016-04-27_0001_C0000_000003.dng
Add Exif.SubImage1.DefaultScale "1/1 5/3" (Rational)

Not sure if maybe something is broken on my computer. Would you mind check if you manage to set the provided default scale tag here?

#9 Updated by Robin Mills 12 months ago

I'm a little lost. Are you saying:
1 "DefaultScale" was working with exiv2 from 2014 and is not working now?
2 You are trying to set "DefaultScale" and it's not working and has never been working?

According to the exiv2 man page, DNG is supported for read/write. DNG is a TIFF and we certainly read/write TIFF.

'DefaultScale' is listed as 0xc61e on http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/EXIF.html. Perhaps this is one of the tags to which Alan referred. Maybe it has been added to CDNG since Adobe original 2011 spec. For sure, it's not registered in src/tags.cpp

By the way, I didn't write exiv2. I'm the build guy (and that's why I know about the build and dyld and stuff). I'd never heard of CDNG until Monday!

#10 Updated by D Anderson 12 months ago

I think it,s present in dng spec. Well, this is confusing. Having lunch atm but I,m pretty sure exiv2 reads the default scale tag.
If of any help I could upload the 2014 build.
Regarding dng/cdng approach. Is it possible to add all dng tags both dng and cdng tags and then the user can use whatever is considered necessary for the files? Isn,t cdng a sort of stripped version of dng anyway?

#11 Updated by D Anderson 12 months ago

The tag seems supported in tags.cpp. Can you tell if it,s writable?

        TagInfo(0xc61e, "DefaultScale", N_("Default Scale"),
                N_("DefaultScale is required for cameras with non-square pixels. It " 
                   "specifies the default scale factors for each direction to convert " 
                   "the image to square pixels. Typically these factors are selected " 
                   "to approximately preserve total pixel count. For CFA images that " 
                   "use CFALayout equal to 2, 3, 4, or 5, such as the Fujifilm SuperCCD, " 
                   "these two values should usually differ by a factor of 2.0."),
                ifd0Id, dngTags, unsignedRational, 2, printValue), // DNG tag

#12 Updated by Robin Mills 12 months ago

  • Estimated time changed from 3.00 to 4.00

D:

Thanks for the info about "DefaultScale". I'll add that in a few minutes. It will be: r4291. Your message arrived when I was about to send the following:

I've tested your command on both v0.23 and v0.24. I don't believe DefaultScale is supported in earlier releases.

545 rmills@rmillsmbp:~/gnu/exiv2/0.23 $ exiv2 -v -M"add Exif.SubImage1.DefaultScale Rational 1/1 5/3" ~/Downloads/M*2.dng
File 1/1: /Users/rmills/Downloads/M28-1507_1_2016-04-27_0001_C0000_000002.dng
Add Exif.SubImage1.DefaultScale "1/1 5/3" (Rational)
546 rmills@rmillsmbp:~/gnu/exiv2/0.23 $ exiv2 -pa  ~/Downloads/M*2.dng | grep -i scale
547 rmills@rmillsmbp:~/gnu/exiv2/0.23 $ exiv2 -V | grep build
exiv2 0.23 001700 (64 bit build)
548 rmills@rmillsmbp:~/gnu/exiv2/0.23 $ exiv2 -v -M"add Exif.SubImage1.DefaultScale Rational 1/1 5/3" ~/Downloads/M*2.dng
File 1/1: /Users/rmills/Downloads/M28-1507_1_2016-04-27_0001_C0000_000002.dng
Add Exif.SubImage1.DefaultScale "1/1 5/3" (Rational)
549 rmills@rmillsmbp:~/gnu/exiv2/0.23 $ exiv2 -pa  ~/Downloads/M*2.dng | grep -i scale
550 rmills@rmillsmbp:~/gnu/exiv2/0.23 $ 
----
551 rmills@rmillsmbp:~/gnu/exiv2 $ cd ../0.24
552 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ makes
make
       6      41     364
sudo make install
      95     528    6545
553 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ exiv2 -V | grep build
exiv2 0.24 001800 (64 bit build)
554 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ exiv2 -v -M"add Exif.SubImage1.DefaultScale Rational 1/1 5/3" ~/Downloads/M*2.dng
File 1/1: /Users/rmills/Downloads/M28-1507_1_2016-04-27_0001_C0000_000002.dng
Add Exif.SubImage1.DefaultScale "1/1 5/3" (Rational)
555 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ exiv2 -pa  ~/Downloads/M*2.dng | grep -i scale
556 rmills@rmillsmbp:~/gnu/exiv2/0.24 $ 

Just listing all the tags and adding them to Exiv2 doesn't seem very difficult. However, this discussion started with 2 tags. Then Alan pointed to the Adobe CDNG spec and I added all 5 listed in the spec. And now it seems that there could be many more tags.

I'm trying to get finished with v0.26 at the moment. I think, regrettably, I have to push this issue on the backlog to be scheduled for a later release. If you, or Alan (or somebody else) can make a short definitive list of the tags to be added, I'll take care of it immediately.

Specifically, what's needed is to extend the table which I've added to src/tags.cpp for the 5 Adobe tags.

        ////////////////////////////////////////
        // http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/cinemadng/pdfs/CinemaDNG_Format_Specification_v1_1.pdf
        TagInfo(0xc763, "TimeCodes", N_("TimeCodes"),
                N_("The optional TimeCodes tag shall contain an ordered array of time codes. " 
                "All time codes shall be 8 bytes long and in binary format. The tag may " 
                "contain from 1 to 10 time codes. When the tag contains more than one time " 
                "code, the first one shall be the default time code. This specification " 
                "does not prescribe how to use multiple time codes.\n\n" 
                "Each time code shall be as defined for the 8-byte time code structure in " 
                "SMPTE 331M-2004, Section 8.3. See also SMPTE 12-1-2008 and SMPTE 309-1999."),
                ifd0Id, dngTags, unsignedByte, 8, printValue), // DNG tag
        TagInfo(0xc764, "FrameRate", N_("FrameRate"),
                N_("The optional FrameRate tag shall specify the video frame " 
                   "rate in number of image frames per second, expressed as a " 
                   "signed rational number. The numerator shall be non-negative " 
                   "and the denominator shall be positive. This field value is " 
                   "identical to the sample rate field in SMPTE 377-1-2009."),
                ifd0Id, dngTags, signedRational, 1, printValue), // DNG tag
        TagInfo(0xc772, "TStop", N_("TStop"),
                N_("The optional TStop tag shall specify the T-stop of the " 
                "actual lens, expressed as an unsigned rational number. " 
                "T-stop is also known as T-number or the photometric " 
                "aperture of the lens. (F-number is the geometric aperture " 
                "of the lens.) When the exact value is known, the T-stop " 
                "shall be specified using a single number. Alternately, " 
                "two numbers shall be used to indicate a T-stop range, " 
                "in which case the first number shall be the minimum " 
                "T-stop and the second number shall be the maximum T-stop."),
                ifd0Id, dngTags, signedRational, 1, printValue), // DNG tag
        TagInfo(0xc789, "ReelName", N_("ReelName"),
                N_("The optional ReelName tag shall specify a name for a " 
                "sequence of images, where each image in the sequence has " 
                "a unique image identifier (including but not limited to file " 
                "name, frame number, date time, time code)."),
                ifd0Id, dngTags, asciiString, -1, printValue), // DNG tag
        TagInfo(0xc7a1, "CameraLabel", N_("CameraLabel"),
                N_("The optional CameraLabel tag shall specify a text label " 
                "for how the camera is used or assigned in this clip. " 
                "This tag is similar to CameraLabel in XMP."),
                ifd0Id, dngTags, asciiString, -1, printValue), // DNG tag
I hope you'll understand that I can't drop everything on which I am working to research and attend to this.

#13 Updated by D Anderson 12 months ago

No problem. This is open source nature. Take your time. I see what I can come up to meanwhile regarding the recommended cdng list.
Just got the clang error back again. No idea what happened. Paths are probably wrong again. See what I can do about it. The command solutions for static compiling won,t work now. Go figure. Computers...

#14 Updated by Robin Mills 12 months ago

I've added you patch: r4291

I doesn't seem to be effective. I'll have to step the code in the debugger to discover why it isn't effective.

574 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -vVg svn
exiv2 0.25 001900 (64 bit build)
svn=4291
575 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pa  ~/Downloads/M*2.dng | grep -i scale
576 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -v -M"add Exif.SubImage1.DefaultScale Rational 1/1 5/3" ~/Downloads/M*2.dng
File 1/1: /Users/rmills/Downloads/M28-1507_1_2016-04-27_0001_C0000_000002.dng
Add Exif.SubImage1.DefaultScale "1/1 5/3" (Rational)
577 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pa  ~/Downloads/M*2.dng | grep -i scale
578 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pR  ~/Downloads/M*2.dng | grep -i scale
579 rmills@rmillsmbp:~/gnu/exiv2/ttt $ 
To answer your question "can I tell if your patch makes the tag writeable?". Sadly, no. I didn't write the code. Our wonderful Makernotes Engineer has returned to college, in addition to his full-time demanding job, and I have taken on responsibility for this part of the code. I'm on a learning curve and haven't investigated this part of the code yet.

My catch-all with build issues, is to cleanup and rebuild:

sudo make distclean ; make config ; ./configure ; make -j ;sudo make install ; make -j samples ; make tests

#15 Updated by Robin Mills 12 months ago

EUREKA

It's working:

581 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -v -M"add Exif.Image.DefaultScale Rational 1/1 5/3" ~/Downloads/M*2.dng
File 1/1: /Users/rmills/Downloads/M28-1507_1_2016-04-27_0001_C0000_000002.dng
Add Exif.Image.DefaultScale "1/1 5/3" (Rational)
582 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pa --grep FrameRate --grep DefaultScale ~/Downloads/M*2.dng 
Exif.Image.DefaultScale                      Rational    2  1/1 5/3
Exif.Image.FrameRate                         SRational   1  2997/50
583 rmills@rmillsmbp:~/gnu/exiv2/ttt $ 
What's not working is Exif.SubImage1.DefaultScale. I think you might be trying to update a sub image of the file and that's got something to do with multi-page tiff images and is discussed here: #1124

About "the open source way". I think the support for Exiv2 is very good. Out-of-sight better than any commercial company. However, as you know, software is a tough master. Exiv2 is rich, robust and reliable. Does it have every possible feature that could be added? Of course not. Could we do with more contributors and engineers? You know the answer! Would you like to volunteer to get involved?

#16 Updated by D Anderson 12 months ago

I can try to focus on contributing around the cdng tags and see what I can come up to regarding to the tags. Been working on a open source script workflow app at Magic lantern the last year but it,s pretty much selfgoing atm so could be nice to do something else. Prepare for questions ;).
And of course, it can only happen when I have the time.

#17 Updated by Robin Mills 12 months ago

I'll be happy to accept your help. And I'll be happy to mentor you and answer questions. Unless you want to urgently work on this, can we leave this for a few months? I want to be v0.26 code-complete in June. And we have major building project going on at home at present. Life will be less hectic by August.

However, if you want to get going sooner, I will answer your questions. You can email me directly at if you wish.

#18 Updated by D Anderson 12 months ago

No probs. No rush.
Tried your altered default scale tag which worked. 5 times on the same dng ;).

Exif.Image.DefaultScale                      Rational    2  1/1 5/3
Exif.Image.DefaultScale                      Rational    2  1/1 5/3
Exif.Image.DefaultScale                      Rational    2  1/1 5/3
Exif.Image.DefaultScale                      Rational    2  1/1 5/3
Exif.Image.DefaultScale                      Rational    2  1/1 5/3
Exif.Image.DefaultScale                      Rational    2  1/1 5/3
Exif.Image.FrameRate

Don,t think ACR liked the new location in metadata cause picture remains unstretched. You can however leave it, I keep searching and digging and get back if I find something intriguing. Thanks a lot for swift feedback. And thanks Alan Pater.

#19 Updated by D Anderson 12 months ago

For clearance. Picture remained "stretched" due to ACR not finding the new default scale location.

#20 Updated by Robin Mills 12 months ago

D:

I think you might need to pull the tiff to pieces with libtiff utility tiffsplit, update the appropriate sub-image and reassemble the image libtiff utility tiffcpt. Not pretty. I discussed this in #1124.

I also thank Alan for contributing to this an many other issues. Alan's worth his weight in gold (or so I believe).

What is your name? Nobody's call D !

#21 Updated by D Anderson 12 months ago

It,s Daniel.
By the way. Can I find source builds from around 2014? Seems at some point it was working. Could check if I can find where it breaks?

#22 Updated by Robin Mills 12 months ago

Daniel

I'm afraid not. Regular daily builds only start in December 2015. However, you don't need to bother testing because:

Exif.Image.DefaultScale works in v0.23 and v0.24. I'm not sure that r4291 (today at lunch-time) makes any difference.
Exif.SubImage1.DefaultScale has never worked and still doesn't work.

Andreas added "DefaultScale" in r1564.

610 rmills@rmillsmbp:~/gnu/exiv2/ttt $ svn blame src/tags.cpp | grep DefaultScale
  1564    ahuggel         TagInfo(0xc61e, "DefaultScale", N_("Default Scale"),
  1564    ahuggel                 N_("DefaultScale is required for cameras with non-square pixels. It " 
  1564    ahuggel                    "DefaultScale has been applied), relative to the top-left corner of " 
  1564    ahuggel                    "DefaultScale has been applied)."),
  1564    ahuggel                    "of the DefaultScale tag need to be multiplied to achieve the best " 
  4291 robinwmill         TagInfo(0xc61e, "DefaultScale", N_("Default Scale"),
  4291 robinwmill                 N_("DefaultScale is required for cameras with non-square pixels. It " 
611 rmills@rmillsmbp:~/gnu/exiv2/ttt $ svn log src/tags.cpp | grep 1564
r1564 | ahuggel | 2008-08-17 16:18:33 +0100 (Sun, 17 Aug 2008) | 1 line
612 rmills@rmillsmbp:~/gnu/exiv2/ttt $ 

For your information:
The buildserver was purchased in August 2014 and was pushed into service during v0.25 development for building branches on code submission. These build are performed as native by which I mean they use ./configure on *ix systems and msvc2005 for MSVC.

The major build project during v0.26 development has been CMake/Visual Studio. The buildserver builds every night about 2am in England using CMake (on 16 platforms).

#23 Updated by D Anderson 12 months ago

This is only if curious. Here is the static build which will write Exif.SubImage1.DefaultScale tag. I found it here.
http://www.l-camera-forum.com/topic/226978-tool-to-remove-dngraw-software-lens-correction-instructions/
It,s supposed to remove opcodes but inside the package you,ll find the 2014 exiv2 version. I guess I have to dig in to more recent build to check what,s happening.

#24 Updated by Robin Mills 12 months ago

Daniel

I'm going to leave you to get on with the digging. I have to get on with my house building project. I'm digging up the old kitchen floor at the moment. Tough work with a hammer and chisel. A lot like software actually. My wife is upstairs painting the bedroom! We are retired and renovating our house from top to bottom.

I just had the Exiv2 build go totally crazy and complain about this x86 stuff, even after $ sudo make distclean . My remedy is to checkout a fresh copy of the code and build that!

$ svn checkout svn://dev.exiv2.org/svn/trunk ttt | wc ; cd ttt ; make config ; ./configure ; make -j ; sudo make install ; make -j samples ; make tests 
I believe checking out a new copy of the code is normal practice for git warriors!

#25 Updated by Robin Mills 12 months ago

  • Status changed from Assigned to Closed

Also available in: Atom PDF

Redmine Appliance - Powered by TurnKey Linux