Feature #1153
Sony ILCE-6000 + Sony E 50mm F1.8 OSS .JPG files without lens model.
100%
Description
Hey!
The lens model is recognized from .ARW, but not in .jpg.(Files from camera, camera in RAW+JPG mode)
Exiftool is able to recognize the lens model in the .jpg.
The exact .jpg is attached.
LOG from exiv2 (0.25) and exiftool(9.46):
tim@tim-ESPRIMO-Mobile-U9210 ~/Bilder/Fotosammlung/2016/01/Mezaparks $ exiv2 -pt DSC01769.ARW | grep -ai lens
Exif.Sony2.LensID Long 1 Manual lens
Exif.Photo.LensSpecification Rational 4 500/10 500/10 18/10 18/10
Exif.Photo.LensModel Ascii 16 E 50mm F1.8 OSS
tim@tim-ESPRIMO-Mobile-U9210 ~/Bilder/Fotosammlung/2016/01/Mezaparks $ exiv2 -pt DSC01769.JPG | grep -ai lens
Error: Offset of directory Sony1, entry 0x2001 is out of bounds: Offset = 0x003ec076; truncating the entry
Exif.Sony1.LensID Long 1 Manual lens
tim@tim-ESPRIMO-Mobile-U9210 ~/Bilder/Fotosammlung/2016/01/Mezaparks $ exiftool DSC01769.JPG | grep -ai lens
Lens Type : E-Mount, T-Mount, Other Lens or no lens
Lens Spec : Unknown (e8 300 1 0 0 00)
Lens Type 2 : Unknown (24576)
Lens Mount : Unknown
Lens Format : Unknown
Lens ID : Sony E 50mm F1.8 OSS
Files
Related issues
Associated revisions
#1153 Fixes for "Sony E PZ 16-50mm F3.5-5.6 OSS"
#1153 test files
#1153. Added missing file to test suite
History
Updated by Robin Mills almost 6 years ago
- Category set to lens
- Status changed from New to Assigned
- Assignee set to Robin Mills
- Target version set to 0.26
- % Done changed from 0 to 20
- Estimated time set to 4.00 h
Tim
Please ignore the message: Error: Offset of directory Sony1, entry 0x2001 is out of bounds: Offset = 0x003ec076; truncating the entry
I dealt with that recently: #1145 so I don't see that message when I run your command on a build from the trunk.
There is information in the metadata of the ARW:
$ exiv2 -pt DSC01769.ARW | grep -ai lens Exif.Sony2.LensID Long 1 Manual lens Exif.Photo.LensSpecification Rational 4 500/10 500/10 18/10 18/10 Exif.Photo.LensModel Ascii 16 E 50mm F1.8 OSSthat has been removed in the JPG:
$ exiv2 -pt DSC01769.JPG | grep -ai lens Exif.Sony1.LensID Long 1 Manual lensDid the JPG come from the camera, or was it created by desktop software? I am wondering "who removed Exif.Photo.LensSepcification" and "Exif.Photo.LensModel" ?
What is the LensID in the .ARW? Can you share the .ARW with me, or run the following command:
$ exiv2 -pv --grep Lens DSC01769.ARW
I can modify the lens detection code to examine the following and deduce the lens:
$ exiv2 -pv --grep lens/i --grep apertu/i --grep focal/i --grep model/i http://dev.exiv2.org/attachments/download/882/DSC01769.JPG 0x0110 Image Model Ascii 10 ILCE-6000 0x9205 Photo MaxApertureValue Rational 1 434/256 0x920a Photo FocalLength Rational 1 500/10 0xb001 Sony1 SonyModelID Short 1 312 0xb027 Sony1 LensID Long 1 65535 0xa405 Photo FocalLengthIn35mmFilm Short 1 75 0x0110 Thumbnail Model Ascii 10 ILCE-6000 $I had another idea (which didn't work). I copied the Lens Specification from the .ARW to the .JPG with:
exiv2 -M"set Exif.Photo.LensSpecification 500/10 500/10 18/10 18/10" DSC01769.JPGThat updated the file, however we still reports LensID == -1 as "Manual Lens" (and ignores the LensSpecification data).
So four questions:
1) Can you share the ARW with me please (attach it, or dropbox it, or something)
2) Can you run the command: exiv2 -pv DSC01769.ARW
3) Did the JPG come from the Camera or Desktop Software?
4) Can you check with Sony for a firmware upgrade for your camera. The user on #1145 believes that a future firmware upgrade may fix this. I think you may able to detect the firmware version with a command such as:
$ exiv2 -pa --grep firm/i DSC01769.ARW
Updated by Anonymous almost 6 years ago
- File DSC01769.ARW.7z DSC01769.ARW.7z added
Robin Mills wrote:
So four questions:
1) Can you share the ARW with me please (attach it, or dropbox it, or something)
2) Can you run the command:exiv2 -pv DSC01769.ARW
3) Did the JPG come from the Camera or Desktop Software?
4) Can you check with Sony for a firmware upgrade for your camera.
1) Attached the ARW file compressed in .7z
2) http://pastebin.com/9Y3T2ENK
3) It came straight from the Camera by using RAW+JPG mode. I use Geeqie + Darktable. Darktable doesn’t change files, and I don’t change anything when using Geeqie
4) My ILCE-6000 is on Version 2.00 which is also the newest firmware.
Updated by Robin Mills almost 6 years ago
Thanks for sending this. It's 11pm in England. I'll look at this on Tuesday and I hope to get it fixed. I'd like to extract the metadata from your image and use it in our test suite as a regression detector. If you can provide more images (at different focal lengths), that would be very helpful.
Geeqie was mention in an issue yesterday: #1151. I thought it was a spelling mistake. Apparently not. #1151 is unlikely to be resolved without more information. If you know Geeqie and understand #1151, I'd appreciate any assistance you can give. http://dev.exiv2.org/issues/1151
Updated by Anonymous almost 6 years ago
Robin Mills wrote:
Thanks for sending this. It's 11pm in England. I'll look at this on Tuesday and I hope to get it fixed. I'd like to extract the metadata from your image and use it in our test suite as a regression detector. If you can provide more images (at different focal lengths), that would be very helpful.
Geeqie was mention in an issue yesterday: #1151. I thought it was a spelling mistake. Apparently not. #1151 is unlikely to be resolved without more information. If you know Geeqie and understand #1151, I'd appreciate any assistance you can give. http://dev.exiv2.org/issues/1151
Thanks for the support!
The used lens is a prime lens ;) But I could provide sample shots with the kit lens (16-55mm). Which focal lengths? ARW or JPG or both? Same behaviour for the kit lens; exiftool recognizes it properly.
About #1151: Resolution is not a problem. Geeqie is not a RAW converter and simply displays the JPG thumbnail in RAW files. For my files the cameras .JPGs are 6000x4000 whereas the thumbnails in RAW files get displayed in Geeqie with 1616x1080. Everything working perfect. (exiv2 0.23 + geeqie 1.1 http://geeqie.sourceforge.net/)
You should try Geeqie ;) Very very fast and powerful image viewer with features like RAW+JPG+XMP file grouping, color management by .icc profiles, etc!
Updated by Robin Mills almost 6 years ago
- Status changed from Assigned to Resolved
- % Done changed from 20 to 80
Tim
Fix submitted: r4181. Thanks for providing the test data. I've extracted the metadata from DSC01769.JPG and added it as test/data/exiv2-1153.exv to our test suite.
Thanks for offering to provide more test data. Let me explain how the fix works and then you'll understand what kind of test file would be helpful. There's a long discussion about this in #1145 which I won't repeat here. Every camera and lens should have a unique ID.
1284 rmills@rmillsmbp:~/gnu/exiv2/ttt $ exiv2 -pv --grep id/i http://dev.exiv2.org/attachments/download/882/DSC01769.JPG 0xb001 Sony1 SonyModelID Short 1 312 0xb027 Sony1 LensID Long 1 65535 1285 rmills@rmillsmbp:~/gnu/exiv2/ttt $
Exif.Sony1.SonyModelID == 312 ==> ILCE-6000 Good.LensID == 65535 == 0xffff == "Manual Lens". Not good.
LensID is not unique to your lens. In fact, 0xffff says "I don't know anything about the lens!". So the code then inspects FocalLength and MaxAperture to say "Could this be Tim's Lens?". You can read the code to see how I do this. http://dev.exiv2.org/projects/exiv2/repository/revisions/4181/diff/trunk/src/minoltamn.cpp
This is pure guesswork. It's quite likely that Sony will upgrade their firmware to provide a unique LensID and you'll be back to say "Fix r4181 isn't working any more.".
Exiv2 and ExifTool are tramping the same ground. I'm working on a cunning plan to enable Phil and I to discover and share knowledge about new lens/cameras. Work in progress: #1034.
And thanks for providing the .ARW. There are lots of differences in the Metadata between the .JPG and the .ARW. I've been working on Exiv2 for almost 8 years, however I only started supporting MakerNotes a few months ago. Niels (our wonderful MakerNote Engineer) has gone back to college (while continuing his very demanding day job) and I've taken on support for the MakerNote code. I'm on a learning curve and one day I'll will understand all those differences. Very illuminating.
Thanks for your help with #1151. You can read my comments in that issue, so I won't repeat them here. I couldn't build geeqie on MacOS-X because I don't have GTK+, however I installed it on kubuntu. For sure it's fast. I use Picasa to manage my photo library on the Mac. I think it's possible that Wil, who reported #1151, has discovered something here. However, I want the geeqie team to investigate and, if necessary, raise an issue with us. I believe our thumbnail/preview code is working well. As I don't know geeqie, I don't know what to expect when working with DNG and RAW files.
I've updated this issue to say "80% resolved". Can you test with other cameras/lens combinations and let me know if you discover other stuff that requires attention.
Updated by Anonymous almost 6 years ago
Thank you!
So it basically checks for camera model + settings/capabilities of the lens and then checks what is the most likely lens? Sounds like a mess... So there is no other Tag? Does Exiftool use a similar strategy?
Just think of the electronic adapters for Canon/Nikon lenses which also support most of the settings.. Those would normally not leave a trace of the name in the EXIF, but will maybe be recognized false positive.
I also tried the patch. I downloaded trunk, but as there was no .configure, so I got that from 0.25 and compiled with that. Same like before, the Lens Model is still not listed.
Was the recognition successful for you with my sample files?
Updated by Anonymous almost 6 years ago
I could just point out to the exiftool documentation for sony. Maybe this helps you understanding how Sony tags work. I don't get anything honestly speaking :D
http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/Sony.html
Updated by Robin Mills almost 6 years ago
Tim
To build use:
$ make config ; ./configure ; make -j ; sudo make install ; make -j samples ; make tests
$ make config generates the ./configure script. It's been this way since before I joined the project. http://dev.exiv2.org/projects/exiv2/wiki/How_do_I_build_Exiv2_on_the_XYZ_platform
I agree that this style of Lens recognition is hackery. When LensID is unambiguously defined, we don't need that nonsense. I got this method from Phil himself. There's his code: http://dev.exiv2.org/issues/1143#note-11 Could you convert that into C++ please?
Here's the output from the trunk:
$ exiv2 -pa --grep lens/i http://dev.exiv2.org/attachments/download/882/DSC01769.JPG Exif.Sony1.LensID Long 1 Sony E 50mm F1.8 OSS $I suspect the scenario is that your camera's firmware predates the lens and thinks "Don't know what lens this is, so I'll set LensID=0xffff" However, it's quite likely that a firmware upgrade to your camera will set LensID correctly and unambiguously.
Uncle Robin and Uncle Phil have to fixed for it for now. A simple "Thank You, Robin and Phil" would be appreciated.
Updated by Anonymous almost 6 years ago
Aaahh.. Im sorry. I used -pv instead of -pa :/
Works good for me now!
About the thank you: I said that before, but maybe it didn’t come over in the right way as I had further questions.
So here I go: Thank You, Robin and Phil! :) I really appreciate the hard work you are putting into Tags, making life easier for people :)
I had another look at the code and checked pictures of my kit lens, to understand what the code is doing.
It is calculating a number from the used focal length, taking the APS-C sensor in account by also using the 35mm equivalent and this focal length max aperture on this focal length.
That means that I have to provide: The steps of MaxAperture and the range of focal length it is available for.
Did I understand that correct?
Updated by Anonymous almost 6 years ago
Ok. So I figured now that the calculation is always 150, as it just recalculates the crop factor, doesn’t it? :D
So I still don’t understand it :/
But I created a table with all the F-Stops for "Sony E PZ 16-50mm F3.5-5.6 OSS":
MaxApertureValue: FocalLength(FocalLengthIn35mmFilm)
F3.5: 16mm(24mm)-18mm(27mm)
F4 : 18mm(27mm)-23mm(34mm)
F4.5: 24mm(36mm)-29mm(43mm)
F5 : 30mm(45mm)-34mm(51mm)
F5.6: 35mm(52mm)-50mm(75mm)
I will upload all the samples, but I have a very poor upload, so it will take the night :D
Updated by Robin Mills almost 6 years ago
Thanks for the files which will arrive overnight. I'm off to bed (it's 11:36pm). I'll look at them tomorrow.
It think the ratio will always be 1.5, however there could be a little rounding - so I test 1.48 .. 1.52 (but not in floating point). I throw in a 100 multiplier to use range 148 .. 152
Yes. I know you said thanks up front. However the pissing and moaning about the solution blunted your appreciation!
The bummer in all of this is that it depends on the Camera/Lens combination what must be computed to make an intelligent guess. MaxAperture and FocalLength are important ingredients in the "heuristic" code. However, you'll see that he calculates log-log stuff at times! Non trivial.
Here are my current thoughts about how to fix this (in #1034)
1) I'm going to have a web service running exiftool.
2) When exiv2 doesn't recognise the lens, I'll generate a .exv file (metadata only JPG) and send it to the web service.
3) If exiftool doesn't recognise it, I'll log that. It's something new (however we have a metadata file for analysis)
4) If exiftool recognises it, I'll log that. I can analyse the metadata and update our C++ code
Other ideas that I am not pursuing:
1: Rewrite the Perl code in C++
Not impossible. Not trivial.
2: Execute exiftool on the user's computer
I think that will throw up a DOS box on Windows
3: To write the Lens recognition in a scripting language that can be executed by exiv2.
A sufficiently rich PostScript or Lisp interpreter/run-time can be implemented in 1000-2000 lines of code
The scripting code would be in libexiv2.14.script in the same directory as libexiv2.14.so
The scripting code could be modified without recompiling libexiv2.14.so
The scripting code would only run when the C++ is unable to recognise the lens, so it wouldn't impose much overhead
4: To have an SQLite database of lens/camera metadata
So the code would ask the database (stored in libexiv2.14.sqlite
SELECT "LensType when LensID=0xffff and Camera_Model=ILCE-6000 bla bla"
If it fails, it could use exiftool as a web service and update the database on the local computer
5: To have an 'overflow' file which is consulted when the lens isn't recognised
We didn't make progress beyond arguing about the language of that file (XML, Json, CSV)
We made no progress on what should be in the 'overflow' file
Please remember, we're only doing this when the LensID is not uniquely set. It's the Camera/Lens manufacturers who have donated this issue to their customers. As always with firmware/hardware issues, it's the software people who have to clean up and endure the criticism for the deficiency of the firmware.
Thanks again for your help with geeqie (#1151). I haven't heard from Wil since Sunday. I suspect that will close with "cannot reproduce/insufficient information".
Updated by Anonymous almost 6 years ago
Updated by Robin Mills almost 6 years ago
Fix submitted: r4182. Test output below. I've extracted the metadata from all your test files (both .ARW and .JPG) and added them to our test suite.
Original test files submitted: r4183. I copied your original test files to svn://dev.exiv2.org/svn/testdata/trunk/1153 for future use. I don't expect to visit those file again as we have the metadata in the test suite. To constrain the size of the test suite, we avoid adding large JPG and RAW files to code branches. Thank You for sharing those files. You may remove your .7z file from Dropbox.
There is conflict concerning the naming of this lens. The .ARW (1775-1784) files contain the Ascii string "E 50mm F1.8 OSS" and I've adopted that name for the associated JPG files. Similarly, 1769 is consistently named "Sony E 50mm F1.8 OSS" by exiv2.
1590 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep Lens/i ../testdata/trunk/1153/*.ARW ../testdata/trunk/1153/DSC01769.ARW Exif.Sony2.LensID Long 1 Sony E 50mm F1.8 OSS ../testdata/trunk/1153/DSC01769.ARW Exif.Photo.LensSpecification Rational 4 500/10 500/10 18/10 18/10 ../testdata/trunk/1153/DSC01769.ARW Exif.Photo.LensModel Ascii 16 E 50mm F1.8 OSS ../testdata/trunk/1153/DSC01775.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01775.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01775.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01776.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01776.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01776.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01777.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01777.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01777.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01778.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01778.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01778.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01779.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01779.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01779.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01780.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01780.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01780.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01781.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01781.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01781.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01782.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01782.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01782.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01783.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01783.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01783.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01784.ARW Exif.Sony2.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01784.ARW Exif.Photo.LensSpecification Rational 4 160/10 500/10 35/10 56/10 ../testdata/trunk/1153/DSC01784.ARW Exif.Photo.LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS 1591 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep Lens/i ../testdata/trunk/1153/*.JPG ../testdata/trunk/1153/DSC01769.JPG Exif.Sony1.LensID Long 1 Sony E 50mm F1.8 OSS ../testdata/trunk/1153/DSC01775.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01776.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01777.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01778.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01779.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01780.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01781.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01782.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01783.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS ../testdata/trunk/1153/DSC01784.JPG Exif.Sony1.LensID Long 1 E PZ 16-50mm F3.5-5.6 OSS 1592 rmills@rmillsmbp:~/gnu/exiv2/trunk $
Updated by Robin Mills almost 6 years ago
- Tracker changed from Bug to Feature
- % Done changed from 80 to 100
- Estimated time changed from 4.00 h to 5.00 h
Updated by Anonymous almost 6 years ago
I had an exam, so just now had the time to test it. Thank you very much! Works perfectly! :)
If I get a new lens(could take some time.. testing out some old russian lenses recently; obviously without any electronics :D), should I open a new ticket or reuse this one?
Again, Thank you very much :)
Updated by Robin Mills almost 6 years ago
Tim: It's nice to work with you. You can update this ticket, or raise a new one. I'm sure you'll find camera/lens combinations that are broken. You know I will investigate and try to fix things.
Updated by Anonymous almost 6 years ago
Ok. I didnt expect such a quick update :D
Im using Darktable for RAW editing. Built the new exiv2 and tried, without sucess. I went to the darktable IRC to ask, and they said that I'll have to buid it against the new library. Now I didnt do that till now as some dependencies are missing for me, but I discovered that the .ARWs are tagged at Exif.Sony2.LensID and the JPGs at Exif.Sony1.LensID.
I asked if that is going to be a problem and houz replied:
[00:52] <houz> $ grep Sony exif.cc | grep Lens
[00:52] <houz> if(FIND_EXIF_TAG("Exif.Sony2.LensID") && pos->toLong() != 65535 && pos->print().find('|') == std::string::npos)
[00:52] <houz> we only look at Sony2
So I guess because of this and for consistency also the JPGs should get interpreted as Exif.Sony2.LensID.
Updated by Robin Mills almost 6 years ago
Gosh, Tim. It's 23:48 in England. I'll look at this tomorrow.
Updated by Robin Mills almost 6 years ago
Tim
I can't understand your message. I assume exif.cc is darktable code. I don't know the difference between Sony1 and Sony2 because I didn't write exiv2. I'm the project build engineer. However in your test files, the metadata in the .ARW files is Sony2 and in the .JPG files it is Sony1.
What is your question?
Updated by Robin Mills almost 6 years ago
Tim
Concerning your russian lenses. Simon recently asked about a couple of "Manual Lenses" and that has been discussed in #1141 and #1142 As you can see from the discussion, it's generally not possible to determine anything about those lenses. If you have one, look at metadata such as LensSpecification - if it's almost empty (almost entirely 0s), exiv2 can do nothing to help you. I believe it's possible to set the lensID in the metadata with exiv2 in every file in a directory with a command such as:
$ exiv2 -M'set Exif.Photo.LensModel my old Russian lens' *.jpgDedicated Linux Warriors will use
find to traverse trees of images.Updated by Anonymous almost 6 years ago
Robin Mills wrote:
What is your question?
In the .ARW files the lens is specified in the tag "Exif.Sony2.LensID". Now the recognition for the correct lens that you created here for .JPGs basically adds the missing tag of the lens as "Exif.Sony1.LensID". But why as "Exif.Sony1.LensID" when we could simply put it as "Exif.Sony2.LensID", so it has the lens name in the same tag as the .ARW files for consistency.
I assume exif.cc is darktable code
Correct. It is just looking for the "Exif.Sony2.LensID" tag. So a tag defined in "Exif.Sony1.LensID" will not show up as lens in darktable. I'm not sure what consequences an addition of "Exif.Sony1.LensID" for lens recognition would have for other sony users in darktable. I guess it has less impact if we define the lens for .JPGs in "Exif.Sony2.LensID".
Concerning your russian lenses.
That's why I wrote "obviously without any electronics" ;D But still thanks for the hint how I can do that manually!
Updated by Robin Mills almost 6 years ago
My guess about Sony1 and Sony2 is the MakerNote decoder. The Exif Spec allows the manufacturer to add a blob of binary data into the tag "MakerNote". We have reverse engineered that. I suspect the decoder for the .ARW is Sony2, and his JPG brother is Sony1. I'll have a read in the code and find out more.
Probably a better fix is for us to define the tag: Exif.Photo.LensModel from the Sony1 and Sony2 decoders. Then, the DarkTable guys should test that tag. I don't know what DarkTable is doing, however it shouldn't really have to know about Sony1 and Sony2 - that sounds a little odd.
Tim: I think I've said this before: I am the Exiv2 build engineer. Niels has been looking after the MakerNote code. I've only taken this on in the last few months. I'm a volunteer sitting at home. There's nobody here with whom to discuss any of this. However I have worked on the project for 8 years, so I know my way around the code and have solved many puzzles. Be gentle on me. I am not a guru. I'm a little old retired engineer. http://dev.exiv2.org/projects/exiv2/wiki/_Robin_Mills_
Updated by Alan Pater almost 6 years ago
My limited understanding is that there are 2 distinct Sony FirmWares. Some cameras use the Sony1 firmware, other models use Sony2. The same applies to other camera manufactures.
Updated by Robin Mills almost 6 years ago
Thanks for popping up here, Alan. I'll have a crawl through the code tomorrow. This metadata business is an enchanted garden. Around every corner lies something new and mysterious. When I extract the metadata from one of my favourite test files:
543 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep Lens/i http://clanmills.com/Stonehenge.jpg Exif.Nikon3.LensType Byte 1 D G VR Exif.Nikon3.Lens Rational 4 18-250mm F3.5-6.3 Exif.Nikon3.LensFStops Undefined 4 4.58333 Exif.NikonLd3.LensIDNumber Byte 1 Sigma 18-250mm F3.5-6.3 DC OS Macro HSM Exif.NikonLd3.LensFStops Byte 1 F4.6 544 rmills@rmillsmbp:~/gnu/exiv2/trunk $What is Nikon3 and NikonLd3? Ah ..... the metadata garden fairy is at work!
Updated by Robin Mills almost 6 years ago
- Status changed from Resolved to Closed
I'm closing this to help me to focus and concentrate on getting v0.26 finished on schedule.
Updated by Anonymous over 5 years ago
Hi!
Today an update for many Sony Cameras has been released. (http://esupport.sony.com/US/p/swu-download.pl?mdl=ILCE6000&upd_id=10716&os_group_id=29)
Excerpt from the changelog:
"- The lens name will be correctly recorded in the EXIF"
I will install this update, test it and report back. But it could turn out that the extra code you created may become obsolete.
Thank you very much for your efforts!
Updated by Robin Mills over 5 years ago
Tim
I'll be really happy if my code for lens recognition is obsolete. I'm a retired Adobe Senior Computer Scientist and my happiest projects have been compilers and interpreters. The compiler is a 100% accurate deterministic software machine. It doesn't guess anything. It converts source into code. I strongly dislike guessing anything. The lens should be 100% defined by the manufacturer in the metadata. Anything else is a bodge or hacking. Thanks for the update.
Updated by Anonymous over 5 years ago
- File DSC02521.JPG DSC02521.JPG added
The issue is fixed by Sony.
Firmware upgrade from 2.0 to 3.1 on A6000 fixes it.
I attached a .JPG for reference.
Again, thank you for the efforts you made!
Updated by Robin Mills over 5 years ago
You've made my day. Thanks for letting me know. And thanks for using exiv2.
Updated by Anonymous over 5 years ago
I actually have to revoke what I said. I guess back then I did not properly revert the exiv2 version and checked with the patched one.
I set up a new system and figured that I was wrong. Nothing changed.
So the patch should be inserted again :/
Thanks!
Updated by Robin Mills over 5 years ago
Tim. I can't compute your message. Do you want me to do something?
Updated by Anonymous over 5 years ago
Consider comment #28 as wrong. The issue was not fixed by Sony. The patch unfortunately is still required for correct lens recognition.
Updated by Robin Mills over 5 years ago
Right. I think you're saying that the Sony Firmware upgrade is not effective and that we should retain patch r4181/r4182/r4208 in the exiv2 code base. That's fine. Thanks for updating this issue with this information.
Updated by Anonymous over 5 years ago
It seems that the behavior is not the same for all lenses. I just tested some other lenses:
Here the results without the patch:
@exiv2 -pv --grep Lens /home/tim/Bilder/Fotosammlung/2016/07/27/DSC07171.JPG
0xb027 Sony1 LensID Long 1 65535
0xa432 Photo LensSpecification Rational 4 350/10 350/10 18/10 18/10
0xa434 Photo LensModel Ascii 16 E 35mm F1.8 OSS
tim@Tims-Laptop ~ $ exiv2 -pv --grep Lens /home/tim/Bilder/Fotosammlung/2016/07/27/DSC07178.JPG
0xb027 Sony1 LensID Long 1 65535
0xa432 Photo LensSpecification Rational 4 300/10 300/10 28/10 28/10
0xa434 Photo LensModel Ascii 12 E 30mm F2.8
tim@Tims-Laptop ~ $ exiv2 -pv --grep Lens /home/tim/Bilder/Fotosammlung/2016/07/27/DSC07177.JPG
0xb027 Sony1 LensID Long 1 65535
0xa432 Photo LensSpecification Rational 4 300/10 300/10 14/10 14/10
0xa434 Photo LensModel Ascii 35 30mm F1.4 DC DN | Contemporary 016
tim@Tims-Laptop ~ $ exiv2 -pv --grep Lens /home/tim/Bilder/Fotosammlung/2016/04/Burger/DSC03880.JPG
0xb027 Sony1 LensID Long 1 65535
tim@Tims-Laptop ~ $ exiv2 -pv --grep Lens /home/tim/Bilder/Fotosammlung/2016/04/Burger/DSC03880.ARW
0xb027 Sony2 LensID Long 1 65535
0xa432 Photo LensSpecification Rational 4 500/10 500/10 18/10 18/10
0xa434 Photo LensModel Ascii 16 E 50mm F1.8 OSS
exiv2 -pv --grep Lens /home/tim/Bilder/Fotosammlung/2016/05/Stipro\ Skrejiens/DSC04924.ARW
0xb027 Sony2 LensID Long 1 65535
0xa432 Photo LensSpecification Rational 4 160/10 500/10 35/10 56/10
0xa434 Photo LensModel Ascii 26 E PZ 16-50mm F3.5-5.6 OSS
tim@Tims-Laptop ~ $ exiv2 -pv --grep Lens /home/tim/Bilder/Fotosammlung/2016/05/Stipro\ Skrejiens/DSC04924.JPG
0xb027 Sony1 LensID Long 1 65535@
As you can see, the Lens is correctly detected in JPG files for photos with "Sony E 35mm F1.8 OSS", "Sigma 30mm F1.4 DC DN" and "Sigma E 30mm F2.8" (I also added the output of the 50mm and 16-50mm, so you see the difference). All photos were shot with A6000 with firmware 3.10. I hope that the patch you included does not interfere with the correct recognition of those lenses.
Thanks again!
Updated by Robin Mills over 5 years ago
I can't recall if we've had this conversation already. I really dislike the idea that Exiv2 does lens recognition. Our mission is to read/write/update meta data in images files. We support many camera manufacturers makernotes, several formats (Exif, IPTC and XMP) and many platforms.
I believe the Exif standard uses a LensID (and integer) and combined with the manufacturer's id, we can recognised the lens. However the manufacturers often use the same LensID for many different lenses. Exiv2 attempts to guess the lens by inspecting other metadata. I don't know how this feature was added to the library, however it's very time consuming to maintain this as new lenses are introduced.
I've added a new feature for v0.26 to use a file ~/.exiv2 (or /Users/username/exiv2.ini) to translate the lensID for a manufacturer into a lens definition. By updating that file on your computer, all applications using exiv2 will know your lens.
#1153 Fix submitted. Thank You, Tim for reporting this and providing test data.