Bug #1035
Lens model not detected ( exiv2 -> LensFun -> darktable )
100%
Description
Hi.
Background details:
1) http://www.darktable.org/redmine/issues/10331
2) https://sourceforge.net/p/lensfun/bugs/18/
Is there anything that can be done in exiv2 to fix this?
Thanks for your time.
Regards.
Files
Related issues
Associated revisions
History
Updated by Niels Kristian Bech Jensen almost 7 years ago
Sony makernote support is in serious need of an update to support E-mount lens detection amongst other things. I do not know when such an update will happen due to lack of time.
Best regards,
Niels Kristian Bech Jensen
Updated by Robin Mills over 6 years ago
- Category set to metadata
- Status changed from New to Assigned
- Assignee set to Robin Mills
- Target version set to 0.26
I'm going to accept this assignment for v0.26 as we can't consider this for v0.25. I'll have to investigate what's involved in this. So, all I am agreeing to do for v0.26 is consider the effort involved. If it's a lot of work, it might be pushed beyond v0.26.
Updated by Robin Mills over 6 years ago
- Target version changed from 0.25 to 0.26
Pushed back to v0.26. I don't why I moved this to v0.25 yesterday.
Updated by Robin Mills almost 6 years ago
- % Done changed from 0 to 10
- Estimated time set to 10.00 h
I'm adding a SWAG to my outstanding issues for v0.26 to get a more accurate assessment of the status of v0.26. I can't remember anything about this - so I'll guess about a day's work.
Updated by Robin Mills almost 6 years ago
- % Done changed from 10 to 50
- Estimated time changed from 10.00 h to 5.00 h
Good News, I believe I can fix this. It's almost identical to #1153 in which .ARW file correctly identifies the Lens and the .JPG file does not. Please attach the DSC00050.JPG (to match DSC00050.ARW) and I'll fix this.
It doesn't have to be DSC00050.JPG - however I need matched pairs of JPG/ARW for every camera/lens combination. It's also very useful to shoot ARW/JPG pairs at every aperture setting of the camera/lens combo.
The lens is correctly identified in the RAW image:
1678 rmills@rmillsmbp:~ $ exiv2 -pa DSC00050.ARW | grep -i lens Exif.Sony2.LensID Long 1 Manual lens Exif.Photo.LensSpecification Rational 4 190/10 190/10 28/10 28/10 Exif.Photo.LensModel Ascii 12 E 19mm F2.8 $However, the LensID is ambiguously set to 0xffff (65535).
1679 rmills@rmillsmbp:~ $ exiv2 -pv DSC00050.ARW | grep -i lens 0xb027 Sony2 LensID Long 1 65535 0xa432 Photo LensSpecification Rational 4 190/10 190/10 28/10 28/10 0xa434 Photo LensModel Ascii 12 E 19mm F2.8 $
Updated by Rodrigo De Leon almost 6 years ago
Hi.
Thanks for the feedback.
I no longer have the original DSC00050.JPG, but I can take a new set of photos and upload them.
Don't remember if this only happen with the 19mm or also with the 30mm, but I'll upload both sets.
Just give me a few minutes...
Updated by Rodrigo De Leon almost 6 years ago
Hi.
There seems to be a 20 MB file upload limit for this issue tracker, but all ARW files measure about 24-25 MB.
I'm gonna have to archive and split to be able to upload.
Hang on...
Updated by Rodrigo De Leon almost 6 years ago
- File 19mm.tar.gz_part_aa 19mm.tar.gz_part_aa added
- File 19mm.tar.gz_part_ab 19mm.tar.gz_part_ab added
- File 19mm.tar.gz_part_ac 19mm.tar.gz_part_ac added
- File 19mm.tar.gz_part_ad 19mm.tar.gz_part_ad added
- File 19mm.tar.gz_part_ae 19mm.tar.gz_part_ae added
- File 19mm.tar.gz_part_af 19mm.tar.gz_part_af added
- File 19mm.tar.gz_part_ag 19mm.tar.gz_part_ag added
- File 19mm.tar.gz_part_ah 19mm.tar.gz_part_ah added
- File 19mm.tar.gz_part_ai 19mm.tar.gz_part_ai added
- File 19mm.tar.gz_part_aj 19mm.tar.gz_part_aj added
Batch 1...
Updated by Rodrigo De Leon almost 6 years ago
- File 19mm.tar.gz_part_ak 19mm.tar.gz_part_ak added
- File 19mm.tar.gz_part_al 19mm.tar.gz_part_al added
- File 19mm.tar.gz_part_am 19mm.tar.gz_part_am added
- File 19mm.tar.gz_part_an 19mm.tar.gz_part_an added
- File 19mm.tar.gz_part_ao 19mm.tar.gz_part_ao added
- File 19mm.tar.gz_part_ap 19mm.tar.gz_part_ap added
- File 19mm.tar.gz_part_aq 19mm.tar.gz_part_aq added
- File 19mm.tar.gz_part_ar 19mm.tar.gz_part_ar added
- File 19mm.tar.gz_part_as 19mm.tar.gz_part_as added
- File 19mm.tar.gz_part_at 19mm.tar.gz_part_at added
Batch 2...
Updated by Rodrigo De Leon almost 6 years ago
- File 19mm.tar.gz_part_au 19mm.tar.gz_part_au added
- File 19mm.tar.gz_part_av 19mm.tar.gz_part_av added
- File 19mm.tar.gz_part_aw 19mm.tar.gz_part_aw added
- File 19mm.tar.gz_part_ax 19mm.tar.gz_part_ax added
- File 19mm.tar.gz_part_ay 19mm.tar.gz_part_ay added
- File 19mm.tar.gz_part_az 19mm.tar.gz_part_az added
- File 19mm.tar.gz_part_ba 19mm.tar.gz_part_ba added
- File 19mm.tar.gz_part_bb 19mm.tar.gz_part_bb added
- File 19mm.tar.gz_part_bc 19mm.tar.gz_part_bc added
Batch 3...
Updated by Rodrigo De Leon almost 6 years ago
- File 30mm.tar.gz_part_aa 30mm.tar.gz_part_aa added
- File 30mm.tar.gz_part_ab 30mm.tar.gz_part_ab added
- File 30mm.tar.gz_part_ac 30mm.tar.gz_part_ac added
- File 30mm.tar.gz_part_ad 30mm.tar.gz_part_ad added
- File 30mm.tar.gz_part_ae 30mm.tar.gz_part_ae added
- File 30mm.tar.gz_part_af 30mm.tar.gz_part_af added
- File 30mm.tar.gz_part_ag 30mm.tar.gz_part_ag added
- File 30mm.tar.gz_part_ah 30mm.tar.gz_part_ah added
- File 30mm.tar.gz_part_ai 30mm.tar.gz_part_ai added
- File 30mm.tar.gz_part_aj 30mm.tar.gz_part_aj added
Batch 4...
Updated by Rodrigo De Leon almost 6 years ago
- File 30mm.tar.gz_part_ak 30mm.tar.gz_part_ak added
- File 30mm.tar.gz_part_al 30mm.tar.gz_part_al added
- File 30mm.tar.gz_part_am 30mm.tar.gz_part_am added
- File 30mm.tar.gz_part_an 30mm.tar.gz_part_an added
- File 30mm.tar.gz_part_ao 30mm.tar.gz_part_ao added
- File 30mm.tar.gz_part_ap 30mm.tar.gz_part_ap added
- File 30mm.tar.gz_part_aq 30mm.tar.gz_part_aq added
- File 30mm.tar.gz_part_ar 30mm.tar.gz_part_ar added
- File 30mm.tar.gz_part_as 30mm.tar.gz_part_as added
- File 30mm.tar.gz_part_at 30mm.tar.gz_part_at added
Batch 5...
Updated by Rodrigo De Leon almost 6 years ago
- File 30mm.tar.gz_part_au 30mm.tar.gz_part_au added
- File 30mm.tar.gz_part_av 30mm.tar.gz_part_av added
- File 30mm.tar.gz_part_aw 30mm.tar.gz_part_aw added
- File 30mm.tar.gz_part_ax 30mm.tar.gz_part_ax added
- File 30mm.tar.gz_part_ay 30mm.tar.gz_part_ay added
- File 30mm.tar.gz_part_az 30mm.tar.gz_part_az added
- File 30mm.tar.gz_part_ba 30mm.tar.gz_part_ba added
- File 30mm.tar.gz_part_bb 30mm.tar.gz_part_bb added
- File 30mm.tar.gz_part_bc 30mm.tar.gz_part_bc added
Batch 6 (final)...
Updated by Rodrigo De Leon almost 6 years ago
Done.
Let me know if you need something else.
Cheers.
Updated by Robin Mills almost 6 years ago
Thanks, Rodrigo. It's 23:03 in England. I'm off to bed. I'll look at this tomorrow.
Updated by Robin Mills almost 6 years ago
Apologies, Rodrigo. It's been a very busy day and I didn't get to your issue. I will deal with it on Saturday.
I had another thought however! Can you get your camera to take lower resolution photos which are within the 15mb limit? After all, I'm only interested in the 50k of metadata - the rest of the file isn't useful!
Updated by Robin Mills almost 6 years ago
- % Done changed from 50 to 70
Rodrigo
Thanks for the files. I've got them, unzipped them and submitted them to our test file repository: svn://dev.exiv2.org/svn/testdata/trunk/1035
What is your expectation here? We are detecting Exif.Photo.LensModel as:
520 rmills@rmillsmbp:~/gnu/exiv2/testdata/trunk/1035 $ exiv2 -pa -K Exif.Photo.LensModel *.JPG DSC05005.JPG Exif.Photo.LensModel Ascii 12 E 30mm F2.8 ... DSC05043.JPG Exif.Photo.LensModel Ascii 12 E 19mm F2.8 521 rmills@rmillsmbp:~/gnu/exiv2/testdata/trunk/1035 $ exiv2 -pa -K Exif.Photo.LensModel *.ARW DSC05005.ARW Exif.Photo.LensModel Ascii 12 E 30mm F2.8 ... DSC05043.ARW Exif.Photo.LensModel Ascii 12 E 19mm F2.8 522 rmills@rmillsmbp:~/gnu/exiv2/testdata/trunk/1035 $I see exiftool reports it's a Sigma Lens:
$ for i in DSC05043.*; do echo --- $i ---- ; (exiftool -all $i| grep -i Lens) ; done --- DSC05043.ARW ---- Lens Type : E-Mount, T-Mount, Other Lens or no lens Lens Spec : E 19mm F2.8 Lens Zoom Position : 0% Lens Mount 2 : E-mount Lens Type 3 : Unknown E-mount lens or other lens Lens E-mount Version : 1.00 Lens Firmware Version : Ver.01 Lens Mount : E-mount Lens Format : APS-C Lens Type 2 : Unknown E-mount lens or other lens Lens Spec Features : E Lens Info : 19mm f/2.8 Lens Model : E 19mm F2.8 Lens ID : Sigma 19mm F2.8 [EX] DN --- DSC05043.JPG ---- Lens Type : E-Mount, T-Mount, Other Lens or no lens Lens Spec : E 19mm F2.8 Lens Zoom Position : 0% Lens Mount 2 : E-mount Lens Type 3 : Unknown E-mount lens or other lens Lens E-mount Version : 1.00 Lens Firmware Version : Ver.01 Lens Mount : E-mount Lens Format : APS-C Lens Type 2 : Unknown E-mount lens or other lens Lens Spec Features : E Lens Info : 19mm f/2.8 Lens Model : E 19mm F2.8 Lens ID : Sigma 19mm F2.8 [EX] DN 524 rmills@rmillsmbp:~/gnu/exiv2/testdata/trunk/1035 $For sure, the 'Sigma' is not in the metadata:
527 rmills@rmillsmbp:~/gnu/exiv2/testdata/trunk/1035 $ exiv2 -pa --grep lens/i DSC05043.* DSC05043.ARW Exif.Sony2.LensID Long 1 Manual lens DSC05043.ARW Exif.Photo.LensSpecification Rational 4 190/10 190/10 28/10 28/10 DSC05043.ARW Exif.Photo.LensModel Ascii 12 E 19mm F2.8 DSC05043.JPG Exif.Sony1.LensID Long 1 Manual lens DSC05043.JPG Exif.Photo.LensSpecification Rational 4 190/10 190/10 28/10 28/10 DSC05043.JPG Exif.Photo.LensModel Ascii 12 E 19mm F2.8 528 rmills@rmillsmbp:~/gnu/exiv2/testdata/trunk/1035 $However it could be deduced from other metadata such as the crop factor (or the LensSpecification above):
530 rmills@rmillsmbp:~/gnu/exiv2/testdata/trunk/1035 $ exiv2 -pa --grep focal/i DSC05043.* DSC05043.ARW Exif.Photo.FocalLength Rational 1 19.0 mm DSC05043.ARW Exif.Photo.FocalLengthIn35mmFilm Short 1 28.0 mm DSC05043.JPG Exif.Photo.FocalLength Rational 1 19.0 mm DSC05043.JPG Exif.Photo.FocalLengthIn35mmFilm Short 1 28.0 mm 531 rmills@rmillsmbp:~/gnu/exiv2/testdata/trunk/1035 $I've never used DarkTable nor LensFun, so I don't understand the information you reported to them. I've checked back a few years to exiv2 v0.23 (about 2012 or so). exiv2 v0.23 reports the same lens data as the current trunk - so upgrading the version of in DarkTable and/or LensFun will change nothing.
I've looked at the exiftool lens identification code and I don't know how then discovered it was a Sigma Lens, I can put more effort into that - or cheat and ask Phil! The (perl) code is here: http://dev.exiv2.org/issues/1143#note-11
However, before I roll up my sleeves, or involve anybody else, I'd like to know what you are looking for here.
Updated by Rodrigo De Leon almost 6 years ago
Hi.
Really sorry for the late update.
OK, to summarize:
- darktable is a raw photo developer, similar to lightroom, capture one, etc.
- they use exiv2 for lens identification and lensfun for lens correction (distortion, TCA, vignetting). Detailed info here:
- they match lens name to (apparently) normalized exiv2 names, list here:
- for the 19mm, 30mm and 60mm Sigma lenses, there seem to be no Sony entries:
- there seem to be valid entries for these lenses on the Olympus side, though:
- 19mm: http://dev.exiv2.org/projects/exiv2/repository/revisions/3048/entry/trunk/src/olympusmn.cpp#L1287
- 30mm: http://dev.exiv2.org/projects/exiv2/repository/revisions/3048/entry/trunk/src/olympusmn.cpp#L1285
- 60mm: http://dev.exiv2.org/projects/exiv2/repository/revisions/3048/entry/trunk/src/olympusmn.cpp#L1293
- so, basically, the "fix" is for exiv2 to return the normalized names for the Sony version of those lenses, so that they can be matched against the names on the lensfun list here:
- http://lensfun.sourceforge.net/lenslist/
(themselves based on the normalized exiv2 names, see olympusmn.cpp).
- http://lensfun.sourceforge.net/lenslist/
Hope that was clear enough.
Anything else, just let me know.
Cheers.
Updated by Rodrigo De Leon almost 6 years ago
Just for for completeness, here are the expected normalized names (complete lens list, here -> http://lensfun.sourceforge.net/lenslist/):
19mm:- Sigma 19mm f/2.8 EX DN
- Sigma 30mm f/2.8 EX DN
- Sigma 60mm f/2.8 DN
Cheers.
Updated by Robin Mills almost 6 years ago
Thanks, Rodrigo
Niels has been dealing with makernote/lens matters and has started a college course in addition to his very demanding job. So, I agreed last September to take over this stuff. It's been quite a learning curve to understand how this works and fix a few lens. I've reached a conclusion: http://dev.exiv2.org/issues/1160#note-1
As I've started to work on your stuff, I'll have a look at this at the weekend. If you have any thoughts about my plan to use exiftool over a web service, I'm interested to hear your comments.
Robin
Updated by Rodrigo De Leon almost 6 years ago
Interesting.
If I understand correctly:
- that method would be for initial identification (exiv2 does not initially know of the lens)?
- it's required to be online.
- afterwards, the info is incorporated to exiv2 for the next release?
Or am I understanding wrong?
Cheers.
Updated by Robin Mills almost 6 years ago
I think your summary is about correct. I'm saying that when the LensID is unique, we will identify it. When we don't know what it is, we'll use a webservice to ask exiftool. I will maintain and add new lens which have a unique LensID.
I don't know how lens recognition arrived in the code base. We read/write metadata. The meaning of that metadata isn't our concern. Guessing the lens from the focal length/aperture doesn't feel good. And it's consuming a lot of my time. At the moment, I'm performing about 70% of the work on Exiv2 and it adds up to an unpaid full-time job. I'd like to reduce the burden that I am carrying.
Updated by Robin Mills over 5 years ago
- Status changed from Assigned to Resolved
- % Done changed from 70 to 100
I've added a new feature with #1034 to read from the file ~/.exiv2. For example:
[sony]
255=My wiz-bang new lens
I intend to add the web-service feature, however this ~/.exiv2 will be faster and can be customized/localized as you think best.
#1035 test files