Project

General

Profile

Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body

Added by marcus san about 4 years ago

Hi,

New here so not sure how to handle this. I found my Sigma 35mm F1.4 DG HSM ART lens is not recognized by exiv2. In exiv2 and thus Darktable it is recognized as > Canon EF 17-35mm f/2.8L
Any suggestion on how to sort this out ? Running exiv2 version 0.25 on Linux.

Thanks in advance.


Replies (41)

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Alan Pater about 4 years ago

Robin Mills wrote:

I recently had a conversation with somebody on this Forum about their Sony Camera and Lenses. A firmware update fixed the Sony business.

I'm going to try using a web service in v0.26 to take advantage of Phil's Lens Recognition in exiftool. We'll see how that goes. However, I honestly believe that lens recognition is not the business of Exiv2.

Would teaming up with the Lensfun project help? http://lensfun.sourceforge.net/

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Robin Mills about 4 years ago

Alan

I think this is something to be investigated. I've heard lensfun mentioned without knowing what it was. I think Niels might contribute to that. If he does, it begs the question "Why don't we use lensfun and get out of the lens recognition business?".

We're going to get out of the lens recognition business. We're in low-level, reliable, robust read/write/modify metadata. The use and meaning of the metadata is not our concern. The feature, $ exiv2 -pt image ... is well intended, however non-deterministic. We should only deal in solid/hard data. Regrettably, I believe we're stuck with this feature.

Lens recognition is big time-sink. I got up at 6am this morning and wanted to deal with issues concerning MSVC2015 (#1169 and #1171). It's 2:40pm and I still have to submit the fix for this damn lens. Project slips another day with nothing much to show for it.

I have agreed a course of action with Marcus and I'll get on with that now. Tomorrow I'll work on the MSVC2015 issues.

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by marcus san about 4 years ago

I wanna thank Robin for his time and effort so far regarding taking the fix for the Sigma lens in consideration. Let me know if I can assist any further.

Regards,
Marcus.

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Robin Mills about 4 years ago

No problem, Marcus. I opened a new issue: #1170. The fix was the simplest I've encountered in Lens Recognition. r4236

Thank You for providing the test data. I've extracted the metadata from your file into test/data/exiv2-bug1170.exv and updated the test suite appropriately.

You should be able to build from the trunk. I believe darktable uses /usr/lib/x86_64-linux-gnu/libexiv2.so.14.0.0 When you build and install exiv2, it will install into /usr/local/lib/libexiv2.so.14.0.0. You'll need to sudo copy the new library. I have inspected your file testfile.jpg with darktable and the fix is "sort of" working for me. I opened the file with darktable, before I updated the library. The lens type was listed as Canon. I stopped darktable, replaced the library and reopened darktable. Still showed Canon. I copied testfile.jpg testfile2.jpg and imported into darktable. Happiness, darktable showed Sigma. Conclusion: When darktable imports images, it either caches the metadata, or the image. I've never used darktable before.

If replacing the library doesn't seem to work, use the utility 'lsof' to discover which the location of libexiv2 in use and replace it.

$ lsof | grep darktable | grep libexiv2

You may prefer to download the daily build (tomorrow) from our buildserver: http://exiv2.dyndns.org:8080/userContent/builds/Categorized/Platform/linux/

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Robin Mills about 4 years ago

I'm going to add another feature to exiv2 for v0.26. It's something that has been discussed inconclusively before: Camera Overflow File.

The user can put the file ~/.exiv2 (in .ini format) in their home directory and define a new lens:

[canon]
169=Sigma 35mm F1.4 DG HSM ART

Exiv2 will consult that file and use definitions in preference to what's compiled into the library. Let's not discuss the performance implication. I'll find a way to make it very fast (don't read if not necessary, then read/cache).

I'm not promising when this will be done. It'll be in v0.26.

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by marcus san about 4 years ago

Hi Robin,

Just to let you know. I downloaded the daily build, copied the new library file and followed your steps. Now the correct lens is found. Thanks a lot for the support !

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Robin Mills about 4 years ago

Thanks for letting me know. I'll let you know when I add the ~/.exiv2 feature and you give that a shot in darkroom.

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by marcus san almost 4 years ago

Hi Robin,

Somehow it stopped working for me and the old canon lens info is used again, instead of the correct sigma lens info. I downloaded the latest nightly build, and copied the libexiv2.so.14.0.0 but no luck.
Can you see if the newest files still have that fix you applied ?

Thanks,
Marcus.

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Robin Mills almost 4 years ago

Marcus:

Before we dive into lots of work, can I ask you to double/triple check that you are calling the latest version of exiv2 (or any version from svn r4236 and later). Run $ exiv2 -vV --grep library --grep svn to determine which library (and revision) you are using. Be careful, the exiv2 command-line program might be using a different copy of the library than darktable. You can see the library used by darktable (while DT is running) with the command: $ lsof | grep exiv2

This lens recognition stuff isn't computer science, it's black magic. I can't think why the fix has broken ... but if it's broken, it's broken. For sure, it's passing the test suite:

764 rmills@rmillsmbp:~/Downloads $ exiv2 -pa --grep lens/i ~/gnu/exiv2/trunk/test/data/exiv2-bug1170.exv 
Exif.CanonCs.LensType                        Short       1  Sigma 35mm f/1.4 DG HSM
Exif.CanonCs.Lens                            Short       3  35.0 mm
Exif.Canon.LensModel                         Ascii      74  35mm
Exif.Photo.LensSpecification                 Rational    4  35/1 35/1 0/1 0/1
Exif.Photo.LensModel                         Ascii       5  35mm
Exif.Photo.LensSerialNumber                  Ascii      11  0000000000
765 rmills@rmillsmbp:~/Downloads $

I have added code to respect the ~/.exiv2 file. At the moment, it is only called by canonmn.cpp I still have to call it from other makernotes src/*mn.cpp).

Perhaps you could try this in ~/.exiv2

[canon]
169=Sigma 35mm F1.4 DG HSM ART changed by Marcus

To be sure that your lens is '169', run

$ exiv2 -pv --grep lenstype/i foo.jpg
The output should be something like:
765 rmills@rmillsmbp:~/Downloads $ exiv2 -pv --grep lenstype/i ~/gnu/exiv2/trunk/test/data/exiv2-bug1170.exv 
0x0016 CanonCs      LensType                    Short       1  169
766 rmills@rmillsmbp:~/Downloads $

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by marcus san almost 4 years ago

Hi Robin,

There was an issue with the file and the symlink. Somehow, when I did . $lsof | grep exiv2
I found that Darktable was using the _BKP file I created before copying. Not sure why, but thats the risk of these kind of fixes I assume. Probably caused by some update or a mistake I made. I fixed the symlink to the library and copied the newest libexiv2.so.14.0.0 again from the nightly build to /usr/lib64/libexiv2.so.14.0.0

Now the Sigma is identified correct again.

I will try to see if the ~/.exiv2 method works, as it seems better resistant against updates form Darktable. What version/ nightly build has the new code change that uses the ~/.exiv2 method ?

Regards,
Marcus

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Robin Mills almost 4 years ago

I'm not to sure what this symlink stuff is about, however I know you understand it!

The fix to recognise your lens was added in r4236 on 2016-03-22 and logged against #1170

The ~/.exiv2 feature was added in r4250 on 2016-03-25 and logged against #1034. It was subsequently corrected for MSVC builds in r4259 on 2016-03-28.

I decided on 2016-03-24 to wait before letting you know about ~/.exiv2 for a couple of reasons:

  1. I haven't investigated deploying this with other cameras.
  2. I haven't investigated testing this feature from our test suite.

Your report about the fix being 'disturbed' by upgrades is strong reinforcement of my design concept to support ~/.exiv2 and/or the web service to resolve ambiguous lens recognition. Once DT supports v0.26, you can update ~/.exiv2 and forget about it. A quiet life for both of us!

There is a way to immunise your system from DT (or system) updates. I think putting libexiv2.so.14.0.0 into ~/lib will work. You may have to update the LD_LIBRARY_PATH environment string in ~/.gnomerc to be certain it's in the Desktop Environment, and therefore inherited by DT when launched from the GUI. http://superuser.com/questions/19044/how-do-i-set-an-environmental-variable-for-a-gnome-session As always with Linux, your setup may be different. As you know, every aspect of Linux is "standard" and yet varies between distros!

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Robin Mills almost 4 years ago

I met Pascal de Bruijn from the Darktable project at the Libre Graphics Meeting in London on Saturday 2016-04-16. A very nice guy. I've had various emails exchanges with Pascal in the past and it was a pleasure to meet him in person.

I knew that my advice to you about LD_LIBRARY_PATH and ~/.gnomerc does not work. Pascal sent me the following email about that matter. Please proceed with caution as Pascal's fix is not tested and could seriously damage your Linux setup. He has advised me to make a snapshot as I run Linux in a VM on MacOS-X.

This is a relevant Debian regression / feature request which seems to apply to Ubuntu as well (being a direct derivative of Debian):
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=685706

So it's essentially a library search path ordering issue...

So renaming /etc/ld.so.conf.d/libc.conf should make /usr/local/lib available as a library override directory, however renaming such a file isn't a good idea, as a package update would put back to old file again.

The "proper" way to do things like this on Debian/Ubuntu is by using dpkg-divert, which allows you to mess with file, while still allows package updates through dpkg to keep these local changes in mind, so something along the line of the following should help:

$ sudo dpkg-divert --add --rename --divert /etc/ld.so.conf.d/libc.conf /etc/ld.so.conf.d/00-libc.conf
$ sudo ldconfig

Please do keep in mind I didn't test the above, so please do snapshot your VM, before trying the above :)

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by marcus san almost 4 years ago

Hi Robin,

I am reading this and must investigate more. I am running OpenSuse Leap 42.1 KDE as my daily production OS. So some difference here for a start.
I will look into re-linking the libraries, but mostly am hoping for a more stable/easy solution, like the ~/exiv2 folder setup you mentioned.

Thanks,
Marcus.

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by Robin Mills almost 4 years ago

Marcus

I was very surprised to learn from Pascal that he has been back-porting every change in Exiv2 concerning lens recognition into v0.23 AND sending it to the Linux distribution folks. In this way, DT keeps up-to-date with our lens code. Pascal's going to do this for v0.25. I'm not sure if he will pull the .ini code into his distributed v0.25 releases of exiv2. He was impressed by that feature, however we didn't discuss if he can back-port it to v0.23 and v0.25. He's a smart guy and will probably figure it out for himself. I will help if he asks.

I believe exiv2 v0.26 will be code complete in June 2016. I'm not involved in release/packaging, however I hope that will be finished by the end of July. So most Linux distributions will have v0.26 with the ~/.exiv2 feature by late 2016. Hopefully happiness will arrive in the world of lens recognition with v0.26.

RE: Sigma 35mm F1.4 DG HSM ART not recognized on Canon 6D body - Added by marcus san almost 4 years ago

Hi Robin,

I am really looking forward to v0.26. The new feature sounds very promising. Let me know if I can help out with testing stuff etc.

Best Regards.

(26-41/41)