Bug #1143

Unable to extract embedded preview from jpg for Sony a77

Added by Michael Waldor over 1 year ago. Updated about 1 year ago.

Status:ClosedStart date:24 Dec 2015
Priority:NormalDue date:
Assignee:Robin Mills% Done:

100%

Category:image formatEstimated time:15.00 hours
Target version:0.26

Description

By accident I have learned that my Sony A77 DSLR stores a preview image within the jpg output. That preview picture can be utilized by DigiKam as a fast preview - and that works fine. But if I try to extract that preview with exiv2 I encounter the error message

exiv2 -p p screen.jpg
Error: Offset of directory Sony1, entry 0x2001 is out of bounds: Offset = 0x00784876; truncating the entry
Preview 1: image/jpeg, 160x120 pixels, 6530 bytes

exiftool -previewimage -b screen.jpg > preview.jpg
sees this picture and can extract it.

But now the ugly thing: During loading of all my figures from the last 4 years I asked digikam to autorotate portrait photos and fix the exifs accordingly. From all these photos even exiftool is unable to extract the preview image. But digikam still can display these previews not accessible to exiv2 or exiftool?!?!

These are the versions used (as being included within Opensuse distribution):
exiftool 10.00
exiv2 0.25 001900
Suse leap 42.1
digikam 4.23

screen.jpg - jpg directly from camera without digicam processing (8.09 MB) Michael Waldor, 24 Dec 2015 12:40

2015-12-06_005.jpg - preprocessed jpeg from digikam - no preview seen by exiv2 or exiftool, but by digikam (6.15 MB) Michael Waldor, 24 Dec 2015 15:04

preview.jpg (570 KB) Robin Mills, 24 Dec 2015 16:24

tamron.tgz - tar ball with jpegs from different tamron lenses (11.2 MB) Michael Waldor, 27 Dec 2015 14:46


Related issues

Related to Exiv2 - Bug #1144: Sigma 10-20mm f/4-5.6 EX DC is detected as Tamaron Closed 26 Dec 2015
Related to Exiv2 - Feature #816: Enable detection of Sigma 55-200mm lens Closed 18 Mar 2012
Related to Exiv2 - Feature #1145: Respect Sony/Minolta lenses with shared LensID such as Ta... Closed 27 Dec 2015
Related to Exiv2 - Feature #1108: Recursively dump sub-files of an image Closed 21 Aug 2015
Related to Exiv2 - Feature #1153: Sony ILCE-6000 + Sony E 50mm F1.8 OSS .JPG files without ... Closed 11 Jan 2016
Related to Exiv2 - Bug #1035: Lens model not detected ( exiv2 -> LensFun -> darktable ) Closed 19 Feb 2015

Associated revisions

Revision 4148
Added by Robin Mills over 1 year ago

#1145. Fix submitted.

I haven't submitted a change to the test suite at this time, however I'll do that when I resolve #1143.

Revision 4155
Added by Robin Mills over 1 year ago

#1143. Partial solution. I have suppressed the warning and allocated memory for Exif.Sony1.Preview. So now the size is revealed to the user. However: for reasons documented in the code, I am unable to locate the image in the source file and copy it into the metadata.

History

#1 Updated by Robin Mills over 1 year ago

  • Category set to not-a-bug
  • Status changed from New to Resolved
  • Assignee set to Robin Mills
  • Target version set to 0.26
  • % Done changed from 0 to 100
  • Estimated time set to 1.00

Ah, you're being gentle on me and giving me an easy puzzle for Christmas. The option -pp lists previews (as in print preview). The option -ep extracts previews. So:

585 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 --verbose --force -ep http://dev.exiv2.org/attachments/download/869/screen.jpg
File 1/1: http://dev.exiv2.org/attachments/download/869/screen.jpg
Error: Offset of directory Sony1, entry 0x2001 is out of bounds: Offset = 0x00784876; truncating the entry
Writing preview 1 (image/jpeg, 160x120 pixels, 6530 bytes) to file screen-preview1.jpg
586 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
The option --verbose prints "Writing preview 1...." and --force means "overwrite files without asking permission". The warning:
Error: Offset of directory Sony1, entry 0x2001 is out of bounds: Offset = 0x00784876; truncating the entry
is a little confusing. It's saying "The Sony makernote is referencing part of your file that is out of the bounds of the makernote.". This is a violation of the Exif specification. This is a common warning from Sony files and discussed here. http://dev.exiv2.org/issues/1108#note-1

The warning is written to errout and can be piped into oblivion.

586 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 --verbose --force -ep http://dev.exiv2.org/attachments/download/869/screen.jpg 2>/dev/null
File 1/1: http://dev.exiv2.org/attachments/download/869/screen.jpg
Writing preview 1 (image/jpeg, 160x120 pixels, 6530 bytes) to file screen-preview1.jpg
587 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 

I'm going to mark this 100% Resolved not-a-bug. If you wish to discuss this further you are welcome to update this issue report. If nothing else arises, I'll close this issue at the end of January.

Happy Holidays

#2 Updated by Robin Mills over 1 year ago

I've had another read at your issue report and more carefully read: But now the ugly thing

You'll have to send me an image with which to investigate this. The preview may been removed, or damaged beyond repair, however DigiKam may have retained a preview in the database.

#3 Updated by Michael Waldor over 1 year ago

Thanks for your fast reply. I come back to my first message ...

1. Of course I could extract the thumbnail image. BUT: Using exiftool I could as well extract a preview figure packed within my screen.jpg with size 1616 x 1080. I assume that this preview image is somehow hidden behind Sony specific manufacturer tags.

... and to your second reply
2. After processing such a picture by digikam (doing an autorotate just to help some jpeg viewers) the main jpeg is rotated and its orientation flag is fixed. But the thumbnail is NOT rotated, and its orientation is not changed. It's not nice, but consistent. BUT: As a sideeffect the preview image seems to be completely obscured and is neither accessible by exiv2 nor by exiftool. exiftool stills knows about the preview's size. But digikam sees it somehow completely (and can even display it if I select the preview display option). I have added such an jpeg...

Thus I still feel that something is going wrong.

Hope you will enjoy Christmas nevertheless:-)

Michael

#4 Updated by Robin Mills over 1 year ago

  • File preview.jpg added
  • Category changed from not-a-bug to image format
  • % Done changed from 100 to 20
  • Estimated time changed from 1.00 to 10.00

Ah, there isn't a Santa Claus after all. You've dumped a couple of corkers on me and I don't mean bottles of Champagne!

1 There's a mysterious hidden 1616x1080 preview image.
2 The thumbnail and image orientation can be in some kind of conflict

1 Mysterious hidden 1616x1080 preview image

I've extracted preview with exiftool and attached him to this issue:

607 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiftool -previewimage -b screen.jpg > preview.jpg
608 rmills@rmillsmbp:~/gnu/exiv2/trunk $ ls -alt preview.jpg
-rw-r--r--+ 1 rmills  staff  583985 24 Dec 15:20 preview.jpg
609 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
He's quite large. My instant guess is "That's what's in the illegal makernote reference that we have (inconveniently) ignored.". 583985 bytes can't be in the makernote which is 38792 bytes.
610 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep makernote/i http://dev.exiv2.org/attachments/download/869/screen.jpg 2>/dev/null
Exif.Photo.MakerNote                         Undefined 38972  (Binary value suppressed)
Exif.MakerNote.Offset                        Long        1  922
Exif.MakerNote.ByteOrder                     Ascii       3  II
I've tried extracting him:
 $ dd bs=1 skip=$((0x00784876)) count=583985 if=screen.jpg of=incredible.jpg 
It's not a JPG. I've sniffed around unsuccessfully with a hex editor in screen.jpg without spotting preview.jpg sitting waiting for me to put some handcuffs on him. I'll look again another day.

2 The thumbnail and image orientation

preview.jpg has metadata:

611 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa preview.jpg 
Exif.Image.XResolution                       Rational    1  72
Exif.Image.YResolution                       Rational    1  72
Exif.Image.ResolutionUnit                    Short       1  inch
Exif.Image.ExifTag                           Long        1  78
Exif.Photo.PixelXDimension                   Long        1  1616
Exif.Photo.PixelYDimension                   Long        1  1080
612 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 

I believe that previews normally do not have metadata. However this preview.jpg has metadata. I believe it's possible to rotate an image by modifying the Orientation metadata. This is almost instant and lossless as it's not necessary to think about the pixels.

Curiously, screen.jpg has distinct keys for Thumbnail and Image Orientation:

615 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep orientation/i http://dev.exiv2.org/attachments/download/869/screen.jpg 2>/dev/null
Exif.Image.Orientation                       Short       1  top, left
Exif.Thumbnail.Orientation                   Short       1  top, left
616 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
And these have been edited independently by DigiKam in 2015-12-06_005.jpg
617 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep orientation/i http://dev.exiv2.org/attachments/download/870/2015-12-06_005.jpg
Exif.Image.Orientation                       Short       1  top, left
Exif.Thumbnail.Orientation                   Short       1  left, bottom
Xmp.tiff.Orientation                         XmpText     1  top, left
618 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
Maybe this is causing an issue. What do you think? Can I ask you to modify Exif.Thumbnail.Orientation (and Exif.Image.Orientation) and examine the consequences in DigiKam.

I've reset this to 20% solved and I'm guessing 10 hours work to nail these puzzles: a further 4 hours on each of items 1 and 2. May I ask you to:

1) Sniff around in screen.jpg to see if you can find preview.jpg - he must be in there somewhere.
2) Investigate the dance between DigiKam and Exif.Thumbnail.Orientation and Exif.Image.Orientation.

#5 Updated by Robin Mills over 1 year ago

  • % Done changed from 20 to 40

OK, I've mostly solved puzzle 1: Mysterious hidden 1616x1080 preview image

It's in screen.jpg starting at byte 0x784800 which is a few bytes before the warning.

661 rmills@rmillsmbp:~/temp $ exiv2 -pa screen.jpg >/dev/null
Error: Offset of directory Sony1, entry 0x2001 is out of bounds: Offset = 0x00784876; truncating the entry
662 rmills@rmillsmbp:~/temp $ ls -alt screen.jpg
-rw-r--r--+ 1 rmills  staff  8486912 24 Dec 13:46 screen.jpg
663 rmills@rmillsmbp:~/temp $ dd bs=1 skip=$((0x00784800)) count=$((8486912-0x00784800)) if=screen.jpg of=mystery.jpg
604160+0 records in
604160+0 records out
604160 bytes transferred in 1.697599 secs (355891 bytes/sec)
664 rmills@rmillsmbp:~/temp $ exiv2 -pS mystery.jpg 
STRUCTURE OF JPEG FILE: mystery.jpg
 address | marker     | length  | data
       2 | 0xd8 SOI   |       0 
       4 | 0xe1 APP1  |     160 | Exif..II*...............>......
     166 | 0xdb DQT   |     132 
     300 | 0xc4 DHT   |     418 
     720 | 0xc0 SOF0  |      17 
     739 | 0xda SOS   |      12 
665 rmills@rmillsmbp:~/temp $ exiv2 -pa mystery.jpg 
Exif.Image.XResolution                       Rational    1  72
Exif.Image.YResolution                       Rational    1  72
Exif.Image.ResolutionUnit                    Short       1  inch
Exif.Image.ExifTag                           Long        1  78
Exif.Photo.PixelXDimension                   Long        1  1616
Exif.Photo.PixelYDimension                   Long        1  1080
666 rmills@rmillsmbp:~/temp $ 
And there's metadata Exiv2.Sony1.PreviewImage that indicates the presence of this additional preview.
680 rmills@rmillsmbp:~/temp $ exiv2 -pa --grep Preview screen.jpg   2>/dev/null
Exif.Sony1.PreviewImage                      Undefined   0  
Exif.Sony1.PreviewImageSize                  Long        2  1080 x 1616
681 rmills@rmillsmbp:~/temp $ 
I have to solve the puzzle of how to know the location is 0x784800.
681 rmills@rmillsmbp:~/temp $ echo $((0x784800))
7882752
682 rmills@rmillsmbp:~/temp $ 
I'll figure that in the next day or two and then consider updating our preview code to respect this guy. There is an APP2 segment which contains a little tiff object.
686 rmills@rmillsmbp:~/temp $ exiv2 -pS screen.jpg 
STRUCTURE OF JPEG FILE: screen.jpg
 address | marker     | length  | data
       2 | 0xd8 SOI   |       0 
       4 | 0xe1 APP1  |   46888 | Exif..II*........... ..........
   46894 | 0xe2 APP2  |     210 | MPF.II*...............0100.....
   47106 | 0xdb DQT   |     132 
   47240 | 0xc4 DHT   |     418 
   47660 | 0xc0 SOF0  |      17 
   47679 | 0xda SOS   |      12 
687 rmills@rmillsmbp:~/temp $ dd bs=1 skip=$((46894+6)) count=$((210-6)) if=screen.jpg of=screen.tif
204+0 records in
204+0 records out
204 bytes transferred in 0.000751 secs (271631 bytes/sec)
688 rmills@rmillsmbp:~/temp $ exiv2 -pS screen.tif 
STRUCTURE OF TIFF FILE (II): screen.tif
 address |    tag                           |      type |    count |   offset | value
      10 | 0xb000                           | UNDEFINED |        4 |808464688 | 
      22 | 0xb001                           |      LONG |        1 |        2 | 2
      34 | 0xb002                           | UNDEFINED |       32 |       50 |  ...
689 rmills@rmillsmbp:~/temp $ 

About puzzle 2: The thumbnail and image orientation

I'm not going to do any work on this matter. I don't feel this is an exiv2 puzzle. Have you discussed this with my friend Gilles at DigiKam?

OK, Turkey Time. I'm off for Christmas lunch with my family.

#6 Updated by Michael Waldor over 1 year ago

Hope you don't spend too much time fiddling with software issues instead of enjoying Christmas season...

I have investigated my exif problem from digikam's perspective. Now I have discovered the reason when pictures got different orientations. Only for those figures I have applied GPS coordinates. And if I add GPS data, the file must be written by digikam. And portrait photos are then rotated automatically (because I have configured that). New (minor) mystery: All pictures get GPS data and are written, but only the portrait ones get additional exif informations (image width/height, processing software) not added for landscape ones. Maybe I'll discuss this within digikam forum... I think that's a feature and not a bug.

BTW I look into orientation problems due to another device: I'm using a linux based digital satellite receiver with openatv. It can also display jpegs, but sometimes portrait pictures are displayed in a wrong orientation. Till yesterday I had believed that all pictures will be rotated during download by digikam. Now I've learned that the rotation is only applied if saving GPS coordinates explaining the "random" behaviour.
Sadly openatv doesn't use your exiv2 library but their own implementation
(https://github.com/openatv/enigma2/blob/master/lib/gdi/picexif.cpp). The orientation extracted by that routine by error reports the orientation of the embedded preview.

Conclusion: The simplest solution to my problem is to rotate all portrait figures with a small shell script/digikam batch if they are not already rotated. And for new downloads I'll have a look onto the options for downloading within digikam (there IS one to enforce auto rotation, I'm quite sure).

#7 Updated by Robin Mills over 1 year ago

Michael: Thanks for looking into all of this while Alison and I have been enjoying a wonderful Christmas Afternoon and Dinner with my son and his family.

I'm confident about solving Puzzle 1 and I'll take care of it in the next few days.

Puzzle 2 is getting more complex and adding this portrait/GPS complication. I was under the impression that DigiKam performs all metadata processing using libexiv2 exclusively. However I also believe that our preview code does not permit metadata in preview images. So I think something outside of libexiv2 is being done by DigiKam to handle Exif.Sony1.PreviewImage.

I didn't write Exiv2 - that honour goes to Andreas with great contributions from Brad, Gilles, Volker and others. However it's my agenda to strongly discourage applications from using home-made metadata code. Why roll-your-own, when you can get a mature, well-written, tested, documented, supported, cross-platform open-source library? I'd like to see Apple, Adobe, Microsoft and Google put their talents and efforts into Exiv2. Everybody would benefit. Spread the word!

For sure, Puzzle 2++ is now so multi-faceted that I can only agree with your conclusion. Scripting to fix your library of images sounds like a good plan. If I can help, please let me know.

#8 Updated by Robin Mills over 1 year ago

Michael: I've just notice that Exiv2 has not detected the lens you are using. Here's what I see:

551 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa --grep Lens ~/temp/screen.jpg 2>/dev/null
Exif.Sony1.LensID                            Long        1  Tamron SP AF 17-50mm F2.8 XR Di II LD Aspherical 
    | Tamron AF 18-250mm F3.5-6.3 XR Di II LD 
    | Tamron AF 55-200mm F4-5.6 Di II LD Macro 
    | Tamron AF 70-300mm F4-5.6 Di LD Macro 1:2 
    | Tamron SP AF 200-500mm F5.0-6.3 Di LD IF 
    | Tamron SP AF 10-24mm F3.5-4.5 Di II LD Aspherical IF 
    | Tamron SP AF 70-200mm F2.8 Di LD IF Macro 
    | Tamron SP AF 28-75mm F2.8 XR Di LD Aspherical IF 
    | Tamron AF 90-300mm F4.5-5.6 Telemacro
Exif.Photo.LensSpecification                 Rational    4  0/10 0/10 0/10 0/10
Exif.Photo.LensModel                         Ascii       5  ----
552 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
Which lens are you using?

#9 Updated by Michael Waldor over 1 year ago

The first one - a Tamron SP AF 17-50mm F2.8 XR Di II LD Aspherical

That would be great if the Tamron lens could be identified correctly. I thought that all Tamron lenses would use the same lens ID.

If it might help I can send to you pictures from several other Tamron lenses, too (90mm, 70-300mm, 18-200mm, 11-17mm) - I exclusively use Tamron lenses. But I should extract the exif information only to reduce the data amount transported. Can I do that with exiv2?

#10 Updated by Robin Mills over 1 year ago

  • Status changed from Resolved to Assigned
  • % Done changed from 40 to 50

Thanks for getting back to me. After I asked you about the Lens, I thought "Ah I'll use exiftool" and of course he knew.

$ exiftool ~/temp/screen.jpg 2>/dev/null | grep -i lens
Lens Type                       : Tamron Lens (255)
Lens Spec                       : Unknown (00 0 0 0 0 00)
Lens Mount                      : A-mount
Lens Format                     : Full-frame
Lens Spec Features              : 
Lens Info                       : 0mm f/0
Lens Model                      : ----
Lens ID                         : Tamron SP AF 17-50mm F2.8 XR Di II LD Aspherical
762 rmills@rmillsmbp:~/clanmills $ 
Your LensId is 0xff:
$ exiv2 -pv --grep lens/i ~/temp/screen.jpg 2>/dev/null 
0xb027 Sony1        LensID                      Long        1  255
0xa432 Photo        LensSpecification           Rational    4  0/10 0/10 0/10 0/10
0xa434 Photo        LensModel                   Ascii       5  ----
$
And we know about 9 Tamron Lenses that share that ID.
$ exiv2 -pa --grep lens/i ~/temp/screen.jpg 2>/dev/null | sed -e 's/|/  \'$'\n''|/g' 
Tamron SP AF 17-50mm F2.8 XR Di II LD Aspherical   
| Tamron AF 18-250mm F3.5-6.3 XR Di II LD   
| Tamron AF 55-200mm F4-5.6 Di II LD Macro   
| Tamron AF 70-300mm F4-5.6 Di LD Macro 1:2   
| Tamron SP AF 200-500mm F5.0-6.3 Di LD IF   
| Tamron SP AF 10-24mm F3.5-4.5 Di II LD Aspherical IF   
| Tamron SP AF 70-200mm F2.8 Di LD IF Macro   
| Tamron SP AF 28-75mm F2.8 XR Di LD Aspherical IF   
| Tamron AF 90-300mm F4.5-5.6 Telemacro
Exif.Photo.LensSpecification                 Rational    4  0/10 0/10 0/10 0/10
Exif.Photo.LensModel                         Ascii       5  ----
$ 
We don't have any magic in our code for Minolta/Sony lenses. Send me sample images using different lens/camera combinations and I'll see what I can do.

Phil (Harvey of exiftool) has already done this and pointed me to his beautiful perl code that contains his magic. It's beautiful and ingenious perl. Can I can transform it into C++? Not easy as it uses perl hashes/arrays/references that can't easily be represented in C++.

I'm thinking I might call exiftool when I encounter a shared LensID. Cheating? Maybe. Effective? We'll see. If I can get this to work, I'll ask Phil to bless this. Exiv2 has a good friendly relationship with Phil.

So, yes please attach images of different camera/lens combinations and I'll see that I can do.

#11 Updated by Robin Mills over 1 year ago

For completeness, here is Phil's code. http://sourceforge.net/p/exiftool/code/ci/master/tree/lib/Image/ExifTool/Exif.pm#l3580

Lines: 3576-3730:

#------------------------------------------------------------------------------
# Attempt to identify the specific lens if multiple lenses have the same LensType
# Inputs: 0) ExifTool object ref, 1) LensType print value, 2) PrintConv hash ref,
#         3) LensSpec print value, 4) LensType numerical value, 5) FocalLength,
#         6) MaxAperture, 7) MaxApertureValue, 8) MinFocalLength, 9) MaxFocalLength,
#         10) LensModel, 11) LensFocalRange, 12) LensSpec
sub PrintLensID($$@)
{
    my ($et, $lensTypePrt, $printConv, $lensSpecPrt, $lensType, $focalLength,
        $maxAperture, $maxApertureValue, $shortFocal, $longFocal, $lensModel,
        $lensFocalRange, $lensSpec) = @_;
    # this logic relies on the LensType lookup:
    return undef unless defined $lensType;
    # get print conversion hash if necessary
    $printConv or $printConv = $$et{TAG_INFO}{LensType}{PrintConv};
    # just copy LensType PrintConv value if it was a lens name
    # (Olympus or Panasonic -- just exclude things like Nikon and Leaf LensType)
    unless (ref $printConv eq 'HASH') {
        if (ref $printConv eq 'ARRAY' and ref $$printConv[0] eq 'HASH') {
            $printConv = $$printConv[0];
            $lensTypePrt =~ s/;.*//;
            $lensType =~ s/ .*//;
        } else {
            return $lensTypePrt if $lensTypePrt =~ /mm/;
            return $lensTypePrt if $lensTypePrt =~ s/(\d)\/F/$1mm F/;
            return undef;
        }
    }
    # get LensSpec information if available (Sony)
    my ($sf0, $lf0, $sa0, $la0);
    if ($lensSpecPrt) {
        ($sf0, $lf0, $sa0, $la0) = GetLensInfo($lensSpecPrt);
        undef $sf0 unless $sa0; # (make sure aperture isn't zero)
    }
    # use MaxApertureValue if MaxAperture is not available
    $maxAperture = $maxApertureValue unless $maxAperture;
    if ($lensFocalRange and $lensFocalRange =~ /^(\d+)(?: (?:to )?(\d+))?$/) {
        ($shortFocal, $longFocal) = ($1, $2 || $1);
    }
    if ($$et{Make} eq 'SONY') {
        # patch for Metabones Canon adapters on Sony cameras (ref Jos Roost)
        # (the Metabones adapters add 0xef00 or 0x7700 to the high byte
        # for 2-byte LensType values, so we need to adjust for these)
        if ($lensType != 0xffff) {
            require Image::ExifTool::Minolta;
            if ($Image::ExifTool::Minolta::metabonesID{$lensType & 0xff00}) {
                $lensType -= ($lensType >= 0xef00 ? 0xef00 : $lensType >= 0xbc00 ? 0xbc00 : 0x7700);
                require Image::ExifTool::Canon;
                $printConv = \%Image::ExifTool::Canon::canonLensTypes;
                $lensTypePrt = $$printConv{$lensType} if $$printConv{$lensType};
            }
        }
    } elsif ($shortFocal and $longFocal) {
        # Canon (and some other makes) include makernote information
        # which allows better lens identification
        require Image::ExifTool::Canon;
        return Image::ExifTool::Canon::PrintLensID($printConv, $lensType,
                    $shortFocal, $longFocal, $maxAperture, $lensModel);
    }
    my $lens = $$printConv{$lensType};
    return ($lensModel || $lensTypePrt) unless $lens;
    return $lens unless $$printConv{"$lensType.1"};
    $lens =~ s/ or .*//s;    # remove everything after "or" 
    # make list of all possible matching lenses
    my @lenses = ( $lens );
    my $i;
    for ($i=1; $$printConv{"$lensType.$i"}; ++$i) {
        push @lenses, $$printConv{"$lensType.$i"};
    }
    # attempt to determine actual lens
    my (@matches, @best, @user, $diff);
    foreach $lens (@lenses) {
        push @user, $lens if $Image::ExifTool::userLens{$lens};
        # sf = short focal
        # lf = long focal
        # sa = max aperture at short focal
        # la = max aperture at long focal
        my ($sf, $lf, $sa, $la) = GetLensInfo($lens);
        next unless $sf;
        # check against LensSpec parameters if available
        if ($sf0) {
            next if abs($sf - $sf0) > 0.5 or abs($sa - $sa0) > 0.15 or
                    abs($lf - $lf0) > 0.5 or abs($la - $la0) > 0.15;
            # the basic parameters match, but also check against additional lens features:
            # for Sony A and E lenses, the full LensSpec string should match with end of LensType,
            # excluding any part between () at the end, and preceded by a space (the space
            # ensures that e.g. Zeiss Loxia 21mm having LensSpec "E 21mm F2.8" will not be
            # identified as "Sony FE 21mm F2.8 (SEL28F20 + SEL075UWC)")
            $lensSpecPrt and $lens =~ / \Q$lensSpecPrt\E( \(|$)/ and @best = ( $lens ), last;
            # exactly-matching Sony lens should have been found above, so only add non-Sony lenses
            push @best, $lens unless $lens =~ /^Sony /;
            next;
        }
        # adjust focal length and aperture if teleconverter is attached (Minolta)
        if ($lens =~ / \+ .*? (\d+(\.\d+)?)x( |$)/) {
            $sf *= $1;  $lf *= $1;
            $sa *= $1;  $la *= $1;
        }
        # see if we can rule out this lens using FocalLength and MaxAperture
        if ($focalLength) {
            next if $focalLength < $sf - 0.5;
            next if $focalLength > $lf + 0.5;
        }
        if ($maxAperture) {
            # it seems that most manufacturers set MaxAperture and MaxApertureValue
            # to the maximum aperture (smallest F number) for the current focal length
            # of the lens, so assume that MaxAperture varies with focal length and find
            # the closest match (this is somewhat contrary to the EXIF specification which
            # states "The smallest F number of the lens", without mention of focal length)
            next if $maxAperture < $sa - 0.15;  # (0.15 is arbitrary)
            next if $maxAperture > $la + 0.15;
            # now determine the best match for this aperture
            my $aa; # approximate maximum aperture at this focal length
            if ($sf == $lf or $sa == $la or $focalLength <= $sf) {
                # either 1) prime lens, 2) fixed-aperture zoom, or 3) zoom at min focal
                $aa = $sa;
            } elsif ($focalLength >= $lf) {
                $aa = $la;
            } else {
                # assume a log-log variation of max aperture with focal length
                # (see http://regex.info/blog/2006-10-05/263)
                $aa = exp(log($sa) + (log($la)-log($sa)) / (log($lf)-log($sf)) *
                                     (log($focalLength)-log($sf)));
                # a linear relationship between 1/FocalLength and 1/MaxAperture fits Sony better (ref 27)
                #$aa = 1 / (1/$sa + (1/$focalLength - 1/$sf) * (1/$la - 1/$sa) / (1/$lf - 1/$sf));
            }
            my $d = abs($maxAperture - $aa);
            if (defined $diff) {
                $d > $diff + 0.15 and next;     # (0.15 is arbitrary)
                $d < $diff - 0.15 and undef @best;
            }
            $diff = $d;
            push @best, $lens;
        }
        push @matches, $lens;
    }
    # return the user-defined lens if it exists
    if (@user) {
        # choose the best match if we have more than one
        if (@user > 1) {
            my ($try, @good);
            foreach $try (\@best, \@matches) {
                $Image::ExifTool::userLens{$_} and push @good, $_ foreach @$try;
                return join(' or ', @good) if @good;
            }
        }
        return join(' or ', @user);
    }
    # return the best match(es) from the possible lenses
    return join(' or ', @best) if @best;
    return join(' or ', @matches) if @matches;
    $lens = $$printConv{$lensType};
    return $lensModel if $lensModel and $lens =~ / or /; # (eg. Sony NEX-5N)
    return $lens;
}

#12 Updated by Michael Waldor over 1 year ago

I have taken some sample shots with all my tamron lenses (appended as tar ball). I have chosen the lowest resolution available, but the jpegs still occupy at least 2MB:-(

The lenses are
Tamron SP AF 11-18mm F4.5-5.6 Di II LD
Tamron SP AF 17-50mm F2.8 XR Di II LD Aspherical
Tamron AF 18-200mm F3.5-6.3 XR Di II LD Macro
Tamron SP AF 70-300mm F4-5.6 Di USD
Tamron SP AF 90mm F2.8 Di Macro 1:1
and the jpegs have the focal length in their names.

Thanks for considering Tamron lenses to be recognized by exiv2,
Michael

#13 Updated by Robin Mills over 1 year ago

  • % Done changed from 50 to 60
  • Estimated time changed from 10.00 to 20.00

Thanks. I opened that as a separate issue for working on the Tamron Lens issues. I will attach your bundle to issue: #1145.

I'm in trouble with Puzzle 1: Mysterious hidden 1616x1080 preview image There are 3 serious difficulties with implementing this:

  1. I have implemented a solution which caused a change in the API. That change ripples across 18 files.
  2. It requires a static in TiffWorker (because the TiffWorker::Decoder is static). The decoder is thread-safe, however a static IO_ reference is not. This will break our thread safety model.
  3. Although I can obtain the size of the preview at 583985 bytes, I don't know exactly where it is located (the warning is near, but not at the location of the hidden preview).

I have to take a break from this for couple of days. I cannot submit this "fix".

The difficulty is being caused by TiffVisitor which is event driven code for parsing Tiff Dictionaries. TiffVisitor does not know the source of the bytes he is fed. To reference the IO object in TiffVisitor, I have to update every image handler in the code (jpg, tiff, cr2, png, tiff, etc). I have to update both the .cpp and .hpp files.

I'm quite sure there's another way to deal with this. Having approximately located this image, and knowing his size, I think I can provide a thread-safe function to extract him. Changing TiffVisitor (and all his clients) is not the correct solution. And once I have the image, I have to visit the preview code and get him to respect the new bandit.

As I said on the day before Christmas, you've dropped a corker on me. I didn't write Exiv2 - so there's a lot of thinking and learning to get this fixed. I've pushed the solution estimate to 20 hours.

#14 Updated by Robin Mills over 1 year ago

  • % Done changed from 60 to 50

#15 Updated by Robin Mills over 1 year ago

  • % Done changed from 50 to 70

Partial fix submitted: r4155. This fix suppresses the warning (only for Exif.Sony1.PreviewImage) and allocates a 'hollow' block of memory.

$ exiv2 -pa --grep Preview http://dev.exiv2.org/attachments/download/869/screen.jpg
Exif.Sony1.PreviewImage                      Undefined 583855  (Binary value suppressed)
Exif.Sony1.PreviewImageSize                  Long        2  1080 x 1616
$ 
This is a difficult issue for reasons explained in r4155. It might be possible to populate the Exif.Sony1.PreviewImage image in a post process when we recover Exif.Sony1.LensID. Andreas might be willing to look at this in 2016.

#16 Updated by Robin Mills about 1 year ago

  • Status changed from Assigned to Closed
  • % Done changed from 70 to 100
  • Estimated time changed from 20.00 to 15.00

This is a very difficult issue for reasons explained in r4155. Sony have violated the Exif Specification and this breaks the Exiv2 architecture. I'm going to reduce the Estimated time to 15 hours and close this matter. I don't believe there is solution for this issue.

Also available in: Atom PDF

Redmine Appliance - Powered by TurnKey Linux