Exiv2

Stephan

Thanks for bringing this to our attention. I've reproduced this with the 0.21.1 code-base. I ran your image through the debugger and it seems that an 18 byte block of PhotoShop data is causing the library to panic when updating the metadata. I think I have a code fix (but no work around) which I'd like to discuss off-line with Andreas.

Robin

Hello Robin,
the image is directly from the camera. I have hundreds of such images.
They have never seen Photoshop.
Would be great to get a code-fix.

Stefan

I'm willing to share my fix privately if you email me: robin@clanmills.com. However I would like Andreas to "bless" the fix before I cause unnecessary confusion.

Presumably this little PhotoShop block was inserted by the camera. My Canon camera inserts a little PhotoShop block (of 100 bytes) and I don't know what's in there. However I believe it's the length of your block (18 bytes) which is causing pain!

Robin

I've created bug #760 for this and checked in a fix.

The sample image contains an APP13 Photoshop PS3 segment with an empty payload. The exiv2 JPEG write function interpreted the empty data as invalid. With the fix, exiv2 accepts the empty block but removes it on write unless there is data to write to it.

Thanks Stefan and Robin for reporting and investigating this.

Andreas

Good to know we'll be getting fewer of those "does not contains" in 0.21 and future versions. Then again, the Adobe code-bit Andreas found in Stefan's example file could be the only one Exiv2 has been "choking on" since PS3 was new. Here's hoping it is, eh?

BZT

Steve Wright wrote:

Good to know we'll be getting fewer of those "does not contains" in 0.21 and future versions. Then again, the Adobe code-bit Andreas found in Stefan's example file could be the only one Exiv2 has been "choking on" since PS3 was new. Here's hoping it is, eh?

BZT

I'm not so sure. I'm now running 0.22 and I'm still getting these errors on a bunch of files. Did Robin's fix go in for more than one version?

BZT

My bad -- it's still 0.21.1 that I'm running in Cygwin [the 0.22 is on my Linux laptop; now the proud (?!) home of Ubuntu 11.0.4 Natty].
For my purposes in Cygwin, most of the time, 0.21.1 suffices. In this instance, with regard to this odd error, I'd much rather patch what I have installed with Robin's fix -- or one from Andreas -- than upgrade.

Of course, I could be persuaded to do otherwise... :)

SJW

Steve Wright wrote:

My bad -- it's still 0.21.1 that I'm running in Cygwin [the 0.22 is on my Linux laptop; now the proud (?!) home of Ubuntu 11.0.4 Natty].
For my purposes in Cygwin, most of the time, 0.21.1 suffices. In this instance, with regard to this odd error, I'd much rather patch what I have installed with Robin's fix -- or one from Andreas -- than upgrade.

Of course, I could be persuaded to do otherwise... :)

SJW

With the files I happen to be working with tonight, upgrading wouldn't make a whole lot of difference, as 0.22 is still giving the "does not contain a valid image" error on my Linux laptop like 0.21.1 did from Cygwin.

Which surprises me, as bug #760 is in the change log of 0.22, I presume: as being fixed. Maybe my files are suffering from another problem?

Robin -- how well does that patch/fix work? Looks like you might still need it every now and then. :-|

SJW

Steve

Andreas registered a bug for this #760 and checked in a fix SVN:2461. I think Andreas' fix was slightly different/simpler than my patch, however they amount to about the same.

Can you send me a file with the fault and I'll investigate. Which version(s) do you want me to investigate? cygwin/0.21.1 and/or linux/0.22 ?

Robin

Robin, Andreas:

I just downloaded the revised jpgimage.cpp from that revision. I guess the next thing to do, if I recall the other patches to things I've done, is re-compile Exiv2 on both machines, using this C++ file instead of the one that came with the source tarball (or in the case of 0.22 on my laptop, the Subversion snapshot).

I still have the page with the file in the Repository open in another tab of my browser as I write this. Looking at the date, I'm assuming this same substitute will compile for both 0.21.1 and 0.22. Am I correct in that assumption?

SJ

Steve

I can't follow what you're saying. I don't know to which patch files you are referring. Patch files are not part of the official release and we don't support them on an ongoing basis.

Thanks for the .jpg which you say exhibits the issue which I believe you are experiencing with 0.21.1 and 0.22 on Linux and Cygwin. Please give a command-line which to reproduce the issue using the image that you have sent and I will investigate.

Robin

Second part first:

On that JPEG file I tried to write an IPTC caption, same words, with the file in the same location from both machines running their respective versions of Exiv2. The same error came up in both instances.

Specifically, the command line was "exiv2 -kM"set Iptc.Application2.Caption This one seems tasteful" (followed by the name of the file).

Now for the first part.

Back when I was keen on getting a few iView Media Pro/MS Expression Media XMP tags into Exiv2, I seem to recall Andreas saying that if I edited one thing and 'tweaked' another, then recompiled the binary, it would recognize those tags without having to provide a custom namespace. Even if, strictly speaking, it wasn't one, I considered the editing and the steps involved to be a patch. That's most likely why the term came up from my end in this discussion. Sorry if it led to any confusion.

SJW

Thanks, Steve. I've reproduced this with exiv2 0.21.1 on Ubuntu 11.04. I attach the image you sent. I'll investigate over the weekend and get back to you.

2029 /home/rmills $ exiv2 -kM"set Iptc.Application2.Caption This one seems tasteful" ~/Desktop/1308.jpg
Exiv2 exception in modify action for file /home/rmills/Desktop/1308.jpg:
Input data does not contain a valid image
2030 /home/rmills $

Robin has emailed me and asked that I let everyone reading this topic thread know that two Adobe applications he uses (one Mac and one not) gave errors on the sample file I sent him. Modifying, and therefore re-writing, the IPTC Core data in another Adobe app (Bridge from CS5, and here I'm presuming it's the Windows build), and subsequently trying the same command that gave the error on my machines (see my last post on this thread), were both successful.

He and I both wonder if Adobe doesn't have a simple tool for correcting IPTC Core data in files, and if there isn't, it makes one wonder why there isn't, as it was Adobe software that produced [the cause for] this error to begin with.

So these "does not contain a valid image" errors that persist do not seem to be caused by empty-byte blocks of the kind Andreas explained to the OP, Stefan Brandi. There might be yet another cause, and the newer exiv2 versions "still can't deal" and so err out.

I'm sure when all the causes and effects are known about this, Andreas will be prompt with a solution.

SJ Wright

Guys

Steve's issue with the IPTC data is different from the original matter which was fixed in issue #760. In the original issue, an empty IPTC block was tripping an exception.

In Steve's image (written by PS CS1) the IPTC block is indeed wrong. PS Elements 6.0 can't even open the file! I modified the IPTC data in the image with Adobe Bridge CS5 and that repaired the IPTC datablock. After that, exiv2 was happy to modify the corrected file.

Other folks have been discussing this matter. For example:
http://photo.net/digital-darkroom-forum/00E1r4

I don't think this matter warrants further attention. exiv2 cannot reasonably be expected to handle corrupt files. PhotoShop CS is a rather elderly product (2003).

All the same, through fits and starts and trial and error, I've come up with an approach that seems to work w/re making these files "valid" and write-able with Exiv2.

My four-step process for "restoring" JPEG files that other apps appear to have put "beyond help." 

1. Extract all metadata to .exv files [exiv2 -ea $foo]
2. Use "jhead -purejpg" to strip them down to rudimentary JFIF data
3. Write the metadata back to the files from the .exv's [exiv2 -kia $foo]
4. Test arbitrary files by writing new metadata to them.

Seems to work so far.

Steve Wright