Exiv2

Obviously I didn't format the carets correctly, but the problem is reasonably apparent even with the lt gt stuff. Sorry about that.

I have a jpg file 2014-2.jpg (attached) where the XMP "Rights" value is:

<dc:rights>
<rdf:Alt>
  <rdf:li xml:lang="x-default">2014 John Scot Huggins, All Rights Reserved</rdf:li>
</rdf:Alt>
</dc:rights>

Xmp.dc.rights   LangAlt 1 lang="x-default" 2014 John Scot Huggins, All Rights Reserved

Great. If I run exiv2 -ex 2014-2.jpg I find this in the 2014-2.exv file...

 <dc:rights> <rdf:Alt> <rdf:li xml:lang="x-default">2014 John Scot Huggins, All Rights Reserved</rdf:li> </rdf:Alt> </dc:rights>

<dc:rights>
   <rdf:Alt>
    <rdf:li xml:lang="x-default">                                                      </rdf:li>
   </rdf:Alt>
  </dc:rights>

All the other fields appear to be moving through the translations fine, but the one containing copyright information from the original embedded XML tag seems to have turned to spaces. Might I be doing something wrong?

I am using 0.25 on Ubuntu, compiled from fresh source, but the older 0.19 version that came with my install of Ubuntu does the same thing.

John

Your message arrived when I was about push 'Submit'. I read'll your message (and look at your drawing) in a moment.

OK. A bit more analysis:

1) Extract the 'raw' XML and inspect it:

$ exiv2 -pX 2014-2.jpg | xmllint --pretty 1 -

      <dc:rights>
        <rdf:Alt>
          <rdf:li xml:lang="x-default">2014 John Scot Huggins, All Rights Reserved</rdf:li>
        </rdf:Alt>
      </dc:rights>

$ exiv2 -eX --force --verbose 2014-2.jpg  
File 1/1: 2014-2.jpg
Writing Exif data from 2014-2.jpg to ./2014-2.xmp
Writing XMP data from 2014-2.jpg to ./2014-2.xmp
$

    <dc:rights>
    <rdf:Alt>
     <rdf:li xml:lang="x-default">                                                      </rdf:li>
    </rdf:Alt>
   </dc:rights>

$ exiv2 -ex --force --verbose 2014-2.jpg
File 1/1: 2014-2.jpg
Writing XMP data from 2014-2.jpg to ./2014-2.exv
$ exiv2 -pX 2014-2.exv | xmllint --pretty 1 -

      <dc:rights>
        <rdf:Alt>
          <rdf:li xml:lang="x-default">2014 John Scot Huggins, All Rights Reserved</rdf:li>
        </rdf:Alt>
      </dc:rights>

Conclusions:

We are on the same page. We agree that:

1) Something is wrong with -eX that is causing the copyright to be blanked.
2) The data was successfully read from the image into XMPsdk, because it's being correctly presented by -pa :

$ exiv2 -pa 2014-2.jpg  | grep John
Exif.Image.Artist                            Ascii      19  John Scott Huggins
Xmp.photoshop.Credit                         XmpText    18  John Scott Huggins
Xmp.photoshop.CaptionWriter                  XmpText    18  John Scott Huggins
Xmp.dc.creator                               XmpSeq      1  John Scott Huggins
Xmp.dc.rights                                LangAlt     1  lang="x-default" 2014 John Scot Huggins, All Rights Reserved
$ 

I'll now focus my attention on -eX. I hope I can find and fix this today. I don't think I've ever worked in that part of the code, so wish me luck!

Thanks for the drawing. Very nice. I can more or less follow your plot and see where exiv2 fits into the scheme of things.

There's a rule in life which English people call "jam side downwards". If a slice of bread and jam slips off the table, it will always land on the jam side to ensure the maximum mess. It might even land on the dog!

In your case, the most important metadata element is what Photo Mechanic calls "Copyright" which is XMP:

<dc:rights> .... </dc:rights>

You now have two work-arounds - itemised above as (1) exiv2 -pX and (3) exiv2 -ex I will investigate (2) exiv2 -eX and let you know what I discover (and hopefully fix).

I think you'll find that (1) exiv2 -pX is a good solution as the XMP/xml can be piped directly from the image to any file of your choosing. When you use exiv2 -ex and exiv2 -eX, there is an intermediate file which may (or may not) be convenient.

Incidentally, there is a feature to be added in v0.26 to redirect extract/import data to/from stdout/stdin #1074. I haven't implemented this yet, however it's intended to work as follows:

$ exiv2 -eX- foo.jpg

$ exiv2 -iX- foo.jpg

$ exiv2 -eC- foo.jpg | exiv2 -iC- another.tiff

$ exiv2 -eC- foo.jpg | exiv2 -iCi a.tiff b.png c.jpg ...

Thank You for using Exiv2.

The stdin/stdout capability is precisely what I was hoping for. Not finding an way to do this at the moment, I do the steps in the green box of my flow chart using bash managing a temporary 2014-2.xmp file. Right now I just copy the 2014-2.xmp to several files matching the derivative file names and then use exiv2 to insert data into each with full control of what each file gets with an exiv2 command with specific options for each. A little clunky, but then again bash makes this a reasonable process.

While the stdin/stdout aspect is certainly the way to go, I need to be sure the appropriate 'translation' occurs when writing XMP data to both XMP and IPTC of the original file. I think I read in the docs when using the .xmp file as source, a proper translation to IPTC occurs when adding to a file.

Looks like you guys have got it figured out for the most part!

I'll just step in and ask if a request has been submitted to PhotoStore asking them to follow the MWG guidelines?

http://www.metadataworkinggroup.org/pdf/mwg_guidance.pdf#page=42

I really like bash. The syntax is ugly because bash scripting was layered on top of the unix command shell. However it is polished, robust and omni-present tool. Every Engineer should learn bash thoroughly.

Alan Pater is our XMP Engineer. Alan's not a C++ guy, he's a work-flow/metadata Engineer and understands the mysterious IPTC<->Exif<->XMP conversions. He has something very big and important happening in his life/business, so let's only consult him when/if we are stuck.

Alan: I've just realised that you've been reading this. Thanks for getting involved. I think we'll get this one fixed today.

Alan Pater wrote:

Looks like you guys have got it figured out for the most part!

I'll just step in and ask if a request has been submitted to PhotoStore asking them to follow the MWG guidelines?

http://www.metadataworkinggroup.org/pdf/mwg_guidance.pdf#page=42

Thanks much for this. I should try to follow this standard myself to make sure my goals are not unrealistic and because standards are "good." A request to read XMP has been forwarded to Ktools, but I'm glad to see someone made a document with solid suggestions on how to survive in the new age, especially when more than one format (IPTC, XML, etc.) exist at the same moment within an image. It's like the old saying... If you have one clock, you know what time it is, where if you have more than one, you are never sure.

In Ktools' defense, I did find some commented out XML parsing code in their files that is clearly a hint of the future of decoding XMP directly so I know they have been thinking about it. Photostore is a pretty old program (2004 or so birth), but is still in development... and precisely what I desire in a stock photo online presence.

In more defense of Ktools Photostore, I will say despite their inability to handle XMP natively (for whatever reason - probably just haven't caught up with the modern ways), they have a large body of well polished PHP code that works astonishingly well. Thus the reason I'm bothering to iron out a few wrinkles (based on my own personal requirements) with the back-end scripting with exiv2. It's kind of amazing they left this nice processing-gap between upload and import providing the opportunity to tweak the files. Life doesn't usually work that way.

In the end, I hope to have a solution that allows me to tag my main archive just once, and only once, with my preferred tools, and have that data flow to various targets automatically. Thanks to exiv2 and this bug tracking system, I am likely less than a week away from meta data nirvana... until the standards change again of course... or I, knowing my luck thus far, stumble on a bug in bash or something :|

Cheers

John

It's been a long day. It's 8:23pm here in England. We're building/remodelling our house and I've had to spend quite a lot of time on that today. Time for some beer. I'll continue on your issue tomorrow. http://clanmills.com/Homepages/2015/Christmas.shtml

You might find a bug in bash. I'll be surprised. I recently found a bug in grep. Unbelievable - but true. #1105 http://dev.exiv2.org/issues/1105

Standards are a good thing. However the computer industry prefers competition. Why do we need Exif, XMP, IPTC, ICC and on and on. Oh, ascii data representation: XML and JSON. Unix shells - bash, csh, ksh. Data containers: TIFF, PDF and many others. The fattest book on my book shelf is "Graphics File Formats" - 1500 pages of duplication, quirkiness and overlapping nonsense. As the saying goes "I love Standards - there are so many to choose from!".

In the Exiv2 project, our aim is to provide a "one stop shop for metadata". We have XMP/IPTC/Exif under one roof on Mac/Linux/Windows. We're adding some ICC and PDF support in v0.26. If the engineers from Adobe, Apple, Google and Microsoft joined Exiv2, we could have a wonderful metadata engine. We could all do less work and achieve more!

Understood on all counts. Thanks for your time.

Some success to report given all I've learned on this thread. This script...

#!/bin/bash

# obtain XML meta tags from primary imaage and store in .xmp sidecar file
/usr/local/bin/exiv2 -pX $1.jpg > $1.xmp

# copy the xmp file to the derivative file root names.xmp
cp $1.xmp icon_$1.xmp
cp $1.xmp sample_$1.xmp
cp $1.xmp thumb_$1.xmp

# strip the tags from the primary and derivative images
/usr/local/bin/exiv2 -d a $1.jpg
/usr/local/bin/exiv2 -d a icon_$1.jpg
/usr/local/bin/exiv2 -d a sample_$1.jpg
/usr/local/bin/exiv2 -d a thumb_$1.jpg

# insert XML back into primary image along with the IPTC for Ktools to use
/usr/local/bin/exiv2 -iixX $1.jpg

# insert only XML into the derivative images
/usr/local/bin/exiv2 -ixX icon_$1.jpg
/usr/local/bin/exiv2 -ixX sample_$1.jpg
/usr/local/bin/exiv2 -ixX thumb_$1.jpg

# delet the xmp sidecar files
rm *$1.xmp

...is now performing my goals quite well as can be seen in the final (temporary) web display of this handsome beast...

http://huggins.photography/media.details.php?mediaID=104

The appropriate data is stored in the Photostore database and the various derivative images are tagged precisely the way I want them to be. The only caveat is the original image has both IPTC and XMP data (just so Photostore can read them), but now they are totally in sync (with IPTC a copy of the primary XMP data).

So the meat and potatoes works. Now I just have to figure out how to get Photostore to run the script after upload, but that's not a problem with exiv2.

There are probably 100 better ways of doing this, but it's nice to have something that works.

This is good news. Well done, John. There's more than one way to do anything. And for sure, getting code to work is step 1 in a process of refinement. For example, I'd use loops to shorten your code. Something like:

for i in "$1.jpg" "icon_$1.jpg" "sample_$1.jpg" "thumb_$1.jpg"; do
    /usr/local/bin/exiv2 -d a "$i" 
done

Here's the bill for help you have been given. I'd like you to visit OpenHub.net and register (it's free).
On the Exiv2 page, click I use this. Give the Exiv2 project a great rating and/or a review and/or give me kudos.

Exiv2: https://www.openhub.net/p/exiv2
Clanmills: https://www.openhub.net/p/exiv2/contributors/60065117754297

I'm not going to submit the code for -eXX today. I've written it and surprised that it is more complicated than expected. It's 9:51pm. I'll sleep on this and do something about it tomorrow.

I'm not sure I understand why you have to run code on Photostore. Does their uploader remove metadata? Anyway, if it's not the concern of exiv2, I don't need to know.

Robin Mills wrote:

Here's the bill for help you have been given. I'd like you to visit OpenHub.net and register (it's free).
On the Exiv2 page, click I use this. Give the Exiv2 project a great rating and/or a review and/or give me kudos.

Exiv2: https://www.openhub.net/p/exiv2
Clanmills: https://www.openhub.net/p/exiv2/contributors/60065117754297

Will do as soon as I figure out a way to get an account without verification via phone or github.

I'm not going to submit the code for -eXX today. I've written it and surprised that it is more complicated than expected. It's 9:51pm. I'll sleep on this and do something about it tomorrow.

I'm not sure I understand why you have to run code on Photostore. Does their uploader remove metadata? Anyway, if it's not the concern of exiv2, I don't need to know.

The photostore upload process leaves the original file intact, but does strip metadata from the three derivative files it generates for thumbs, samples, etc. The second step of the process is to import them into the gallery structure. It is here the meta tags in the original file are examined to populate the details of the photo. As of today, it appears photostore only reads IPTC. With the exiv2 script I have now, my orginal and the derivative images are tagged with XMP and have IPTC copy added to the original file (just so photostore gets the information into its database.

Convoluted? Yes. Ideal? No, but it seems to work.

John