Project

General

Profile

Empty elements in a rdf:Seq

Added by Tobias E. over 7 years ago

Hello,

after reading the API docs for a while now without finding anything that's helping me I guess I need your help.

I have to read XMP files containing a rdf:Seq which can contain empty elements. The XML looks something like this:

  <rdf:Seq>
    <rdf:li>foo</rdf:li>
    <rdf:li>bar</rdf:li>
    <rdf:li/>
    <rdf:li>baz</rdf:li>
  </rdf:Seq>

In my use case not only the order of these matters, but also that the 3rd entry is an empty one. If I use findKey() I can get an iterator to that sequence, but count() only returns 3. Consequently toString(3) crashes. Is there any way to make libexiv2 honour these empty entries, too?

Thanks
Tobias


Replies (24)

RE: Empty elements in a rdf:Seq - Added by Andreas Huggel over 7 years ago

Tobias,

That doesn't sound right and it certainly shouldn't crash. Can you provide an image or xmp file that I could play with? Please attach it here or send it to ahuggel <at> gmx <dot> net

Andreas

RE: Empty elements in a rdf:Seq - Added by Tobias E. over 7 years ago

Hi Andreas,

here is a small XMP file and test program.

Tobias

crash.xmp (506 Bytes) crash.xmp XMP file with a rdf:Seq that has one empty element
crash.cc (720 Bytes) crash.cc C++ test that crashes on crash.xmp in toString()

RE: Empty elements in a rdf:Seq - Added by Tobias E. over 7 years ago

I would like to make it clear that I don't think that crashing is the bug here (well, a minor one maybe), but that the empty element is not accounted for. And most importantly, I would like to know if there is any way at all to get that empty rdf:li element.

RE: Empty elements in a rdf:Seq - Added by Tobias E. about 5 years ago

Is there anyone who can help? This issue is causing major problems in darktable and I would like to find a way to get that solved.

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

I'll help you with this, Tobias. I'm not aware of the issue. I'm not sure what I'm looking for as I don't get a crash (on current trunk build). It looks as though you have 4 elements in Seq:

685 rmills@rmillsmbp:~/Downloads $ xmllint -pretty 1 crash.xmp 
<?xml version="1.0"?>
<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
  <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description xmlns:darktable="http://darktable.sf.net/" rdf:about="">
      <darktable:multi_name>
        <rdf:Seq>
          <rdf:li>foo</rdf:li>
          <rdf:li>bar</rdf:li>
          <rdf:li/>
          <rdf:li>baz</rdf:li>
        </rdf:Seq>
      </darktable:multi_name>
    </rdf:Description>
  </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>
686 rmills@rmillsmbp:~/Downloads $ 
Exiv2 is reporting 3:
683 rmills@rmillsmbp:~/Downloads $ exiv2 -pa crash.xmp 
Xmp.darktable.multi_name                     XmpSeq      3  foo, bar, baz
684 rmills@rmillsmbp:~/Downloads $
I've changed your code a little:
// g++ -W -Wall -g `pkg-config --cflags --libs exiv2` -o crash crash.cc
#include <exiv2/easyaccess.hpp>
#include <exiv2/xmp.hpp>
#include <exiv2/error.hpp>
#include <exiv2/image.hpp>
#include <exiv2/exif.hpp>

#include <stdio.h>

int main()
{
  const char* filename = "crash.xmp";
  Exiv2::Image::AutoPtr image = Exiv2::ImageFactory::open(filename);
  if(image.get() == 0) {
      printf("unable to open file %s\n",filename);
      return -1 ;
  }

  int result = 0 ;
  image->readMetadata();

  Exiv2::XmpData                 &xmpData = image->xmpData();
  Exiv2::XmpData::const_iterator  pos     = xmpData.findKey(Exiv2::XmpKey("Xmp.darktable.multi_name"));
  if(pos != xmpData.end()) {
    printf("found %ld entries\n", pos->count());
    for ( int i = 0 ; i < pos->count() ; i++ ) {
        std::cout << i <<" : " << pos->toString(i) << std::endl;
    }
  } else {
      printf("not found\n");
      result = 1;
  }
  return result;
}
When I run ./crash, it returns 3 in the Seq.
689 rmills@rmillsmbp:~/Downloads $ g++ -W -Wall -g `pkg-config --cflags --libs exiv2` -o crash crash.cc
690 rmills@rmillsmbp:~/Downloads $ ./crash
found 3 entries
0 : foo
1 : bar
2 : baz
691 rmills@rmillsmbp:~/Downloads $ 
Can we agree on the issue here.

1) There is no crash here. Correct?
2) We are reporting 3 li elements in Seq and you are expecting 4. Is that correct?

I don't know anything much about XMP. I'm a C++ build engineer. Is there a rule in XMP that says "ignore empty Seq/li elements?"

I've changed crash.xml. First, I changed <rdf:li/> to <rdf:li></rdf:li> and that made no difference. Then I changed it to <rdf:li>bullshit</rdf:li>. Good result:

693 rmills@rmillsmbp:~/Downloads $ ./crash
found 4 entries
0 : foo
1 : bar
2 : bullshit
3 : baz
694 rmills@rmillsmbp:~/Downloads $ 
It feels to me that the XMPsdk is hiding the empty <rdf:li><rdf:li/> tag. What do you expect it to do?

Curiously, if I change "bullshit" in crash.xml to " " (a single space), the count == 4 and element 3 is a space. When I change "bullshit" in crash.xml to the empty string, the count returns to 3. There's something determined to hide empty list elements.

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

There's a discussion here about something similar: http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=5237.msg25351#msg25351

Phil (Harvey) says:

The empty lists were removed, but no actual metadata was lost.

Darktable should not be writing empty lists. It is a waste of space.

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

More amazing discoveries after attempting to understand Adobe's and W3C's documents about XMP and RDF.

<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about="" xmlns:darktable="http://darktable.sf.net/" >
   <darktable:multi_name>
    <rdf:Seq>
     <rdf:li>foo</rdf:li>
     <rdf:li>bar</rdf:li>
     <rdf:li rdf:resource=""/>
     <rdf:li>baz</rdf:li>
    </rdf:Seq>
   </darktable:multi_name>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>
Run this through ./crash:
713 rmills@rmillsmbp:~/Downloads $ ./crash
found 3 entries
0 : foo
1 : bar
2 : baz
714 rmills@rmillsmbp:~/Downloads $ 
Now change resource="" to resource="911" :
./crash
found 4 entries
0 : foo
1 : bar
2 : 911
3 : baz
715 rmills@rmillsmbp:~/Downloads $ 
There's code inside the xmpsdk (which came from Adobe) which does not like empty list elements in a Seq.

RE: Empty elements in a rdf:Seq - Added by Tobias E. about 5 years ago

Hello Robin,

thanks for looking into this. I am still on 0.25 which does crash. But as I said, that's only a side issue. The real thing is, as you correctly understood, the missing empty element. I haven't tried compiling libexiv2 myself, but I can give it a try to see how that works.

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

I did some debugging on this last night. There is no doubt in my mind that XMPsdk removes empty list elements. However, I don't know why. I'm not an expert in metadata.

I've searched the Adobe/XMP specs, the W3C/RDF spec and the comments in the code in xmpsdk. I haven't found anything to say that an empty li element is legal or illegal. For sure, it seems an odd thing to put into XMP. There are three very similar RDF containers. Bag, Seq, and Alt. For a 'Bag', you should think of a mathematical set of entities. A 'Seq' is an ordered set of entities. An 'Alt' is an unordered set from which you are expected to choose one. So, it's kind of odd to say "I want an empty element" in a 'Seq'. It's like saying "Waiter! I want a black hole in my soup!".

I've also looked at your file crash.xmp with the latest version of Adobe's XMPsdk (released July 2016).

553 rmills@rmillsmbp:~/gnu/xmpsdk/XMP-Toolkit-SDK-CC201607 $ samples/target/macintosh/intel_64/Release/ReadingXMP ~/Downloads/crash.xmp 
No smart handler available for /Users/rmills/Downloads/crash.xmp
Trying packet scanning.

/Users/rmills/Downloads/crash.xmp is opened successfully
dc:title in English = 
dc:title in French = 

XMP dumped to XMPDump.txt
554 rmills@rmillsmbp:~/gnu/xmpsdk/XMP-Toolkit-SDK-CC201607 $ cat XMPDump.txt

Dumping XMPMeta object ""  (0x0)

   darktable:  http://darktable.sf.net/  (0x80000000 : schema)
      darktable:multi_name  (0x600 : isOrdered isArray)
         [1] = "foo" 
         [2] = "bar" 
         [3] = "baz" 
555 rmills@rmillsmbp:~/gnu/xmpsdk/XMP-Toolkit-SDK-CC201607 $ 
100% proof that the Adobe XMPsdk quietly rejects empty list elements.

I'd like to know more about why this is causing you to be unhappy. Why is DarkTable generating this XMP? Have you spoken to our friends at DT about this?

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

I've raised a topic on the Adobe XMPsdk Forum about this matter. https://forums.adobe.com/message/8968850#8968850

RE: Empty elements in a rdf:Seq - Added by Alan Pater about 5 years ago

Would updating the XMPSDK help with this issue?

http://dev.exiv2.org/issues/941

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

Regrettably not. As you can see above, I built the latest XMPsdk and ran the sample application ReadingXMP on Tobias' code. The <rdf:li/> tag was silently ignored.

I think we'll have to get input from an XMP "language lawyer" at Adobe to say why this is ignored. Equally, we need to hear from DarkTable about the meaning of this code. I'm not sure the XMPsdk (and therefore libexiv2) can create that XMP. So, lots of strange things about this. I believe exiv2 is blameless in this situation.

Incidentally, I have not been able to reproduce the crash that is alleged with v0.25. However, Tobias has been explicit in asking us to focus on the empty element. So the crash is of no importance.

BTW, I downloaded and built the latest Adobe XMPsdk about a week ago. I have started looking at that in anticipation of integrating it into Exiv2 in v0.27.

RE: Empty elements in a rdf:Seq - Added by Tobias E. about 5 years ago

The XMP files with the empty elements were created by libexiv2. In my sample I just deleted some unrelated things to make it more obvious what is going on. To explain why darktable relies on those empty elements I have to show a more complete example. I still removed some unrelated elements, but the important parts are there:

<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about="" 
    xmlns:darktable="http://darktable.sf.net/">
   <darktable:history_operation>
    <rdf:Seq>
     <rdf:li>sharpen</rdf:li>
     <rdf:li>flip</rdf:li>
     <rdf:li>basecurve</rdf:li>
     <rdf:li>tonecurve</rdf:li>
     <rdf:li>tonecurve</rdf:li>
     <rdf:li>tonecurve</rdf:li>
    </rdf:Seq>
   </darktable:history_operation>
   <darktable:history_params>
    <rdf:Seq>
     <rdf:li>000000400000003f0000003f</rdf:li>
     <rdf:li>ffffffff</rdf:li>
     <rdf:li>gz09eJxjYIAAM6vnNnqyn22E9n235b6aa3cy6rVdRaK9/Y970fYf95bbMzA0QPEoGEqADYnNhEUeAHH1EJo=</rdf:li>
     <rdf:li>gz06eJxjYICAcvFa669/NltdSOewY77w1NY+e5ndCreddgESofZT1qXYMzA0QPHAAK7ri21BeNbMmXYgDBQCucXe2NgYjOVbs8F4oNw5WN3HBsTsUIwMGJFoAHvBKzs=</rdf:li>
     <rdf:li>gz12eJxjYIAA6/BO2x0r620YGBrsIXjQAXsIHnUfKYAZCTMhYUaoPIgGALM+CPw=</rdf:li>
     <rdf:li>gz13eJxjYICAe7b29vrVpfYMDA1QPOiAPQSPuo8UwIyEmZAwI1QeRAMAvvwIfw==</rdf:li>
    </rdf:Seq>
   </darktable:history_params>
   <darktable:multi_name>
    <rdf:Seq>
     <rdf:li/>
     <rdf:li/>
     <rdf:li/>
     <rdf:li/>
     <rdf:li>1</rdf:li>
     <rdf:li>2</rdf:li>
    </rdf:Seq>
   </darktable:multi_name>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

In darktable:history_operation there is an ordered list of image editing operations. In darktable:history_params there are the settings for those, and darktable:multi_name holds the instance names for those to make the multiple applications of the tonecurve distinguishable. If the empty elements are kept darktable can assign an empty name to the first 4 operations and "1" and "2" to the last two. When empty elements are skipped however, the non-empty names are applied to sharpen and flip which is wrong.

Of course that layout isn't ideal and some day™ the whole schema should be redesigned to have one set of settings per operation which includes everything, but as there are millions of those XMP files out in the wild it would be great if reading them back as intended was somehow possible.

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

Thanks for the clarification. I'll investigate how to create such a pattern of XMP with libexiv2. I hope somebody on the Adobe Forum provides some insight concerning this matter as this behaviour of XMPsdk seems quite intentional.

Here's a proposal about how to fix this. We can read/rewrite the XMP using expat to convert the empty tags from <rdf:li/> to <rdf:li>X</rdf:li>. The content of X is any string except the empty string. It could be a (space) , 0 or any string such as empty. So the following transform would occur:

   FROM                    ->   TO
   <darktable:multi_name>       <darktable:multi_name>                                               
    <rdf:Seq>                    <rdf:Seq>                 
     <rdf:li/>                    <rdf:li>empty</rdf:li>                                     
     <rdf:li/>                    <rdf:li>empty</rdf:li>                
     <rdf:li/>                    <rdf:li>empty</rdf:li>                
     <rdf:li/>                    <rdf:li>empty</rdf:li>                
     <rdf:li>1</rdf:li>           <rdf:li>1</rdf:li>                         
     <rdf:li>2</rdf:li>           <rdf:li>2</rdf:li>                         
    </rdf:Seq>                   </rdf:Seq>
The code to fix this could reside in DT. DT can extract the "raw" XMP using image->printStructure(), modify it (with expat) and if there were changes write it back using image->setXmpPacket(). Or we could add a new API to libexiv2 such as image->rewriteEmptySeqLiElements("empty").

Let's talk some more about this. For sure, I'm confident that we can achieve your goal to respect the millions of existing DT files with this troublesome XMP. We'll raise a Redmine Feature Request on libexiv2 when we have agreed on the appropriate remedy.

RE: Empty elements in a rdf:Seq - Added by Tobias E. about 5 years ago

Thank you, that helped a lot. Using image->xmpPacket() and image->setXmpPacket() I can manually fix the XML data just fine. No need for any extra libexiv2 API I guess.

However, this leads me to a second question: In order to avoid problems like this in the future I would like to change the data layout to something like this:

<darktable:history>
  <rdf:Seq>
    <rdf:li>
      <darktable:modversion></darktable:modversion>
      <darktable:enabled></darktable:enabled>
      <darktable:operation></darktable:operation>
      <darktable:params></darktable:params>
      <darktable:blendop_params></darktable:blendop_params>
      <darktable:blendop_version></darktable:blendop_version>
      <darktable:multi_priority></darktable:multi_priority>
        <darktable:multi_name></darktable:multi_name>
    </rdf:li>
  <rdf:Seq>
</darktable:history

Following the examples from the API docs I am able to put everything into attributes of the <rdf:li> using code like this

  Exiv2::XmpProperties::registerNs("http://darktable.sf.net/", "darktable");

  Exiv2::XmpData xmpData;

  Exiv2::XmpTextValue tv("");

  tv.setXmpArrayType(Exiv2::XmpValue::xaSeq);
  xmpData.add(Exiv2::XmpKey("Xmp.darktable.history"), &tv); // Set the array type.
  tv.setXmpArrayType(Exiv2::XmpValue::xaNone);

  tv.read("flip");
  xmpData.add(Exiv2::XmpKey("Xmp.darktable.history[1]/darktable:operation"), &tv);

  tv.read("1");
  xmpData.add(Exiv2::XmpKey("Xmp.darktable.history[1]/darktable:priority"), &tv);

  tv.read("spot");
  xmpData.add(Exiv2::XmpKey("Xmp.darktable.history[2]/darktable:operation"), &tv);

  tv.read("2");
  xmpData.add(Exiv2::XmpKey("Xmp.darktable.history[2]/darktable:priority"), &tv);

Is there any way to get the format I want, i.e., having proper XML tags in the <rdf:li>?

Tobias

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

I'm a little lost. Can we divide your question into parts:

1 What XML do you want?
2 Is it legal XMP?
3 How do you get the exiv2 API to build it?

Example: DT.xmp:

617 rmills@rmillsmbp:~/gnu/exiv2/trunk $ xmllint --pretty 1 DT.xmp 
<?xml version="1.0"?>
<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
  <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description xmlns:darktable="http://darktable.sf.net/" rdf:about="">
      <darktable:multi_name>
        <rdf:Seq>
          <rdf:li>
            <rdf:Bag>
              <darktable:modversion>xxx</darktable:modversion>
              <darktable:enabled>yyy</darktable:enabled>
            </rdf:Bag>
          </rdf:li>
        </rdf:Seq>
      </darktable:multi_name>
    </rdf:Description>
  </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>
618 rmills@rmillsmbp:~/gnu/exiv2/trunk $ xmllint --pretty 1 DT.xmp  | wc
      19      30     659
619 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa DT.xmp 
Error: XMP Toolkit error 102: Named children only allowed for schemas and structs
Warning: Failed to decode XMP metadata.
620 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
When you have a pattern of XML/XMP that is accepted by exiv2/xmpsdk, play with the exiv2 executable to figure out how to generate it.

However before we discuss the API, can you define the XMP/XML please.

RE: Empty elements in a rdf:Seq - Added by Tobias E. about 5 years ago

Using exiv2 to have a look at the actual Xmp path to be used was a good hint, it seems that Xmp.darktable.history[1]/?darktable:mod version does the trick. However, the final XML is quite bloated so I guess I will just keep everything in attributes.

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

Like this?

629 rmills@rmillsmbp:~/gnu/exiv2/trunk $ cat DT.xmp 
<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about="" xmlns:darktable="http://darktable.sf.net/" >
   <darktable:multi_name>
    <rdf:Seq>
     <rdf:li rdf:resource="1" darktable:modversion="abc" darktable:parameters="ABC"/>
     <rdf:li rdf:resource="2" darktable:modversion="xyz" darktable:parameters="XYZ"/>
    </rdf:Seq>
   </darktable:multi_name>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>630 rmills@rmillsmbp:~/gnu/exiv2/trunk $ exiv2 -pa DT.xmp 
Xmp.darktable.multi_name                     XmpText     0  type="Seq" 
Xmp.darktable.multi_name[1]                  XmpText     1  1
Xmp.darktable.multi_name[1]/?darktable:modversion XmpText     3  abc
Xmp.darktable.multi_name[1]/?darktable:parameters XmpText     3  ABC
Xmp.darktable.multi_name[2]                  XmpText     1  2
Xmp.darktable.multi_name[2]/?darktable:modversion XmpText     3  xyz
Xmp.darktable.multi_name[2]/?darktable:parameters XmpText     3  XYZ
631 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

Good. I think this has a happy ending with no action required by Exiv2. Do you agree?

RE: Empty elements in a rdf:Seq - Added by Tobias E. about 5 years ago

Indeed. Thank you for your help!

RE: Empty elements in a rdf:Seq - Added by Tobias E. about 5 years ago

Sorry to bother you again. I didn't want to start a new topic for this follow up question.

I am now writing XMP files like this:

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about="" 
    xmlns:darktable="http://darktable.sf.net/" 
   darktable:xmp_version="2">
   <darktable:history>
    <rdf:Seq>
     <rdf:li>
      <rdf:Description
       darktable:operation="sharpen" 
       darktable:enabled="1">
      <darktable:settings>
       <rdf:Seq>
        <rdf:li
         darktable:name="width" 
         darktable:type="int" 
         darktable:value="23"/>
        <rdf:li
         darktable:name="height" 
         darktable:type="int" 
         darktable:value="42"/>
       </rdf:Seq>
      </darktable:settings>
      </rdf:Description>
     </rdf:li>
     <rdf:li
      darktable:operation="flip" 
      darktable:enabled="1"/>
     <rdf:li
      darktable:operation="tonecurve" 
      darktable:enabled="1"/>
    </rdf:Seq>
   </darktable:history>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Using libexiv2 I can get an iterator to Xmp.darktable.history and iterate that, however since that isn't a simple array I get XmpText values only, containing everything in that subtree:

Xmp.darktable.history = type="Seq" 
Xmp.darktable.history[1] = type="Struct" 
Xmp.darktable.history[1]/darktable:operation = sharpen
Xmp.darktable.history[1]/darktable:enabled = 1
Xmp.darktable.history[1]/darktable:settings = type="Seq" 
Xmp.darktable.history[1]/darktable:settings[1] = type="Struct" 
Xmp.darktable.history[1]/darktable:settings[1]/darktable:name = width
Xmp.darktable.history[1]/darktable:settings[1]/darktable:type = int
Xmp.darktable.history[1]/darktable:settings[1]/darktable:value = 23
Xmp.darktable.history[1]/darktable:settings[2] = type="Struct" 
Xmp.darktable.history[1]/darktable:settings[2]/darktable:name = height
Xmp.darktable.history[1]/darktable:settings[2]/darktable:type = int
Xmp.darktable.history[1]/darktable:settings[2]/darktable:value = 42
Xmp.darktable.history[2] = type="Struct" 
Xmp.darktable.history[2]/darktable:operation = flip
Xmp.darktable.history[2]/darktable:enabled = 1
Xmp.darktable.history[3] = type="Struct" 
Xmp.darktable.history[3]/darktable:operation = tonecurve
Xmp.darktable.history[3]/darktable:enabled = 1

Is there any way to access that in a more structured way, without the type attributes and as arrays per level? Or am I required to do string matching against the keys and parse that myself?

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

You're not bothering me. I'm always happy to help you and all our friends at Dark Table (and GIMP, digiKam and all users of Exiv2 technology).

I'm not sure I've really understood your question, however I think it's involved with the rather awkward syntax for querying attributes. I attach the file Tobias.jpg which I made from test/data/Reagan.jpg and inserting your XMP with the v0.26 command $ exiv2 -iXX Tobias.jpg which reads Tobias.xmp and replaces the XMP.

Run sample/exiv2json.cpp on Tobias.jpg and the XMP will be converted to JSON which you can read with your favourite JSON parser.

579 rmills@rmillsmbp:~/gnu/exiv2/trunk $ bin/exiv2json -x Tobias.jpg  
{
    "Xmp": {
        "darktable": {
            "xmp_version": "2",
            "history": [
                {
                    "darktable": {
                        "operation": "sharpen",
                        "enabled": "1",
                        "settings": [
                            {
                                "darktable": {
                                    "name": "width",
                                    "type": "int",
                                    "value": "23" 
                                }
                            },
                            {
                                "darktable": {
                                    "name": "height",
                                    "type": "int",
                                    "value": "42" 
                                }
                            }
                        ]
                    }
                },
                {
                    "darktable": {
                        "operation": "flip",
                        "enabled": "1" 
                    }
                },
                {
                    "darktable": {
                        "operation": "tonecurve",
                        "enabled": "1" 
                    }
                }
            ]
        },
        "xmlns": {
            "darktable": "http:\/\/darktable.sf.net\/" 
        }
    }
}
580 rmills@rmillsmbp:~/gnu/exiv2/trunk $
samples/exiv2json.cpp is sample code and not part of the library. You are welcome to copy samples/exiv2json.cpp (and Jzon.cpp and Jzon.h) into DT if you find that code useful.

Another approach is to extract the XMP with image->printStructure() or image->xmpPacket() and manipulate it with an XML library such as expat or Sax.

If you don't have exiv2json available, you can build it from source, or download it for your platform from our buildserver: http://exiv2.dyndns.org:8080/userContent/builds/Categorized/Latest/

RE: Empty elements in a rdf:Seq - Added by Robin Mills about 5 years ago

I should have pointed out another couple of things:

1 samples/exiv2json.cpp is a deep recursive parser of the Exiv2/XMP Syntax. It rebuilds the data into a tree of JSON objects and arrays. The code is non-trivial.
2 you can use your favourite JSON parser to decode the JSON or you can use the api in Jzon.h/Jzon.cpp to navigate the tree.

//// I'm a little bored and have wondered "Is there a tool like XSLT or SQL for JSON?" Yes. jq (Json Query, I think).

706 rmills@rmillsmbp:~/gnu/exiv2/trunk $ jq ".Xmp.darktable.history|length" Tobias.json
3
707 rmills@rmillsmbp:~/gnu/exiv2/trunk $ jq ".Xmp.darktable.history[2]" Tobias.json
{
  "darktable": {
    "operation": "tonecurve",
    "enabled": "1" 
  }
}
708 rmills@rmillsmbp:~/gnu/exiv2/trunk $ for i in $(seq 0 1 $(( $(jq ".Xmp.darktable.history|length" Tobias.json) -1)));do echo operation: $(jq ".Xmp.darktable.history[$i].darktable.operation" Tobias.json); done
operation: "sharpen" 
operation: "flip" 
operation: "tonecurve" 
709 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 
You can even read the XMP over the internet using Exiv2/webready and not bother with having the file Tobias.json
722 rmills@rmillsmbp:~/gnu/exiv2/trunk $ json=$(bin/exiv2json -x http://dev.exiv2.org/attachments/download/1048/Tobias.jpg)
723 rmills@rmillsmbp:~/gnu/exiv2/trunk $ for i in $(seq 0 1 $(( $(echo $json|jq ".Xmp.darktable.history|length") -1)));do echo operation: $(echo $json|jq ".Xmp.darktable.history[$i].darktable.operation"); done
operation: "sharpen" 
operation: "flip" 
operation: "tonecurve" 
724 rmills@rmillsmbp:~/gnu/exiv2/trunk $ 

    (1-24/24)