Project

General

Profile

Identified (but un-delete-able) XMP key data.

Added by Steve Wright almost 12 years ago

All:

I copied two JPEG files from some older Mac-authored data CDs, and wanted to purge as many of the out-of-date metadata as I could before opening them with any of my current image editors etc. I challenged myself by choosing to start with the XMP fields. I've run into one key that exiv2 (from the command-line) seems to have trouble deleting. The line reads:

Xmp.mediapro.People                          XmpBag      1  

The command I'm using to delete the key is:

exiv2 -M"del Xmp.mediapro.People" modify trin08.jpg 
(filename truncated for convenience)

And the error I'm getting is

-M option 1: Invalid key `Xmp.mediapro.People'

Which of course is as true as it goes, since this particular key doesn't appear in any of the lists. exiv2 can read it by name, however, so there must be another method by which it can be deleted without having to "lay waste" to the whole XMP packet.

Any suggestions?

SIlversleeves


Replies (26)

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel almost 12 years ago

Does it work if you add -M'reg mediapro myNamespace/' to the command to register the namespace? (I'm not sure if instead of myNamespace/ you need to use the actual URL for that namespace as it is defined in the image)

Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

Too late now for this particular JPEG file. I "bit the bullet" and deleted the key via Photoshop. But the other file I copied from the Mac disk was, I recall, similarly tagged (I was using iView Media Pro quite a bit in those days). So I'll keep your suggestion in mind when I work on that other file.

I just got the URL for iView/MS Expression Media from one of the latter company's online docs. It's http://ns.iview-multimedia.com/mediapro/1.0/ .
That's sure to help in the future, as I say. Hope it helps others, too.

Thanks again for the prompt reply.

Silversleeves

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

I was curious to see if your registering the namespace did indeed enable exiv2 to remove the "assets" in the XMP packet written by iView/Expression Media 2. So I found a JPEG that had one written in, not "People" (what iView calls "Contact") but "Event" (their name for the IPTC Application2 tag "FixtureId"). I tried -M"reg mediapro http://ns.iview-multimedia.com/mediapro/1.0/" followed by the name of the file, and it didn't return an error. However, when I typed in the command
exiv2 -M"del Xmp.mediapro.Event" foo.jpg
I got the same error as above (Invalid key). I'm thinking it may be a matter of syntax. When you use reg, are those supposed to be single quotes, or were you just in a hurry with your reply?

Silversleeves.

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel almost 12 years ago

Silversleeves,
Try with both commands in the same run of exiv2, e.g.,

exiv2 -M"reg mediapro http://ns.iview-multimedia.com/mediapro/1.0/" -M"del Xmp.mediapro.Event" foo.jpg

Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

Both commands on the same line appear to work. I also tried to delete another iView-originated XMP "asset" in the same JPEG file without including the -M"reg..." command before it, and it gave me the "Invalid Key" error. Is this 'normal' behavior for exiv2 when it comes to adding or deleting a key it doesn't recognize?

Silversleeves

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel almost 12 years ago

Yes, that's perfectly normal. Exiv2 doesn't know the namespace, so you need to tell it about its existance or it will complain. And since it has no external database to store and remember such information, you need to tell it on the same command line or in the same command file that contains the actual commands that you want to perform.

Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

As I'm sure I'm not the only exiv2 user who also uses MS Expression Media or its predecessor, iView Media Pro, is it at all possible that some time in the near future the XMP schemas for these apps could be added to exiv2's capabilities/"internal list"? If it's a matter of knowing the parameters etc., I could share as much information as I come across to facilitate this addition. It may mean something as simple as emailing a fully-annotated image file (which is to say, one with all the fields unique to Expression Media written into it) and your looking at the "raw hex" or the XML of the entries and puzzling out how best to make exiv2 write to and/or delete the field data.

This puts me in mind of another request I made, when I was still using a Mac, to Thorsten Lemke, author and maintainer of GraphicConverter X. I asked that an iView (it was still only iView at the time) field, "People," be added to his application's list of IPTC tags. Little did I know that in the IPTC 4 standard, "People" is actually known as "Contact."

This seems to be more than somewhat different. Nonetheless, I hope you'll give it some thought.

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

I've been giving some thought to the possibility of making a patch to the list of valid XMP data keys, adding the ones peculiar to iView and its MS successor, Expression Media. Which files should I look at to modify? And how many entries should I be looking for in those files?

Apropos of not much at all: I contrived an analogy for coding "half-way," doing one thing, leaving out two other things that were necessary to make the one thing work, and watching a minor error like this cascade to the point where even the unmodified files don't compile/integrate. Think of it as "trying to spell CAT with just the A."

S.

Steve Wright wrote:

Andreas,

As I'm sure I'm not the only exiv2 user who also uses MS Expression Media or its predecessor, iView Media Pro, is it at all possible that some time in the near future the XMP schemas for these apps could be added to exiv2's capabilities/"internal list"? If it's a matter of knowing the parameters etc., I could share as much information as I come across to facilitate this addition. It may mean something as simple as emailing a fully-annotated image file (which is to say, one with all the fields unique to Expression Media written into it) and your looking at the "raw hex" or the XML of the entries and puzzling out how best to make exiv2 write to and/or delete the field data.

This puts me in mind of another request I made, when I was still using a Mac, to Thorsten Lemke, author and maintainer of GraphicConverter X. I asked that an iView (it was still only iView at the time) field, "People," be added to his application's list of IPTC tags. Little did I know that in the IPTC 4 standard, "People" is actually known as "Contact."

This seems to be more than somewhat different. Nonetheless, I hope you'll give it some thought.

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel almost 12 years ago

I've been giving some thought to the possibility of making a patch to the list of valid XMP data keys, adding the ones peculiar to iView and its MS successor, Expression Media. Which files should I look at to modify? And how many entries should I be looking for in those files?

Only properties.cpp needs modifications. You can look at the definitions for an existing namespace and do something similar. E.g., look for the 3 occurences of 'Kipi' in that file.

Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

I couldn't find 'kipi' (being an off-again-on-again KDE user I know what it is), so I used one of the Photoshop constants as a base and referred back when there seemed to be a key type discrepancy or mismatch. I started off by annotating a JPEG with as many tags as could fit of all three field types (EXIF, IPTC, and XMP). Then I ran an exiv2 -pa command on the file, redirecting the output to a plain text file. From there I was able to copy the names for the constants and the fields. I stuck them at the end in the "class member definitions" section, then pasted in an edited bunch of constants (with my own definitions for any of the ones that didn't have counterparts/analogues elsewhere), also at the end.

Attached you'll find the final edit of properties.cpp I worked on. You've probably got better code checkers than just g++, so I'll leave it to you to check for validity, etc.

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel almost 12 years ago

Steve,

Thanks for the modified file. Can you please also add the namespaces to the xmpNsInfo[] array? Then I can add it.

Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

Done and done. I wondered where those URL-style namespaces went.

The modified file has a slightly-different name to distinguish it from the first so-called "final draft" (an oxymoron in itself).

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Tangential but as I was thinking about it, I thought I'd mention it.

A curious thing happens when you add data to the "IPTC Country Code' field in Expression Media. It gets written to the file as "Iptc.Application2.ExpirationTime," which, of course, causes exiv2 to complain that there is an "Unsupported time format" in the file. No doubt: IPTC Country Codes are as often as not two or three alpha characters in length: nowhere near a "time format" by any stretch of the imagination. I don't know whether, or if, you could write something into exiv2 that could cross-check the data length of Iptc.Application2.ExpirationTime against the presence of XMP fields of the iView or MSXM2 "group," but it might head off another newbie user's thread along the lines of "Getting 'Unsupported Time Format' errors and don't know why."

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel almost 12 years ago

Steve,

Thanks again for your contribution. I've checked in the changes with r2031. If you still have the code checked-out from the repository around somewhere, you can run 'svn update' in the exiv2 top directory, re-compile and try it out. Just a few more questions:

  • Is it correct that the two namespaces have exactly the same properties?
  • Could you please upload one or two samples here with metadata of both types?
  • Do you want this contribution attributed to 'Steve Wright' or 'Silversleeves' (or any other name) in the ChangeLog for the next release?

Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

Answering your questions in order:

  • Nearly so. I think I put in a Microsoft one for expressionmedia and the old indie one for mediapro. If I didn't I'll make that correction asap.
    What you have here is one of those occasions where M$ bought out a smaller company on the skids. Unlike the practise that got Microsoft summonsed by American state attorneys-general and the Federal Department of Justice, though, in this case the folks running the independent outfit closed up shop voluntarily. At any rate, Microsoft kept the field names and their parameters, and only changed the "label" and the namespace to reflect the change in ownership of the application as intellectual property.
  • Sure thing.
  • Silversleeves, please. Though more often I use the three-letter "BZT" (the Z said in the OED/French fashion, "zed"), Silversleeves will be fine.

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

Hope you don't mind a .zip file. There's a readme inside describing its contents.

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

After writing the readme for that .zip, I looked at the iView-generated XML in a text editor and found out they were sans namespace URLs for the app-specific data fields. This is the same zip, plus Photoshop CS1-made XM+P+ files corresponding to each picture in the archive. Now the line from the readme about "...will be just as good a source of the necessary changes as I am," means what I intended it to. I'd forgotten that one of the reasons Adobe applications can't make use of iView/MSXM XMLs is because the latter excludes namespace lines (just the thing we both wanted to see at this stage).

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel almost 12 years ago

Thanks for all this info.

In the attached files I can only find the Event property from the two namespaces in questions:

$ exiv2 -px *.xmp *.jpg *.exv | grep -e mediapro -e expressionmedia
Trolley.xmp           Xmp.mediapro.Event                           XmpText    16  Vacation Weekend
Trolley.xmp           Xmp.expressionmedia.Event                    XmpText    16  Vacation Weekend
ProvincetownTrolley19.10.2006-1152.jpg  Xmp.mediapro.Event                           XmpText    16  Vacation Weekend
ProvincetownTrolley19.10.2006-1152.jpg  Xmp.expressionmedia.Event                    XmpText    16  Vacation Weekend
ProvincetownTrolley19.10.2006-1152.exv  Xmp.mediapro.Event                           XmpText    16  Vacation Weekend
ProvincetownTrolley19.10.2006-1152.exv  Xmp.expressionmedia.Event                    XmpText    16  Vacation Weekend

But that can now be deleted easily:

$ exiv2 -v -M'del Xmp.expressionmedia.Event' Trolley.xmp
File 1/1: Trolley.xmp
Del Xmp.expressionmedia.Event
$ exiv2 -v -M'del Xmp.mediapro.Event' Trolley.xmp
File 1/1: Trolley.xmp
Del Xmp.mediapro.Event

As for the XML file, its contents are simply not in XMP format.

Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright almost 12 years ago

Andreas,

I ran the bikini pic again through MSXM2, adding a Catalog Set, Status and Contact. This time I'll upload just the pic.
Why the bikini pic, someone else's work, instead of the trolley pic which was my own (Olympus DC-M40 on very conservative settings)?
Because I happen to have established lists of catalog sets, "People" definitions and two Status entries, and the Trolley pic doesn't fit any of the above.

S.

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright over 11 years ago

Andreas,

In one respect, adding the iView/MSXM XMP elements may have been an exercise in futility. Last week, I concocted a special User Field (Custom Field in the GUI app) called "GarbageField". I made value entries, and tried to delete them via Exiv2 on the command line. Then I re-sync'ed the annotations on the appropriate files in Expression Media, and the field values came back, even though an 'exiv2 -pa' showed them as gone. So I looked at one of the JPEG files in a hex editor. Sure enough, in the ASCII column, just below the end of the XMP packet, there were the exact same values as strings. Which leads me to believe that, somewhere along the line, either iView Multimedia, LLC or Microsoft deemed it necessary that files with Custom Fields should only have those field values removed by way of their application/s.

The same is also so for the more commonly-used Catalog Sets value strings (I just checked on that using, again, a bikini pic for which I had set and 'running' catalog set values in my iView catalogs and the same hex viewer/editor). And, ironically and probably most disappointingly of all, the "People" tag that was discussed in the first few posts in this thread.

So unless there is by some means familiar to you a way to remove these 'stubborn' strings that appear outside the XMP packet bedises the more ordinary "del Xmp.expressionmedia.thisnthat", it would probably be courteous somewhere along the line to make mention of the fact that some iView/MSXM strings don't trash completely when deleted via Exiv2.

Steve Wright

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel over 11 years ago

Hi Steve,

Can you please send me (or attach) a picture with this kind of extra data, before any modification with Exiv2 was done to it. I'd like to also look at in in a hex editor.

Thanks,
Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright over 11 years ago

Good as done.'

Included are my own "dumps" from HxD v1.7.7 in Windows XP.

The picture is a keeper, too, as I hope you'll agree.

Steve W.

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright about 11 years ago

Andreas,

Any progress on this, or do the "surfer dudes" who wrote the original app and devised the means of "burying" this information for later "recall" -- I personally don't credit M$ with being that clever (not with initials SB still on the payroll) -- have you stumped?

Steve Wright

RE: Identified (but un-delete-able) XMP key data. - Added by Andreas Huggel about 11 years ago

This one fell through the cracks. I haven't looked at it yet. Will put it higher on the list...

Andreas

RE: Identified (but un-delete-able) XMP key data. - Added by Steve Wright about 11 years ago

It looks like you might want to take a look at "extract and import" (to .exv files). I just created one, then viewed it in a hex editor. It had the "below-the-packet" stuff that the iView MP guys (or Microsoft, can't tell which) coded into the app to stymie deletion of some data by any other means than MSXM. This is the same place the app 'stows' the custom field data...

...though in my case this morning it was just repetition of pretty ordinary PS IFD stuff and/or standard IPTCv4 data.

I'll run a "delete all" on this same JPEG and check it to see if this 'parroted' stuff below the usual XMP packet's bottom end is still left behind in 0.20 as it was in 0.19. I suspect it will be, but it's a place to start at any rate.

Steve W

(1-25/26)