Video metadata support
|Status:||Closed||Start date:||19 Feb 2012|
|Assignee:||Abhinav Badola||% Done:|
Exiv2 should support video metadata.
The same task was also a GSoC project in 2011. However the proposal we received wasn't ranked high enough to get one of the slots allocated to the KDE project. The proposal is still online , I'm not sure if it's accessible publicly, but if so, go have a look at that too.
In a nutshell, digiKam uses the Exiv2 library  for metadata support. Exiv2 supports reading and writing Exif, IPTC and XMP metadata in images of various formats . It does not currently support any video formats. Extending Exiv2 to know about video metadata is the fundamental part of this project and will enable digiKam to deal with video metadata just like it deals with image metadata.
The programming task for the GSoC project will involve changes in both Exiv2 and digiKam. This Exiv2 feature is for the Exiv2 portion of the changes.
The Exiv2 library needs to be extended to support video metadata and/or video formats. For each video format which contains Exif, IPTC or XMP metadata, this essentially means adding an Image subclass that understands the format of the video file . Check out some of the existing Exiv2 image classes (files *image.[ch]pp) for details, e.g., JpegImage .
For video formats which contain video metadata different from Exif, IPTC and XMP, the task includes adding support for the new metadata standard, which is significantly more challenging than just adding Image subclasses. (I'll elaborate later if required.)
Some research and planning is necessary before any coding. Discussions should take place here in this feature and can be part of an eventual GSoC proposal: What are the popular video formats, are their specifications easily available, how is metadata stored in these formats and based on that, what are good candidates for this project?
[gsoc2012] #813: Added Some pre-existing samples of xmpData from the existing namespace XmpDM
[gsoc2012] #813: Created Custom Namespace Xmp.video and made all entries in riffvideo.cpp
[gsoc2012] #813: Implemented audioEncodingValues, removed some useless elements from riffvideo.cpp
[gsoc2012] #813: Added quicktimevideo.cpp to Makefile, Added WebM support in matroskavideo.cpp
[gsoc2012] #813: Applied patch in riffvideo.cpp and matroskavideo.cpp for xmp.video.codec
[gsoc2012] #813: Completed matroskavideo.cpp, no more modifications required for now. Only need to add Aspect Ratio.
[gsoc2012] #813: Patch riffvideo.cpp, fixed Info Decoder, prevented crash on scanning of WAVE file
[gsoc2012] #813: Simplified ignoreList() and dataIgnoreList() to avoid hardcoded list sizes.
[gsoc2012] #813: Removed ignoreList() and dataIgnoreList() completely now, since they are only used once each and consist of only one line of code.
[gsoc2012] #813: Addded Nikon NCTG tag decoding in quicktimevideo.cpp added some new lens in canonmn.cpp
[gsoc2012] #813: Added Most of the decoding functions in asfvideo.cpp, Worked on details of NCTG tags in quicktimevideo.cpp
[gsoc2012] #813: Changed the calculation of duration of all the videos to milliseconds from seconds
[gsoc2012] #813: More code simplifications: Use a single byte rather than a DataBuf when that is all we need.
[gsoc2012] #813: Changed all properties in Xmp.[video, audio] to Intial Capital Letter
[gsoc2012] #813: Changes in AsfVideo::contentDescription
- Added temp. assertion hack to catch potential crash when the buffer is not large enough
- Tried to fix use of types (not done: parameter size should be long)
- Removed unnecessary namespace identifier
- Removed localization call for tag names; they are not localized
[gsoc2012] #813: Addded Aspect Ratio in all the files, though a better unified method still needs to be made
[gsoc2012] #813: Fixed Content Creation Size Limit, created DataBuf inside loop, used byte array where required
#813: More work-in-progress Matroska polishing
- Pass const byte* buf and long size instead of DataBuf& buf to functions if no ownership transfer is involved
- Use bit-wise operations for calculation of tag and tag data (still needs work for negative numbers)
#813: Disabled some debug messages (temporary measure, just to avoid cluttering the video test output).
#3 Updated by Andreas Huggel over 6 years ago
KDE has been accepted as a mentoring organization for GSoC 2012 so this project is real! Looking forward to your proposals!
#5 Updated by Phil Harvey over 6 years ago
I feel sorry for the poor soul assigned to this task. Video metadata is a real hairball.
- There is a wild proliferation of video formats -- many different containers and many different video codecs for each container. Some container formats (ie. M2TS) don't support metadata at all, so one must parse the video stream to extract anything useful from these files (see http://owl.phy.queensu.ca/~phil/exiftool/commentary.html#AVCHD ).
- Even though some different video formats use the same container, there is no consistency between formats about how the metadata is stored in the container. The stupidity of the designers of these video formats often boggles the mind.
- The specifications for some video formats are imprisoned by ISO and expensive to acquire, if available at all. Publicly available documentation is often very poor, or non-existent.
- Much of the interesting metadata in videos from current digital cameras is stored in proprietary formats -- often very few of the "standard" metadata fields are populated. And worse, it seems that the camera companies are always changing their minds about where this information should be located, so as a result it is often stored in different locations for different camera models.
#6 Updated by Andreas Huggel about 6 years ago
We're wrapping up the GSoC project, just had this email exchange with Abhinav:
Generally, as far as the metadata support is concerned, I'm sure you have noticed by now, that this can be quite a never-ending story. The more you know about a format the more metadata you see that could be extracted. At this time, I'm more concerned about wrapping up what you have and polish that a bit. Don't spend more time implementing new stuff anymore if it's not really needed for now.
Yes, I have realized it in these three months. Metadata support is a definitely a never ending story. =)
And I was left with extracting camera specific ,metadata. But it seems all camera manufacturers love to toil with the files, the wrong way. They hide metadata in such obsolete and sometime in such ridiculous manner, that it become difficult even to understand it.
I just happened to go back to #813 (http://dev.exiv2.org/issues/813), and couldn't help laughing since.
I remember one line from Phil's comment on this. To quote it -
"I feel sorry for the poor soul assigned to this task. Video metadata is a real hairball."
I can only add that Phil is an amazing dude, if he decoded all those tags all by himself. If it had not been for his online documentation, I don't think I would have been able to pull off this project in three months. Exiftool is seriously an awesome product and with such an awesome set of complete and neat documentation.
I even took help from his perl source code at times when I got completely stuck ( I think it was good that I had little understanding about scripting languages).