Feature #813
Video metadata support
100%
Description
Exiv2 should support video metadata.
This feature has been requested by the digiKam [1] project team and is one of the ideas [2] of the KDE project for their GSoC 2012 [3] application.
The same task was also a GSoC project in 2011. However the proposal we received wasn't ranked high enough to get one of the slots allocated to the KDE project. The proposal is still online [4], I'm not sure if it's accessible publicly, but if so, go have a look at that too.
In a nutshell, digiKam uses the Exiv2 library [5] for metadata support. Exiv2 supports reading and writing Exif, IPTC and XMP metadata in images of various formats [6]. It does not currently support any video formats. Extending Exiv2 to know about video metadata is the fundamental part of this project and will enable digiKam to deal with video metadata just like it deals with image metadata.
The programming task for the GSoC project will involve changes in both Exiv2 and digiKam. This Exiv2 feature is for the Exiv2 portion of the changes.
The Exiv2 library needs to be extended to support video metadata and/or video formats. For each video format which contains Exif, IPTC or XMP metadata, this essentially means adding an Image subclass that understands the format of the video file [7]. Check out some of the existing Exiv2 image classes (files *image.[ch]pp) for details, e.g., JpegImage [8].
For video formats which contain video metadata different from Exif, IPTC and XMP, the task includes adding support for the new metadata standard, which is significantly more challenging than just adding Image subclasses. (I'll elaborate later if required.)
Some research and planning is necessary before any coding. Discussions should take place here in this feature and can be part of an eventual GSoC proposal: What are the popular video formats, are their specifications easily available, how is metadata stored in these formats and based on that, what are good candidates for this project?
[1] http://www.digikam.org/
[2] http://community.kde.org/GSoC/2012/Ideas#Project:_Video_metadata_support
[3] http://www.google-melange.com/gsoc/homepage/google/gsoc2012
[4] http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/liviu_r2/1
[5] http://www.exiv2.org
[6] http://dev.exiv2.org/projects/exiv2/wiki/Supported_image_formats
[7] http://www.exiv2.org/doc/
[8] http://exiv2.org/doc/classExiv2_1_1JpegImage.html
Associated revisions
[gsoc2012] #813: Added Some pre-existing samples of xmpData from the existing namespace XmpDM
[gsoc2012] #813: Created Custom Namespace Xmp.video and made all entries in riffvideo.cpp
[gsoc2012] #813: Implemented audioEncodingValues[], removed some useless elements from riffvideo.cpp
[gsoc2012] #813: Added proper working Model of matroskavideo files
[gsoc2012] #813: Added quicktimevideo.cpp to Makefile, Added WebM support in matroskavideo.cpp
[gsoc2012] #813: Removed Segmentation fault error from matroskavideo.cpp
[gsoc2012] #813: Implemented switch case in matroskavideo.cpp
[gsoc2012] #813: Implemented basic read structure in quicktimevideo.cpp
[gsoc2012] #813: Removed Reading bugs from quicktimevideo.cpp
[gsoc2012] #813: Removed Reading bugs from quicktimevideo.cpp
[gsoc2012] #813: Added User Data Reading in quicktimevideo.cpp
[gsoc2012] #813: Reduced read time access in matroskavideo.cpp
[gsoc2012] #813: Applied patch in riffvideo.cpp and matroskavideo.cpp for xmp.video.codec
[gsoc2012] #813: Modified matroskavideo.cpp, added new entries
[gsoc2012] #813: Completed matroskavideo.cpp, no more modifications required for now. Only need to add Aspect Ratio.
[gsoc2012] #813: Fixed bug in matroskavideo.cpp,
[gsoc2012] #813: Fixed bug in matroskavideo.cpp
[gsoc2012] #813: Fixed bug in matroskavideo.cpp
[gsoc2012] #813: Fixed Duration bug for Matroska File Support
[gsoc2012] #813: Restructured Riffvideo.cpp, changed the method of decoding of Tags
[gsoc2012] #813: Applied patch in riffvideo.cpp, added odml tag decoder function
[gsoc2012] #813: Patch riffvideo.cpp, removed Info Decoder bug
[gsoc2012] #813: Patch riffvideo.cpp, fixed Info Decoder, prevented crash on scanning of WAVE file
[gsoc2012] #813: Added Stream Data Tag Decoder in riffvideo.cpp
[gsoc2012] #813: Added Nikon Tag Decoder function in riffvideo.cpp
[gsoc2012] #813: Added TODO in NikonTagDecoder function in riffvideo.cpp
[gsoc2012] #813: Added TODO in NikonTagDecoder function in riffvideo.cpp
[gsoc2012] #813: Added Pentax Junk Tags decoding in riffvideo.cpp
[gsoc2012] #813: Added initial versions of asfvideo files
[gsoc2012] #813: Added GUID creation function and GUID reference list in asfvideo.cpp
[gsoc2012] #813: Added Basic File Properties Decoding Function in asfvideo.cpp
[gsoc2012] #813: Completed basic structure of asfvideo.cpp
[gsoc2012] #813: Updated src/CMakeLists.cpp
[gsoc2012] #813: Fixed buffer size bug in matroskavideo.cpp
[gsoc2012] #813: Fixed handler description bug and trak bug in quicktimevideo.cpp
[gsoc2012] #813: Added basic test-driver for video files.
[gsoc2012] #813: Applied standard header file format, changed some comments.
[gsoc2012] #813: Applied standard file format and some formatting changes.
[gsoc2012] #813: Simplified ignoreList() and dataIgnoreList() to avoid hardcoded list sizes.
[gsoc2012] #813: Use C++ include file.
[gsoc2012] #813: Removed ignoreList() and dataIgnoreList() completely now, since they are only used once each and consist of only one line of code.
[gsoc2012] #813: Addded Nikon NCTG tag decoding in quicktimevideo.cpp added some new lens in canonmn.cpp
[gsoc2012] #813: Added detailed decoding of Nikon Tags in quicktimevideo.cpp
[gsoc2012] #813: Added Most of the decoding functions in asfvideo.cpp, Worked on details of NCTG tags in quicktimevideo.cpp
[gsoc2012] #813: Changed the calculation of duration of all the videos to milliseconds from seconds
[gsoc2012] #813: Fixed Audio channel type bug in asfvideo.cpp
[gsoc2012] #813: Updated video-test output.
[gsoc2012] #813: Some renaming.
[gsoc2012] #813: Simplified code, changed types to avoid implicit type conversions.
[gsoc2012] #813: Use constant instead of hardcoded value.
[gsoc2012] #813: More code simplifications: Use a single byte rather than a DataBuf when that is all we need.
[gsoc2012] #813: Alligned break so that the code looks neater
[gsoc2012] #813: Formatted asfvideo.cpp and asfvideo.hpp according to Exiv2 standards
[gsoc2012] #813: Applied formatting to Riff video
[gsoc2012] #813: Completed Info Tags in Riff Video
[gsoc2012] #813: Deleted a few unneccessary comments in Riff Video
[gsoc2012] #813: Updated video test output file
[gsoc2012] #813: Added ASF test video with chapters.
[gsoc2012] #813: Minor fix, should make diff output visible now, if tests fail.
[gsoc2012] #813: Formatted quicktimevideo.[cpp,hpp] file according to Exiv2 format
[gsoc2012] #813: Changed all properties in Xmp.[video, audio] to Intial Capital Letter
[gsoc2012] #813: Fixed Chinese decoding bug in asfvideo.cpp
[gsoc2012] #813: Updated test output, added unicode test file.
[gsoc2012] #813: Fixed creation of std::string from Exiv2::DataBuf.
[gsoc2012] #813: Changes in AsfVideo::contentDescription
- Added temp. assertion hack to catch potential crash when the buffer is not large enough
- Tried to fix use of types (not done: parameter size should be long)
- Removed unnecessary namespace identifier
- Removed localization call for tag names; they are not localized
- Formatting
[gsoc2012] #813: Updated test output.
[gsoc2012] #813: Addded Aspect Ratio in all the files, though a better unified method still needs to be made
[gsoc2012] #813: Increased the size of Data Buffer
[gsoc2012] #813: Added some new statements to prevent Aspect Ratio problem
[gsoc2012] #813: Fixed Content Creation Size Limit, created DataBuf inside loop, used byte array where required
[gsoc2012] #813: Declared byte array inside the loop
[gsoc2012] #813: Updated test output.
#813: AsfVideo::contentDescription: tweaks, error handling.
#813: More Matroska polishing (work-in-progress).
#813: More work-in-progress Matroska polishing
- Pass const byte* buf and long size instead of DataBuf& buf to functions if no ownership transfer is involved
- Use bit-wise operations for calculation of tag and tag data (still needs work for negative numbers)
#813: Matroska polishing, added warning.
#813: Addded TrackAperture Function and Nikon MOV tags decoding in quicktimevideo.cpp
#813: Disabled some debug messages (temporary measure, just to avoid cluttering the video test output).
#813: Corrected a minor size glitche in riffvideo.cpp
History
Updated by PARTHASARATHY GOPAVARAPU almost 10 years ago
Is there any information about the commonly used metadata information in popular video formats, so that we can group them into ones that are already existing in the exif,iptc or xmp metadata and ones that are not?
Updated by Andreas Huggel almost 10 years ago
I like the approach. There are some links in the proposal [4] which can hopefully get you started. Beyond that you'll need to do your own research.
Updated by Andreas Huggel over 9 years ago
KDE has been accepted as a mentoring organization for GSoC 2012 so this project is real! Looking forward to your proposals!
http://teom.wordpress.com/2012/03/16/kde-accepted-for-google-summer-of-code-2012/
Updated by Gilles Caulier over 9 years ago
Andreas,
Of course, project entry need to be created in Google Melange interface when your mentoring request will be accepted...
Gilles
Updated by Phil Harvey over 9 years ago
I feel sorry for the poor soul assigned to this task. Video metadata is a real hairball.
- There is a wild proliferation of video formats -- many different containers and many different video codecs for each container. Some container formats (ie. M2TS) don't support metadata at all, so one must parse the video stream to extract anything useful from these files (see http://owl.phy.queensu.ca/~phil/exiftool/commentary.html#AVCHD ).
- Even though some different video formats use the same container, there is no consistency between formats about how the metadata is stored in the container. The stupidity of the designers of these video formats often boggles the mind.
- The specifications for some video formats are imprisoned by ISO and expensive to acquire, if available at all. Publicly available documentation is often very poor, or non-existent.
- Much of the interesting metadata in videos from current digital cameras is stored in proprietary formats -- often very few of the "standard" metadata fields are populated. And worse, it seems that the camera companies are always changing their minds about where this information should be located, so as a result it is often stored in different locations for different camera models.
- Phil
Updated by Andreas Huggel over 9 years ago
We're wrapping up the GSoC project, just had this email exchange with Abhinav:
Generally, as far as the metadata support is concerned, I'm sure you have noticed by now, that this can be quite a never-ending story. The more you know about a format the more metadata you see that could be extracted. At this time, I'm more concerned about wrapping up what you have and polish that a bit. Don't spend more time implementing new stuff anymore if it's not really needed for now.
Yes, I have realized it in these three months. Metadata support is a definitely a never ending story. =)
And I was left with extracting camera specific ,metadata. But it seems all camera manufacturers love to toil with the files, the wrong way. They hide metadata in such obsolete and sometime in such ridiculous manner, that it become difficult even to understand it.
I just happened to go back to #813 (http://dev.exiv2.org/issues/813), and couldn't help laughing since.
I remember one line from Phil's comment on this. To quote it -
"I feel sorry for the poor soul assigned to this task. Video metadata is a real hairball."
(Absolutely True)
I can only add that Phil is an amazing dude, if he decoded all those tags all by himself. If it had not been for his online documentation, I don't think I would have been able to pull off this project in three months. Exiftool is seriously an awesome product and with such an awesome set of complete and neat documentation.
I even took help from his perl source code at times when I got completely stuck ( I think it was good that I had little understanding about scripting languages).
Updated by Andreas Huggel over 8 years ago
- Status changed from New to Closed
- Assignee set to Abhinav Badola
- Target version set to 0.24
- % Done changed from 0 to 100
[gsoc2012] #813: Added initial version of the riffvideo files