Feature #813

Video metadata support

Added by Andreas Huggel about 5 years ago. Updated over 3 years ago.

Status:ClosedStart date:19 Feb 2012
Priority:NormalDue date:
Assignee:Abhinav Badola% Done:

100%

Category:image format
Target version:0.24

Description

Exiv2 should support video metadata.

This feature has been requested by the digiKam [1] project team and is one of the ideas [2] of the KDE project for their GSoC 2012 [3] application.

The same task was also a GSoC project in 2011. However the proposal we received wasn't ranked high enough to get one of the slots allocated to the KDE project. The proposal is still online [4], I'm not sure if it's accessible publicly, but if so, go have a look at that too.

In a nutshell, digiKam uses the Exiv2 library [5] for metadata support. Exiv2 supports reading and writing Exif, IPTC and XMP metadata in images of various formats [6]. It does not currently support any video formats. Extending Exiv2 to know about video metadata is the fundamental part of this project and will enable digiKam to deal with video metadata just like it deals with image metadata.

The programming task for the GSoC project will involve changes in both Exiv2 and digiKam. This Exiv2 feature is for the Exiv2 portion of the changes.

The Exiv2 library needs to be extended to support video metadata and/or video formats. For each video format which contains Exif, IPTC or XMP metadata, this essentially means adding an Image subclass that understands the format of the video file [7]. Check out some of the existing Exiv2 image classes (files *image.[ch]pp) for details, e.g., JpegImage [8].
For video formats which contain video metadata different from Exif, IPTC and XMP, the task includes adding support for the new metadata standard, which is significantly more challenging than just adding Image subclasses. (I'll elaborate later if required.)

Some research and planning is necessary before any coding. Discussions should take place here in this feature and can be part of an eventual GSoC proposal: What are the popular video formats, are their specifications easily available, how is metadata stored in these formats and based on that, what are good candidates for this project?

[1] http://www.digikam.org/
[2] http://community.kde.org/GSoC/2012/Ideas#Project:_Video_metadata_support
[3] http://www.google-melange.com/gsoc/homepage/google/gsoc2012
[4] http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/liviu_r2/1
[5] http://www.exiv2.org
[6] http://dev.exiv2.org/projects/exiv2/wiki/Supported_image_formats
[7] http://www.exiv2.org/doc/
[8] http://exiv2.org/doc/classExiv2_1_1JpegImage.html

Associated revisions

Revision 2753
Added by Abhinav Badola almost 5 years ago

[gsoc2012] #813: Added initial version of the riffvideo files

Revision 2754
Added by Abhinav Badola almost 5 years ago

[gsoc2012] #813: Added Some pre-existing samples of xmpData from the existing namespace XmpDM

Revision 2755
Added by Abhinav Badola almost 5 years ago

[gsoc2012] #813: Created Custom Namespace Xmp.video and made all entries in riffvideo.cpp

Revision 2757
Added by Abhinav Badola almost 5 years ago

[gsoc2012] #813: Implemented audioEncodingValues[], removed some useless elements from riffvideo.cpp

Revision 2758
Added by Abhinav Badola almost 5 years ago

[gsoc2012] #813: Added proper working Model of matroskavideo files

Revision 2759
Added by Abhinav Badola almost 5 years ago

[gsoc2012] #813: Added quicktimevideo.cpp to Makefile, Added WebM support in matroskavideo.cpp

Revision 2760
Added by Abhinav Badola almost 5 years ago

[gsoc2012] #813: Removed Segmentation fault error from matroskavideo.cpp

Revision 2761
Added by Abhinav Badola almost 5 years ago

[gsoc2012] #813: Implemented switch case in matroskavideo.cpp

Revision 2762
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Implemented basic read structure in quicktimevideo.cpp

Revision 2763
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Removed Reading bugs from quicktimevideo.cpp

Revision 2764
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Removed Reading bugs from quicktimevideo.cpp

Revision 2765
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added User Data Reading in quicktimevideo.cpp

Revision 2768
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Reduced read time access in matroskavideo.cpp

Revision 2784
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Applied patch in riffvideo.cpp and matroskavideo.cpp for xmp.video.codec

Revision 2785
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Modified matroskavideo.cpp, added new entries

Revision 2786
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Completed matroskavideo.cpp, no more modifications required for now. Only need to add Aspect Ratio.

Revision 2787
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed bug in matroskavideo.cpp,

Revision 2788
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed bug in matroskavideo.cpp

Revision 2789
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed bug in matroskavideo.cpp

Revision 2790
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed Duration bug for Matroska File Support

Revision 2792
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Restructured Riffvideo.cpp, changed the method of decoding of Tags

Revision 2797
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Applied patch in riffvideo.cpp, added odml tag decoder function

Revision 2798
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Patch riffvideo.cpp, removed Info Decoder bug

Revision 2799
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Patch riffvideo.cpp, fixed Info Decoder, prevented crash on scanning of WAVE file

Revision 2800
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added Stream Data Tag Decoder in riffvideo.cpp

Revision 2801
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added Nikon Tag Decoder function in riffvideo.cpp

Revision 2802
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added TODO in NikonTagDecoder function in riffvideo.cpp

Revision 2803
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added TODO in NikonTagDecoder function in riffvideo.cpp

Revision 2804
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added Pentax Junk Tags decoding in riffvideo.cpp

Revision 2805
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added initial versions of asfvideo files

Revision 2806
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added GUID creation function and GUID reference list in asfvideo.cpp

Revision 2807
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added Basic File Properties Decoding Function in asfvideo.cpp

Revision 2808
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Completed basic structure of asfvideo.cpp

Revision 2809
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Updated src/CMakeLists.cpp

Revision 2810
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed buffer size bug in matroskavideo.cpp

Revision 2811
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed handler description bug and trak bug in quicktimevideo.cpp

Revision 2812
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Added basic test-driver for video files.

Revision 2813
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Applied standard header file format, changed some comments.

Revision 2814
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Applied standard file format and some formatting changes.

Revision 2815
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Simplified ignoreList() and dataIgnoreList() to avoid hardcoded list sizes.

Revision 2816
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Use C++ include file.

Revision 2817
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Removed ignoreList() and dataIgnoreList() completely now, since they are only used once each and consist of only one line of code.

Revision 2818
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Addded Nikon NCTG tag decoding in quicktimevideo.cpp added some new lens in canonmn.cpp

Revision 2819
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added detailed decoding of Nikon Tags in quicktimevideo.cpp

Revision 2820
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added Most of the decoding functions in asfvideo.cpp, Worked on details of NCTG tags in quicktimevideo.cpp

Revision 2821
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Changed the calculation of duration of all the videos to milliseconds from seconds

Revision 2822
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed Audio channel type bug in asfvideo.cpp

Revision 2824
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Updated video-test output.

Revision 2825
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Some renaming.

Revision 2826
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Simplified code, changed types to avoid implicit type conversions.

Revision 2827
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Use constant instead of hardcoded value.

Revision 2828
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: More code simplifications: Use a single byte rather than a DataBuf when that is all we need.

Revision 2829
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Alligned break so that the code looks neater

Revision 2830
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Formatted asfvideo.cpp and asfvideo.hpp according to Exiv2 standards

Revision 2831
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Applied formatting to Riff video

Revision 2833
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Completed Info Tags in Riff Video

Revision 2834
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Deleted a few unneccessary comments in Riff Video

Revision 2835
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Updated video test output file

Revision 2836
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Added ASF test video with chapters.

Revision 2837
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Minor fix, should make diff output visible now, if tests fail.

Revision 2838
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Formatted quicktimevideo.[cpp,hpp] file according to Exiv2 format

Revision 2839
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Changed all properties in Xmp.[video, audio] to Intial Capital Letter

Revision 2840
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed Chinese decoding bug in asfvideo.cpp

Revision 2841
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Updated test output, added unicode test file.

Revision 2842
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Fixed creation of std::string from Exiv2::DataBuf.

Revision 2843
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Changes in AsfVideo::contentDescription
- Added temp. assertion hack to catch potential crash when the buffer is not large enough
- Tried to fix use of types (not done: parameter size should be long)
- Removed unnecessary namespace identifier
- Removed localization call for tag names; they are not localized
- Formatting

Revision 2844
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Updated test output.

Revision 2845
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Addded Aspect Ratio in all the files, though a better unified method still needs to be made

Revision 2846
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Increased the size of Data Buffer

Revision 2847
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Added some new statements to prevent Aspect Ratio problem

Revision 2848
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Fixed Content Creation Size Limit, created DataBuf inside loop, used byte array where required

Revision 2849
Added by Abhinav Badola over 4 years ago

[gsoc2012] #813: Declared byte array inside the loop

Revision 2850
Added by Andreas Huggel over 4 years ago

[gsoc2012] #813: Updated test output.

Revision 2853
Added by Andreas Huggel over 4 years ago

#813: AsfVideo::contentDescription: tweaks, error handling.

Revision 2854
Added by Andreas Huggel over 4 years ago

#813: More Matroska polishing (work-in-progress).

Revision 2855
Added by Andreas Huggel over 4 years ago

#813: More work-in-progress Matroska polishing
- Pass const byte* buf and long size instead of DataBuf& buf to functions if no ownership transfer is involved
- Use bit-wise operations for calculation of tag and tag data (still needs work for negative numbers)

Revision 2856
Added by Andreas Huggel over 4 years ago

#813: Matroska polishing, added warning.

Revision 2863
Added by Abhinav Badola over 4 years ago

#813: Addded TrackAperture Function and Nikon MOV tags decoding in quicktimevideo.cpp

Revision 2885
Added by Andreas Huggel over 4 years ago

#813: Disabled some debug messages (temporary measure, just to avoid cluttering the video test output).

Revision 2888
Added by Abhinav Badola over 4 years ago

#813: Corrected a minor size glitche in riffvideo.cpp

History

#1 Updated by PARTHASARATHY GOPAVARAPU about 5 years ago

Is there any information about the commonly used metadata information in popular video formats, so that we can group them into ones that are already existing in the exif,iptc or xmp metadata and ones that are not?

#2 Updated by Andreas Huggel about 5 years ago

I like the approach. There are some links in the proposal [4] which can hopefully get you started. Beyond that you'll need to do your own research.

#3 Updated by Andreas Huggel about 5 years ago

KDE has been accepted as a mentoring organization for GSoC 2012 so this project is real! Looking forward to your proposals!

http://teom.wordpress.com/2012/03/16/kde-accepted-for-google-summer-of-code-2012/

#4 Updated by Gilles Caulier about 5 years ago

Andreas,

Of course, project entry need to be created in Google Melange interface when your mentoring request will be accepted...

Gilles

#5 Updated by Phil Harvey almost 5 years ago

I feel sorry for the poor soul assigned to this task. Video metadata is a real hairball.

  • There is a wild proliferation of video formats -- many different containers and many different video codecs for each container. Some container formats (ie. M2TS) don't support metadata at all, so one must parse the video stream to extract anything useful from these files (see http://owl.phy.queensu.ca/~phil/exiftool/commentary.html#AVCHD ).
  • Even though some different video formats use the same container, there is no consistency between formats about how the metadata is stored in the container. The stupidity of the designers of these video formats often boggles the mind.
  • The specifications for some video formats are imprisoned by ISO and expensive to acquire, if available at all. Publicly available documentation is often very poor, or non-existent.
  • Much of the interesting metadata in videos from current digital cameras is stored in proprietary formats -- often very few of the "standard" metadata fields are populated. And worse, it seems that the camera companies are always changing their minds about where this information should be located, so as a result it is often stored in different locations for different camera models.

- Phil

#6 Updated by Andreas Huggel over 4 years ago

We're wrapping up the GSoC project, just had this email exchange with Abhinav:

Generally, as far as the metadata support is concerned, I'm sure you have noticed by now, that this can be quite a never-ending story. The more you know about a format the more metadata you see that could be extracted. At this time, I'm more concerned about wrapping up what you have and polish that a bit. Don't spend more time implementing new stuff anymore if it's not really needed for now.

Yes, I have realized it in these three months. Metadata support is a definitely a never ending story. =)
And I was left with extracting camera specific ,metadata. But it seems all camera manufacturers love to toil with the files, the wrong way. They hide metadata in such obsolete and sometime in such ridiculous manner, that it become difficult even to understand it.

I just happened to go back to #813 (http://dev.exiv2.org/issues/813), and couldn't help laughing since.
I remember one line from Phil's comment on this. To quote it -
"I feel sorry for the poor soul assigned to this task. Video metadata is a real hairball."

(Absolutely True)

I can only add that Phil is an amazing dude, if he decoded all those tags all by himself. If it had not been for his online documentation, I don't think I would have been able to pull off this project in three months. Exiftool is seriously an awesome product and with such an awesome set of complete and neat documentation.
I even took help from his perl source code at times when I got completely stuck ( I think it was good that I had little understanding about scripting languages).

#7 Updated by Andreas Huggel over 3 years ago

  • Status changed from New to Closed
  • Assignee set to Abhinav Badola
  • Target version set to 0.24
  • % Done changed from 0 to 100

Also available in: Atom PDF

Redmine Appliance - Powered by TurnKey Linux