Project

General

Profile

Feature #813

Video metadata support

Added by Andreas Huggel about 7 years ago. Updated almost 6 years ago.

Status:
Closed
Priority:
Normal
Category:
image format
Target version:
Start date:
19 Feb 2012
Due date:
% Done:

100%

Estimated time:

Description

Exiv2 should support video metadata.

This feature has been requested by the digiKam [1] project team and is one of the ideas [2] of the KDE project for their GSoC 2012 [3] application.

The same task was also a GSoC project in 2011. However the proposal we received wasn't ranked high enough to get one of the slots allocated to the KDE project. The proposal is still online [4], I'm not sure if it's accessible publicly, but if so, go have a look at that too.

In a nutshell, digiKam uses the Exiv2 library [5] for metadata support. Exiv2 supports reading and writing Exif, IPTC and XMP metadata in images of various formats [6]. It does not currently support any video formats. Extending Exiv2 to know about video metadata is the fundamental part of this project and will enable digiKam to deal with video metadata just like it deals with image metadata.

The programming task for the GSoC project will involve changes in both Exiv2 and digiKam. This Exiv2 feature is for the Exiv2 portion of the changes.

The Exiv2 library needs to be extended to support video metadata and/or video formats. For each video format which contains Exif, IPTC or XMP metadata, this essentially means adding an Image subclass that understands the format of the video file [7]. Check out some of the existing Exiv2 image classes (files *image.[ch]pp) for details, e.g., JpegImage [8].
For video formats which contain video metadata different from Exif, IPTC and XMP, the task includes adding support for the new metadata standard, which is significantly more challenging than just adding Image subclasses. (I'll elaborate later if required.)

Some research and planning is necessary before any coding. Discussions should take place here in this feature and can be part of an eventual GSoC proposal: What are the popular video formats, are their specifications easily available, how is metadata stored in these formats and based on that, what are good candidates for this project?

[1] http://www.digikam.org/
[2] http://community.kde.org/GSoC/2012/Ideas#Project:_Video_metadata_support
[3] http://www.google-melange.com/gsoc/homepage/google/gsoc2012
[4] http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/liviu_r2/1
[5] http://www.exiv2.org
[6] http://dev.exiv2.org/projects/exiv2/wiki/Supported_image_formats
[7] http://www.exiv2.org/doc/
[8] http://exiv2.org/doc/classExiv2_1_1JpegImage.html

Associated revisions

Revision 2753 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added initial version of the riffvideo files

Revision 2754 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added Some pre-existing samples of xmpData from the existing namespace XmpDM

Revision 2755 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Created Custom Namespace Xmp.video and made all entries in riffvideo.cpp

Revision 2757 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Implemented audioEncodingValues[], removed some useless elements from riffvideo.cpp

Revision 2758 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added proper working Model of matroskavideo files

Revision 2759 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added quicktimevideo.cpp to Makefile, Added WebM support in matroskavideo.cpp

Revision 2760 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Removed Segmentation fault error from matroskavideo.cpp

Revision 2761 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Implemented switch case in matroskavideo.cpp

Revision 2762 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Implemented basic read structure in quicktimevideo.cpp

Revision 2763 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Removed Reading bugs from quicktimevideo.cpp

Revision 2764 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Removed Reading bugs from quicktimevideo.cpp

Revision 2765 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added User Data Reading in quicktimevideo.cpp

Revision 2768 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Reduced read time access in matroskavideo.cpp

Revision 2784 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Applied patch in riffvideo.cpp and matroskavideo.cpp for xmp.video.codec

Revision 2785 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Modified matroskavideo.cpp, added new entries

Revision 2786 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Completed matroskavideo.cpp, no more modifications required for now. Only need to add Aspect Ratio.

Revision 2787 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed bug in matroskavideo.cpp,

Revision 2788 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed bug in matroskavideo.cpp

Revision 2789 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed bug in matroskavideo.cpp

Revision 2790 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed Duration bug for Matroska File Support

Revision 2792 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Restructured Riffvideo.cpp, changed the method of decoding of Tags

Revision 2797 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Applied patch in riffvideo.cpp, added odml tag decoder function

Revision 2798 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Patch riffvideo.cpp, removed Info Decoder bug

Revision 2799 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Patch riffvideo.cpp, fixed Info Decoder, prevented crash on scanning of WAVE file

Revision 2800 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added Stream Data Tag Decoder in riffvideo.cpp

Revision 2801 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added Nikon Tag Decoder function in riffvideo.cpp

Revision 2802 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added TODO in NikonTagDecoder function in riffvideo.cpp

Revision 2803 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added TODO in NikonTagDecoder function in riffvideo.cpp

Revision 2804 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added Pentax Junk Tags decoding in riffvideo.cpp

Revision 2805 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added initial versions of asfvideo files

Revision 2806 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added GUID creation function and GUID reference list in asfvideo.cpp

Revision 2807 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added Basic File Properties Decoding Function in asfvideo.cpp

Revision 2808 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Completed basic structure of asfvideo.cpp

Revision 2809 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Updated src/CMakeLists.cpp

Revision 2810 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed buffer size bug in matroskavideo.cpp

Revision 2811 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed handler description bug and trak bug in quicktimevideo.cpp

Revision 2812 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Added basic test-driver for video files.

Revision 2813 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Applied standard header file format, changed some comments.

Revision 2814 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Applied standard file format and some formatting changes.

Revision 2815 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Simplified ignoreList() and dataIgnoreList() to avoid hardcoded list sizes.

Revision 2816 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Use C++ include file.

Revision 2817 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Removed ignoreList() and dataIgnoreList() completely now, since they are only used once each and consist of only one line of code.

Revision 2818 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Addded Nikon NCTG tag decoding in quicktimevideo.cpp added some new lens in canonmn.cpp

Revision 2819 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added detailed decoding of Nikon Tags in quicktimevideo.cpp

Revision 2820 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added Most of the decoding functions in asfvideo.cpp, Worked on details of NCTG tags in quicktimevideo.cpp

Revision 2821 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Changed the calculation of duration of all the videos to milliseconds from seconds

Revision 2822 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed Audio channel type bug in asfvideo.cpp

Revision 2824 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Updated video-test output.

Revision 2825 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Some renaming.

Revision 2826 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Simplified code, changed types to avoid implicit type conversions.

Revision 2827 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Use constant instead of hardcoded value.

Revision 2828 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: More code simplifications: Use a single byte rather than a DataBuf when that is all we need.

Revision 2829 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Alligned break so that the code looks neater

Revision 2830 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Formatted asfvideo.cpp and asfvideo.hpp according to Exiv2 standards

Revision 2831 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Applied formatting to Riff video

Revision 2833 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Completed Info Tags in Riff Video

Revision 2834 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Deleted a few unneccessary comments in Riff Video

Revision 2835 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Updated video test output file

Revision 2836 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Added ASF test video with chapters.

Revision 2837 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Minor fix, should make diff output visible now, if tests fail.

Revision 2838 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Formatted quicktimevideo.[cpp,hpp] file according to Exiv2 format

Revision 2839 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Changed all properties in Xmp.[video, audio] to Intial Capital Letter

Revision 2840 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed Chinese decoding bug in asfvideo.cpp

Revision 2841 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Updated test output, added unicode test file.

Revision 2842 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Fixed creation of std::string from Exiv2::DataBuf.

Revision 2843 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Changes in AsfVideo::contentDescription
- Added temp. assertion hack to catch potential crash when the buffer is not large enough
- Tried to fix use of types (not done: parameter size should be long)
- Removed unnecessary namespace identifier
- Removed localization call for tag names; they are not localized
- Formatting

Revision 2844 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Updated test output.

Revision 2845 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Addded Aspect Ratio in all the files, though a better unified method still needs to be made

Revision 2846 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Increased the size of Data Buffer

Revision 2847 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Added some new statements to prevent Aspect Ratio problem

Revision 2848 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Fixed Content Creation Size Limit, created DataBuf inside loop, used byte array where required

Revision 2849 (diff)
Added by Abhinav Badola almost 7 years ago

[gsoc2012] #813: Declared byte array inside the loop

Revision 2850 (diff)
Added by Andreas Huggel almost 7 years ago

[gsoc2012] #813: Updated test output.

Revision 2853 (diff)
Added by Andreas Huggel almost 7 years ago

#813: AsfVideo::contentDescription: tweaks, error handling.

Revision 2854 (diff)
Added by Andreas Huggel over 6 years ago

#813: More Matroska polishing (work-in-progress).

Revision 2855 (diff)
Added by Andreas Huggel over 6 years ago

#813: More work-in-progress Matroska polishing
- Pass const byte* buf and long size instead of DataBuf& buf to functions if no ownership transfer is involved
- Use bit-wise operations for calculation of tag and tag data (still needs work for negative numbers)

Revision 2856 (diff)
Added by Andreas Huggel over 6 years ago

#813: Matroska polishing, added warning.

Revision 2863 (diff)
Added by Abhinav Badola over 6 years ago

#813: Addded TrackAperture Function and Nikon MOV tags decoding in quicktimevideo.cpp

Revision 2885 (diff)
Added by Andreas Huggel over 6 years ago

#813: Disabled some debug messages (temporary measure, just to avoid cluttering the video test output).

Revision 2888 (diff)
Added by Abhinav Badola over 6 years ago

#813: Corrected a minor size glitche in riffvideo.cpp

History

#1

Updated by PARTHASARATHY GOPAVARAPU about 7 years ago

Is there any information about the commonly used metadata information in popular video formats, so that we can group them into ones that are already existing in the exif,iptc or xmp metadata and ones that are not?

#2

Updated by Andreas Huggel about 7 years ago

I like the approach. There are some links in the proposal [4] which can hopefully get you started. Beyond that you'll need to do your own research.

#3

Updated by Andreas Huggel about 7 years ago

KDE has been accepted as a mentoring organization for GSoC 2012 so this project is real! Looking forward to your proposals!

http://teom.wordpress.com/2012/03/16/kde-accepted-for-google-summer-of-code-2012/

#4

Updated by Gilles Caulier about 7 years ago

Andreas,

Of course, project entry need to be created in Google Melange interface when your mentoring request will be accepted...

Gilles

#5

Updated by Phil Harvey about 7 years ago

I feel sorry for the poor soul assigned to this task. Video metadata is a real hairball.

  • There is a wild proliferation of video formats -- many different containers and many different video codecs for each container. Some container formats (ie. M2TS) don't support metadata at all, so one must parse the video stream to extract anything useful from these files (see http://owl.phy.queensu.ca/~phil/exiftool/commentary.html#AVCHD ).
  • Even though some different video formats use the same container, there is no consistency between formats about how the metadata is stored in the container. The stupidity of the designers of these video formats often boggles the mind.
  • The specifications for some video formats are imprisoned by ISO and expensive to acquire, if available at all. Publicly available documentation is often very poor, or non-existent.
  • Much of the interesting metadata in videos from current digital cameras is stored in proprietary formats -- often very few of the "standard" metadata fields are populated. And worse, it seems that the camera companies are always changing their minds about where this information should be located, so as a result it is often stored in different locations for different camera models.

- Phil

#6

Updated by Andreas Huggel almost 7 years ago

We're wrapping up the GSoC project, just had this email exchange with Abhinav:

Generally, as far as the metadata support is concerned, I'm sure you have noticed by now, that this can be quite a never-ending story. The more you know about a format the more metadata you see that could be extracted. At this time, I'm more concerned about wrapping up what you have and polish that a bit. Don't spend more time implementing new stuff anymore if it's not really needed for now.

Yes, I have realized it in these three months. Metadata support is a definitely a never ending story. =)
And I was left with extracting camera specific ,metadata. But it seems all camera manufacturers love to toil with the files, the wrong way. They hide metadata in such obsolete and sometime in such ridiculous manner, that it become difficult even to understand it.

I just happened to go back to #813 (http://dev.exiv2.org/issues/813), and couldn't help laughing since.
I remember one line from Phil's comment on this. To quote it -
"I feel sorry for the poor soul assigned to this task. Video metadata is a real hairball."

(Absolutely True)

I can only add that Phil is an amazing dude, if he decoded all those tags all by himself. If it had not been for his online documentation, I don't think I would have been able to pull off this project in three months. Exiftool is seriously an awesome product and with such an awesome set of complete and neat documentation.
I even took help from his perl source code at times when I got completely stuck ( I think it was good that I had little understanding about scripting languages).

#7

Updated by Andreas Huggel almost 6 years ago

  • Status changed from New to Closed
  • Assignee set to Abhinav Badola
  • Target version set to 0.24
  • % Done changed from 0 to 100

Also available in: Atom PDF