Feature #992

Better raw file support and test

Added by Robin Mills over 2 years ago. Updated 5 months ago.

Status:AssignedStart date:18 Sep 2014
Priority:NormalDue date:
Assignee:Robin Mills% Done:

0%

Category:testing
Target version:0.27

Description

See topic: 1817

New lens identifications by using Lensfun's RAW database. http://dev.exiv2.org/boards/3/topics/1817


Related issues

Related to Exiv2 - Bug #867: Lens not detected on Olympus E-1 Assigned 10 Nov 2012
Related to Exiv2 - Support #995: Writing Exif to TIFF-like Hasselblad RAW *.3fr Closed 04 Nov 2014
Related to Exiv2 - Feature #1076: Detect raw image size in Fuji RAF New 10 May 2015
Related to Exiv2 - Bug #1081: Read XMP values from CR2 raw file when stored in XMLPacket Closed 15 May 2015
Related to Exiv2 - Feature #1176: Validate HUGE file support New 31 Mar 2016
Related to Exiv2 - Bug #770: Exif.CanonFi.FileNumber value is always (0) New 18 May 2011
Related to Exiv2 - Patch #1179: Update Fujifilm Filmmode tag Closed 17 Apr 2016
Related to Exiv2 - Bug #1182: Exiv2 is unable to update any Exif.SubImageN.xxx tag such... Closed 02 May 2016
Related to Exiv2 - Feature #668: "TIFF-safe" setExifData variant Closed 03 Jan 2010
Related to Exiv2 - Feature #1048: Add support for webm video format. New 27 Mar 2015
Related to Exiv2 - Bug #1189: Updating time changes Makernote Closed 29 May 2016
Related to Exiv2 - Patch #1197: Add support for Sigma 150-500 on Canon with 1.4X TC Closed 19 Jul 2016 09 Aug 2016
Related to Exiv2 - Feature #1245: Better I/O implementation when EXV_HAVE_MMAP is not set Assigned 17 Oct 2016

History

#1 Updated by Robin Mills over 2 years ago

  • Tracker changed from Bug to Feature

#3 Updated by Robin Mills almost 2 years ago

I've linked this report to #995 Writing Exif to TIFF-like Hasselblad RAW *.3fr

I've also discovered this site which has sample images for many raw formats: http://www.rawsamples.ch/index.php/en/

Thanks to Tuan's webready code, we will be able to read the metadata from those files without copying them locally.

1268 rmills@rmillsmbp:~/Downloads $ exiv2 -pS RAW_NIKON_D5300.NEF 
STRUCTURE OF TIFF FILE: RAW_NIKON_D5300.NEF
 address |    tag                      |      type |    count |   offset | value
      10 | 0x00fe NewSubfileType       |      LONG |        1 |        1 | 1
      22 | 0x0100 ImageWidth           |      LONG |        1 |      160 | 160
      34 | 0x0101 ImageLength          |      LONG |        1 |      120 | 120
      46 | 0x0102 BitsPerSample        |     SHORT |        3 |      328 | 
      58 | 0x0103 Compression          |     SHORT |        1 |        1 | 1
      70 | 0x0106 PhotometricInterpret |     SHORT |        1 |        2 | 2
      82 | 0x010f Make                 |     ASCII |       18 |      336 | NIKON CORPORATION.
      94 | 0x0110 Model                |     ASCII |       12 |      356 | NIKON D5300.
     106 | 0x0111 StripOffsets         |      LONG |        1 |   168448 | 37376
     118 | 0x0112 Orientation          |     SHORT |        1 |        1 | 1
     130 | 0x0115 SamplesPerPixel      |     SHORT |        1 |        3 | 3
     142 | 0x0116 RowsPerStrip         |      LONG |        1 |      120 | 120
     154 | 0x0117 StripByteCounts      |      LONG |        1 |    57600 | 57600
     166 | 0x011a XResolution          |  RATIONAL |        1 |      368 | 
     178 | 0x011b YResolution          |  RATIONAL |        1 |      376 | 
     190 | 0x011c PlanarConfiguration  |     SHORT |        1 |        1 | 1
     202 | 0x0128 ResolutionUnit       |     SHORT |        1 |        2 | 2
     214 | 0x0131 Software             |     ASCII |       10 |      384 | Ver.1.00 .
     226 | 0x0132 DateTime             |     ASCII |       20 |      396 | 2014:04:02 11:39:52.
     238 | 0x014a SubIFDs              |      LONG |        3 |      416 | 
     250 | 0x0214 ReferenceBlackWhite  |  RATIONAL |        6 |      428 | 
     262 | 0x02bc XMLPacket            |      BYTE |     2048 |      476 | <?xpacket begin="..." id="W5M0Mp ...
     274 | 0x8769 ExifTag              |      LONG |        1 |     2544 | 2544
     286 | 0x8825 GPSTag               |      LONG |        1 |   167936 | 36864
     298 | 0x9003 DateTimeOriginal     |     ASCII |       20 |     2524 | 2014:04:02 11:39:52.
     310 | 0x9216 TIFFEPStandardID     |      BYTE |        4 |        1 | I*..
1269 rmills@rmillsmbp:~/Downloads $ exiv2 -pS http://www.rawsamples.ch/raws/nikon/RAW_NIKON_D5300.NEF
STRUCTURE OF TIFF FILE: http://www.rawsamples.ch/raws/nikon/RAW_NIKON_D5300.NEF
 address |    tag                      |      type |    count |   offset | value
      10 | 0x00fe NewSubfileType       |      LONG |        1 |        1 | 1
      22 | 0x0100 ImageWidth           |      LONG |        1 |      160 | 160
      34 | 0x0101 ImageLength          |      LONG |        1 |      120 | 120
      46 | 0x0102 BitsPerSample        |     SHORT |        3 |      328 | 
      58 | 0x0103 Compression          |     SHORT |        1 |        1 | 1
      70 | 0x0106 PhotometricInterpret |     SHORT |        1 |        2 | 2
      82 | 0x010f Make                 |     ASCII |       18 |      336 | NIKON CORPORATION.
      94 | 0x0110 Model                |     ASCII |       12 |      356 | NIKON D5300.
     106 | 0x0111 StripOffsets         |      LONG |        1 |   168448 | 37376
     118 | 0x0112 Orientation          |     SHORT |        1 |        1 | 1
     130 | 0x0115 SamplesPerPixel      |     SHORT |        1 |        3 | 3
     142 | 0x0116 RowsPerStrip         |      LONG |        1 |      120 | 120
     154 | 0x0117 StripByteCounts      |      LONG |        1 |    57600 | 57600
     166 | 0x011a XResolution          |  RATIONAL |        1 |      368 | 
     178 | 0x011b YResolution          |  RATIONAL |        1 |      376 | 
     190 | 0x011c PlanarConfiguration  |     SHORT |        1 |        1 | 1
     202 | 0x0128 ResolutionUnit       |     SHORT |        1 |        2 | 2
     214 | 0x0131 Software             |     ASCII |       10 |      384 | Ver.1.00 .
     226 | 0x0132 DateTime             |     ASCII |       20 |      396 | 2014:04:02 11:39:52.
     238 | 0x014a SubIFDs              |      LONG |        3 |      416 | 
     250 | 0x0214 ReferenceBlackWhite  |  RATIONAL |        6 |      428 | 
     262 | 0x02bc XMLPacket            |      BYTE |     2048 |      476 | <?xpacket begin="..." id="W5M0Mp ...
     274 | 0x8769 ExifTag              |      LONG |        1 |     2544 | 2544
     286 | 0x8825 GPSTag               |      LONG |        1 |   167936 | 36864
     298 | 0x9003 DateTimeOriginal     |     ASCII |       20 |     2524 | 2014:04:02 11:39:52.
     310 | 0x9216 TIFFEPStandardID     |      BYTE |        4 |        1 | I*..
1270 rmills@rmillsmbp:~/Downloads $ 

#4 Updated by Robin Mills almost 2 years ago

  • Assignee deleted (Robin Mills)

#5 Updated by Robin Mills 11 months ago

I think this should be considered for GSoC 2017 for the following reasons:

1 The project is essentially "time-bounded". We can make a list of file formats to be considered. They are implemented and tested one-at-a-time. Everything achieved is useful. What can't be done in the time, doesn't defeat the project.
2 The level of supervision/mentoring is quite low.
3 It has no library, build or platform dependences
4 Development can proceed on Linux. The buildserver provides multi-platform build support.

There are a couple of issues relating to SubIFDs #1182 and #668 and these should be carefully considered in this project. Perhaps we need options in exiv2 which we can switch on/off at will. One could be "allow editing of SubImage IFDs".

We should consider adding webp and webm support #1048

The video code should be reviewed and brought into a better state. #1068. However that's out of scope for #992.

#6 Updated by Robin Mills 9 months ago

  • Assignee set to Robin Mills
  • Target version changed from 1.0 to 0.27

#7 Updated by Robin Mills 5 months ago

I have downloaded the 5Gbyte zip of the raw files available from: https://www.rawsamples.ch/index.php/en/all-files-as-7z-archive They are now available on the buildserver at testfiles/raw and on the web at: http://exiv2.dyndns.org:8080/userContent/testfiles/raw/

I am very happy to report that exiv2 can successfully read every file. A total of 322 files.

~/Jenkins/testfiles/raw $ for i in $(find . -type f); do exiv2 -pa --verbose --grep DateTimeDigitized $i 2>/dev/null ; done  | grep File | wc
    322     966   13689
~/Jenkins/testfiles/raw $ (for i in $(find . -type f); do echo $i; done) | wc
    322     322   10469
~/Jenkins/testfiles/raw $ 

Also available in: Atom PDF

Redmine Appliance - Powered by TurnKey Linux