Project

General

Profile

Why? Trying to write jpeg in memory and save to file does NOT work.

Added by Rich Cook about 7 years ago

I'm trying to do the following:
1. Create a memory buffer with valid jpg image.
2. Add Exif data to the jpg image in memory
3. Write the jpg to file on disk.

Instead I find I have to do this:
1. Create a memory buffer with valid jpg image.
2. Write the jpg to file on disk
3. Open the jpg file using Exiv2 library factory call
4. Add Exif data to the Exiv2 object
5. Save jpg to disk again

In other words, for the following code, if I set inMemory = false, it works fine, but if I set inMemory = true, I get a file with no exif data. Can someone explain why? I prefer to do this all in memory as it seems more elegant, although hey, it works the other way.

Thanks!

bool inMemory = false; 
if (inMemory) {
// create an exiv2 image in memory using the jpeg buffer
Exiv2::Image::AutoPtr image = Exiv2::ImageFactory::open(buffer, buffersize);
image->setExifData(exifData);
image->writeMetadata();
buffersize = image->io().size();
cout << str(boost::format("image class has io size of 1\n")%buffersize);
// ---------------------------------------------------------
// Write the buffer, now containing Exif metadata, to disk
FILE * file = fopen(imgname.c_str(), "w");
fwrite(buffer, buffersize, 1, file);
fclose(file);
} else {
// "out of core"
FILE * file = fopen(imgname.c_str(), "w");
fwrite(buffer, buffersize, 1, file);
fclose(file);
Exiv2::Image::AutoPtr image = Exiv2::ImageFactory::open(imgname);
image->setExifData(exifData);
image->writeMetadata();
}

Replies (8)

RE: Why? Trying to write jpeg in memory and save to file does NOT work. - Added by Robin Mills about 7 years ago

Rich

Writing out the file and reading again is the "safe/conservative" approach.

Exiv2 uses memory mapping when available. I suspect Exiv2::ImageFactory::open(buffer,buffersize) has failed to execute something that is called by Exiv2::ImageFactory::open(imgname). It may be as simple as image->get(). As you have the code to hand, may I ask you could step the library in the debugger and determine the difference.

Robin

RE: Why? Trying to write jpeg in memory and save to file does NOT work. - Added by Robin Mills about 7 years ago

Rich

I've done some more work on this. I think it's not possible to read an image from memory and write it to disk! Here's my code (modified a little from yours)

// ***************************************************************** -*- C++ -*-
// exifprint.cpp, $Rev: 3090 $
// Sample program to print the Exif metadata of an image

#include <exiv2/exiv2.hpp>

#include <iostream>
#include <fstream>
#include <iomanip>
#include <cassert>

#include <stdio.h>

int main(int argc, char* const argv[])
try {

    if (argc != 4) {
        std::cout << "Usage: " << argv[0] << " from_path to_path mm_path\n";
        return 1;
    }

    // metaData container to be written to image at path
    const std::string from_path(argv[1]);
    const std::string to_path  (argv[2]);
    const std::string mm_path  (argv[3]);
    Exiv2::ExifData exifData  ;

    // On what file are we working?
    std::ifstream  file(from_path, std::ios::binary | std::ios::ate);
    if ( ! file ) {
        std::cout << "from_path: " << from_path << " does not exist!\n" ;
        return 2;
    }

    // What's the length?
    long buffersize = file.tellg();
    file.close();

    // copy the file into a buffer
    void*   buffer = ::malloc(buffersize);
    FILE*   f      = ::fopen(from_path.c_str(),"rb");
    ::fread(buffer,buffersize,1,f);
    :: fclose(f);

    {
        // Get the meta data and copy it to exifData
        Exiv2::Image::AutoPtr image = Exiv2::ImageFactory::open(from_path);
        assert(image.get() != 0);
        image->readMetadata();
        image->readMetadata();

        Exiv2::ExifData &data = image->exifData();
        for ( Exiv2::ExifData::const_iterator i = data.begin(); i != data.end() ; ++i ) {
            // std::cout << i->key() << " -> " << i->value().toString() << std::endl;
            exifData[i->key()] = i->value();
        }
        exifData["Exif.Image.Software"]="Robin's Wee Test Program" ;
    }

    bool inMemory = false ;
    if ( ! inMemory ) {
        // "out of core"  
        FILE * f = ::fopen(to_path.c_str(), "w"); 
        ::fwrite(buffer, buffersize, 1, f); 
        ::fclose(f); 

        Exiv2::Image::AutoPtr image = Exiv2::ImageFactory::open(to_path);
        image->setExifData(exifData); 
        image->writeMetadata(); 
    }

    inMemory = true ;
    if ( inMemory ) {
        // create an exiv2 image in memory using the jpeg buffer
        Exiv2::Image::AutoPtr i = Exiv2::ImageFactory::open((const Exiv2::byte*)buffer, buffersize);
        i->setExifData(exifData); 
        i->writeMetadata(); 
        buffersize = i->io().size(); 
        std::cout << "image class has io size of " << buffersize << "\n";
        // ---------------------------------------------------------
        // Write the buffer, now containing Exif metadata, to disk
        FILE * file = ::fopen(mm_path.c_str(), "w"); 
        ::fwrite(buffer, buffersize, 1, file); 
        ::fclose(file); 
    }
}

//catch (std::exception& e) {
//catch (Exiv2::AnyError& e) {
catch (Exiv2::Error& e) {
    std::cout << "Caught Exiv2 exception '" << e.what() << "'\n";
    return -1;
}

I put the code into samples/exifprint and here's the result:

681 rmills@rmillsmbp:~/gnu/exiv2/trunk/samples $ make exifprint ; ../bin/exifprint test.jpg to.jpg mm.jpg ; exiv2 -g Software -pa *.jpg
g++ -O2 -fvisibility=hidden -fvisibility-inlines-hidden -Wall -Wcast-align -Wpointer-arith -Wformat-security -Wmissing-format-attribute -Woverloaded-virtual -W -MMD `pkg-config exiv2 --cflags` -c -o exifprint.o exifprint.cpp
mkdir -pv ../bin 2>&1 > /dev/null
../libtool --mode=link g++ `pkg-config exiv2 --libs` -rpath /usr/local/lib -L/usr/local/lib  -o ../bin/exifprint exifprint.o
libtool: link: g++ -o ../bin/exifprint exifprint.o -Wl,-bind_at_load  -L/usr/local/lib /usr/local/lib/libexiv2.dylib -L/Users/rmills/gnu/exiv2/trunk/xmpsdk/src /usr/local/lib/libintl.dylib -lc -liconv -lz /usr/local/lib/libexpat.dylib -ldl
image class has io size of 2223932
mm.jpg                Exif.Image.Software                          Ascii       7  Picasa
test.jpg              Exif.Image.Software                          Ascii       7  Picasa
to.jpg                Exif.Image.Software                          Ascii      25  Robin's Wee Test Program
682 rmills@rmillsmbp:~/gnu/exiv2/trunk/samples $ ls -alt *.jpg
-rw-r--r--+ 1 rmills  staff  2223932 20 Nov 01:02 mm.jpg
-rw-r--r--+ 1 rmills  staff  2223932 20 Nov 01:02 to.jpg
-rwx------@ 1 rmills  staff  2226521 20 Nov 00:46 test.jpg
683 rmills@rmillsmbp:~/gnu/exiv2/trunk/samples $ 

I think we've copied the buffer and modified it correctly. However the library buffer != buffer and I'm not sure there's a way to tell the image_ptr "write to this file" (transfer might do the trick).

I'll have another look at this tomorrow and update this topic.

Robin

RE: Why? Trying to write jpeg in memory and save to file does NOT work. - Added by Robin Mills about 7 years ago

I've found the fix. The if ( inMemory ) block should read:

    inMemory = true ;
    if ( inMemory ) {
        // create an exiv2 image in memory using the jpeg buffer
        Exiv2::Image::AutoPtr i = Exiv2::ImageFactory::open((const Exiv2::byte*)buffer, buffersize);
        i->setExifData(exifData); 
        i->writeMetadata(); 

        // copy the library buffer to buff
        buffersize = i->io().size(); 
        std::cout << "image class has io size of " << buffersize << "\n";
        i->io().seek(0,Exiv2::BasicIo::beg);
        Exiv2::DataBuf buff = i->io().read(buffersize);

        // ---------------------------------------------------------
        // Write the buff to disk
        FILE * file = ::fopen(mm_path.c_str(), "w"); 
        ::fwrite(buff.pData_, buffersize, 1, file); 
        ::fclose(file); 
    }

The reason is because the library cannot use your memory buffer to do its work if the modified metadata exceeds the buffer you provided. So, you have to create a new Exiv2::DataBuf containing a copy of the library's metadata buffer. Then you can write that buffer copy to disk.

709 rmills@rmillsmbp:~/gnu/exiv2/trunk/samples $ make exifprint ; ../bin/exifprint test.jpg to.jpg mm.jpg ; ls -alt *.jpg ; exiv2 -g Software *.jpg
mkdir -pv ../bin 2>&1 > /dev/null
../libtool --mode=link g++ `pkg-config exiv2 --libs` -rpath /usr/local/lib -L/usr/local/lib  -o ../bin/exifprint exifprint.o
libtool: link: g++ -o ../bin/exifprint exifprint.o -Wl,-bind_at_load  -L/usr/local/lib /usr/local/lib/libexiv2.dylib -L/Users/rmills/gnu/exiv2/trunk/xmpsdk/src /usr/local/lib/libintl.dylib -lc -liconv -lz /usr/local/lib/libexpat.dylib -ldl
image class has io size of 2223932
-rw-r--r--+ 1 rmills  staff  2223932 20 Nov 07:52 mm.jpg
-rw-r--r--+ 1 rmills  staff  2223932 20 Nov 07:52 to.jpg
-rwx------@ 1 rmills  staff  2226521 20 Nov 00:46 test.jpg
mm.jpg                Exif.Image.Software                          Ascii      25  Robin's Wee Test Program
test.jpg              Exif.Image.Software                          Ascii       7  Picasa
to.jpg                Exif.Image.Software                          Ascii      25  Robin's Wee Test Program
710 rmills@rmillsmbp:~/gnu/exiv2/trunk/samples $ 

If you're happy with this solution, I'd like to delete Topic#1858.

Robin

RE: Why? Trying to write jpeg in memory and save to file does NOT work. - Added by Andreas Huggel about 7 years ago

You can also write it back to a file using Exiv2 functionality, without the need to fallback on cstdio:

    Exiv2::DataBuf buf = Exiv2::readFile("image.jpg");
    Exiv2::Image::AutoPtr image = Exiv2::ImageFactory::open(buf.pData_, buf.size_);
    image->readMetadata();
    Exiv2::ExifData &exifData = image->exifData();
    exifData["Exif.Image.Model"] = "A long model name to make the Exif metadata grow";
    image->writeMetadata();
    Exiv2::FileIo file("image.jpg");
    file.open();
    file.write(image->io());

Attached is the complete sample program.

Andreas

memio.cpp (907 Bytes) memio.cpp Manipulating metadata on an in-memory image

RE: Why? Trying to write jpeg in memory and save to file does NOT work. - Added by Robin Mills about 7 years ago

Good stuff, Andreas. Your solution is much better than mine. I'm going to delete Topic#1858 as erroneous (although Rich was well intended).

Robin

RE: Why? Trying to write jpeg in memory and save to file does NOT work. - Added by Alexander Lyubushin almost 7 years ago

Dear Andreas

The solution discussed above does not resolve the problem of composition of edited images. For example, I have MPO file to be edited. The MPO file consists of 2 images, each of them have Exif-data. Suppose I want to crop images or edit histograms or edit red eyes. I convert edited bitmaps into jpeg parts and those parts are started with JFIF info. But I would like to keep Exif info before each of future 2 parts of MPO files. I also need to modify slightly Exif with changed sizes and something else. After that I have to compose the whole edited MPO file and save it on hard drive.

I did not find any acceptable sample how to use Exiv2 library for my task.

It will be nice if the list of samples will be extended by appropriate sample for the problem described.

Alex

RE: Why? Trying to write jpeg in memory and save to file does NOT work. - Added by evan pan about 6 years ago

Hi, Andreas.
As for myself, I have seldom tried to write jpeg in memory and save to file . I wonder have you ever worked it out? Do I need another 3rd party manual toolkit? When it comes JPG conversion (see: http://www.pqscan.com/convert-pdf/to-jpg-csharp.html ) process, I have another question, I wonder have you ever tried to convert pdf to jpg files (see: http://www.pqscan.com/convert-pdf/to-jpg.html ) before? As for myself, I am testing the related PDF to JPGG converting, PDF to BMP converting , and PDF to PNG converting programs these days. Do you have experience about it? Any suggestion will be appreciated. Thanks in advance.

Best regards,
Pan

RE: Why? Trying to write jpeg in memory and save to file does NOT work. - Added by Robin Mills about 6 years ago

I was the lead engineer at Adobe on the PDFscanSDK which converts scanned images into PDF. I believe you want to do the opposite which is simpler. There are numerous toolkits for doing this. Converting file formats (and rendering) are outside the domain of Exiv2 and you will have to search elsewhere for something to meet your needs.

Ghost Script can generate tiff from the command-line. http://ghostscript.com If you're working in C# (or any of the .Net languages), there are toolkits which are probably better integrated with Windows. PDFlib and Foxit provide impressive PDF libraries and tools: http://www.pdflib.com https://www.foxitsoftware.com/products/sdk/pdf-sdk/

It hope to add PDF support to samples/exiv2json.cpp for v0.26 and thinking about using libpodofo to parse the PDF file. podofo is a PDF parse/create library and does not support rendering. So it's unable to create a JPG (or any other bitmap format) from a PDF. There is no plan to make PDF a fully supported file format in libexiv2. I consider this to be a fun (low priority) project for wet winter evenings.

    (1-8/8)