MSVC & Unicode

Added by Arnold Wiegert about 3 years ago

From a brief look through this forum, it seems that Unicode is a problem for exiv2.
Is that correct?
I am trying to integrate exiv2 into an application created by someone else (vvvP) and am running into a number of issues, without much of a good idea (yet) as to whether they originate with the old application (supposedly compiled with CMake using exiv2 and Unicode).
Not being a CMake guy, and preferring MSVC, I can compile both the app and exiv2 under MSVC 2013, using Unicode strings and the multi-threaded debug dll (/MTd)
- just now using only the debug version.
The both compiles ( app & exiv2 lib) go well, but I am running into unresolved externals in the link step.
So, the first real question is: is exiv2 known to work with Unicode?


Replies (38)

RE: MSVC & Unicode - Added by Arnold Wiegert about 3 years ago

My assertion of a good Unicode compile was inaccurate.
I get a number of errors and will have to try and sort those out.
(Does this forum not allow editing my own post?? )

RE: MSVC & Unicode - Added by Robin Mills about 3 years ago

As an Forum Admin, I can edit posts. I don't believe that permission is granted to users. I can't grant you permission to edit posts.

I believe you can build this with msvc2005/exiv2.sln using Visual Studio 2005/8/10/12/13/15.

Edit include/exiv2/exv_msvc.h to set the EXV_UNICODE_PATH build switch.

Change:

/* Windows unicode path support */
// #define EXV_UNICODE_PATH 1
To:
/* Windows unicode path support */
#define EXV_UNICODE_PATH 1
Open msvc2005/exiv2.sln in Visual Studio, select your target { Static|Dynamic Win32|x64 Debug|Release) and build!

I've never actually built or tested the EXV_UNICODE_PATH configuration, however I believe it works.

RE: MSVC & Unicode - Added by Robin Mills about 3 years ago

I've setup this and it builds and (mostly) passes the test suite. I've opened an issue report to validate all of this. #1169.

At the moment, part of our test suite is failing - however I suspect this is because some sample/test applications don't respect EXV_UNICODE_PATH. Our most important test script test/bugfixes.sh passes. It uses exiv2.exe, so the build is basically sound.

I found and fixed a compilation unsigned/signed mismatch error in src/actions.cpp which is part of exiv2.exe (and not the library). I've also updated msvc2005/ReadMe.txt to discuss EXV_UNICODE_PATH. r4235

Thank You for using Exiv2

RE: MSVC & Unicode - Added by Arnold Wiegert about 3 years ago

I have now been able to build a more or less clean Unicode debug library of exiv2.

Setting the #define EXV_UNICODE_PATH 1 made no difference to my results, but then I have not tested any of the exiv2 executables as I am only interested in the library.
There I am stuck now with two unresolved externals which I have to sort out:

1>image_preview.obj : error LNK2019: unresolved external symbol "public: static class std::auto_ptr<class Exiv2::Image> __cdecl Exiv2::ImageFactory::open(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (?open@ImageFactory@Exiv2@SA?AV?$auto_ptr@VImage@Exiv2@std@ABV?$basic_string@DU?$char_traits@D@std@V?$allocator@D@2@4@Z) referenced in function "public: static enum CImagePreview::ReturnValue __cdecl CImagePreview::ExtractImagePreview(class wxString const &,int,class wxImage &,class CMemoryChunk &,enum CMetadata::ColorSpace &)" (?ExtractImagePreview@CImagePreview@SA?AW4ReturnValue@1@ABVwxString@HAAVwxImage@AAVCMemoryChunk@AAW4ColorSpace@CMetadata@@Z)

1>metadata.obj : error LNK2001: unresolved external symbol "public: static class std::auto_ptr<class Exiv2::Image> __cdecl Exiv2::ImageFactory::open(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (?open@ImageFactory@Exiv2@SA?AV?$auto_ptr@VImage@Exiv2@std@ABV?$basic_string@DU?$char_traits@D@std@V?$allocator@D@2@4@@Z)

1>uuidwrapper.obj : error LNK2019: unresolved external symbol _uuid_create referenced in function "public: static class wxString __cdecl CUUIDWrapper::ComputeUUID(void)" (?ComputeUUID@CUUIDWrapper@SA?AVwxString@XZ)

RE: MSVC & Unicode - Added by Arnold Wiegert about 3 years ago

OK, got rid of the second unresolved symbol _uuid_create (an existing file was not part of the project files list), so now I am left only with the one(s) from Exiv2

RE: MSVC & Unicode - Added by Arnold Wiegert about 3 years ago

The unresolved external persists with or without the #define EXV_UNICODE_PATH 1
Probably will have to revise the calling args by the app to be consistent with Exiv2

RE: MSVC & Unicode - Added by Robin Mills about 3 years ago

I'll look at this tomorrow. I didn't write Exiv2. We will fix this together. I notice you haven't said "Thank You" for my support.

RE: MSVC & Unicode - Added by Arnold Wiegert about 3 years ago

Yes, I agree, I was i bit preoccupied with trying to master the compile for Unicode.
My apologies and "thank you" for your comments.

Though, I am not sure there is anything that is not working with exiv2.
The code I am trying to resurrect is several years old and I have no idea which version of exiv2 it was originally compiled with.

RE: MSVC & Unicode - Added by Robin Mills about 3 years ago

Good idea to bring your code up to date with the current trunk. When we're both "on the same page" we'll fix everything quickly. ImageFactory::open() was changed in 2013 for the Web Ready project to enable URLs as a path. From memory, I believe it was changed from:

open(std::string path);
to:
open(std::string path,bool bUseCurl=false);
I'm not bothered by the test suite failures as I am quite certain they are issues concerning the sample applications not respecting EXV_UNICODE_PATH.

Looking after an open source project is a lot of unpaid work. The build matrix of 64/32/static/dynamic/release/debug is huge. MSVC 2003/5/8/10/12/13/15 all have to be built and tested in addition to cygwin/mingw/linux/macosx. Different compilers: gcc/clang/cl. Build options such as EXV_UNICODE_PATH, with/without curl/expat/zlib/libssh/openssl. Build environments: CMake/MSVC/Autotools. User correspondence: issues/forum/email. After that comes the on-going development of the library itself. A few words of appreciation for this effort are appreciated and seldom received.

RE: MSVC & Unicode - Added by Arnold Wiegert about 3 years ago

It is good to see you've taken on the job of maintenance for Exiv2. It certainly is and was appreciated that there is someone who will take the time to answer newb questions - thank you.
I am very new to it so I have no idea at all about its history or the people who were and/or are involved.
As far as I can tell, my code ought to be up-to-date, as I have down loaded the latest? 0.25 tarball.
Also, as I mentioned, I don't see where there is a problem - at least for me - with the exicv2 code.

In fact, I am not even familiar enough yet, as to how or where that app I am trying to resurrect is using Exiv2. :-)
I have commented out the calls which cause me trouble and it still does all I have been able to do (or rather all I have had a chance to test to this point).
From what I see, the issue I have to resolve is to translate a wxWidgets string to one of the acceptable std:: strings - which hopefully won't be too hard.
At present the issue only shows up at link time when the munged function name is not found instead at compile time, though for all I can tell, the proper headers are included.

As well, I am new to MSVC 2013 and find it way more temperamental than 2010, which I have been using mostly up to now.
I am sure to try 2015 ASAP to see if it is more reliable.

As for testing, I have not even tried any of the command line utilities that come with the library.

All I need is the library, though I may well have to dig deeper if I really want to use the facilities provided by exiv2. My main interest for resurrecting the old app is to be able to use more of the metadata facilities of graphics formats & files

RE: MSVC & Unicode - Added by Arnold Wiegert about 3 years ago

OK, got all my compile issues for the old app sorted out.
The old all had included some - evidently obsolete - exiv2 headers.
Pointing the compiler to the current headers - part of the exiv2 package - cleared it all up.

Expanding the exiv2 based capabilities of the app, will no doubt have me back asking more questions. :-)

Again, thank you for your help.
If I can help in any way, let me know.

RE: MSVC & Unicode - Added by Robin Mills about 3 years ago

I would really like to recruit somebody to help with the C++ code. Have a look in the news, and you'll see what's on my plate. If you can help with any of the outstanding stuff for v0.26, let me know. I'm very happy to mentor as you get to understand the code. http://dev.exiv2.org/projects/exiv2/news

Another time-sink for me is Forum/Issue Correspondence. If you can answer anything, please answer it. Don't worry about making mistakes. I make mistakes all the time. I always try to give the correct answer - however to err is human. Some of the correspondence is very simple. Somebody's asking about exiv2/exiftool today. You don't need to know much about Exiv2 to answer those questions.

Thanks for using Exiv2. I will happily help you if you encounter issues with Exiv2 in your project.

RE: MSVC & Unicode - Added by Arnold Wiegert almost 3 years ago

I had a brief look at the link you gave, in particular for issue # 1121

I am curious why you still want to support back to MSVC 2003. From my perspective that is so ancient that it seems close to looking for support for DOS 6.2 :-)

Is there any particular need for anything before even MSVC 2010? Aside from the fact that it can be done

While I am no CMake guru or fan, my experience being almost exclusively MSVC, I would be interested in running some test in the MSVC environment. I believe you mentioned that you had something along those lines working at some time in the past?

Presently my experience in testing is rather limited, though I have played with and used cppunit for a couple of my projects.

My own plate is also quite full, but I am hoping that I can always squeeze a bit of time for things such as exiv2, which I am hoping will also help me with my own projects. ;-)

RE: MSVC & Unicode - Added by Robin Mills almost 3 years ago

I've been working with Exiv2 since 2008. I started on the project by developing the msvc2003 projects and they built with Visual Studio 2003 and 2005 because those systems were in use everywhere at that time. The code cannot build with DevStudio6.0 because it doesn't support STL. Later, I added msvc2005 to build 64 and/or 32 bit. I've kept msvc2003 and msvc2005 alive as Microsoft have shipped new versions of Visual Studio. Support for msvc2003 will be dropped after v0.26. Significant new features such as webready are not supported by msvc2003.

I put 200 hours into our CMake/VisualStudio support last fall. I believe it's working well for Visual Studio 2005/8/10/12/13/15 <exiv2dir>/contrib/cmake/msvc

I don't like CMake much for MSVC. I much prefer the solution/project files. Daniel thinks we should dump the solution/project files. However Daniel is a real Linux guy. I think most Windows/Visual Studio users will not like CMake. If you have some time, you could take a look at CMake/Visual Studio and give me feedback. Don't be surprised if you discover it's broken. I haven't looked at it since before Christmas and you may discover there is something horribly wrong. I will revisit this before we ship v0.26 in the summer.

Our test suite is written in bash and requires cygwin to run the test suite. On Windows, the suite can test msvc builds as well as Cygwin and MinGW builds. I've thought about rewriting the test suite in python to avoid the dependency on cygwin. I will never have time to do that. http://dev.exiv2.org/projects/exiv2/wiki/How_do_I_run_the_test_suite_for_Exiv2

I have quite a lot of things I would like to do for Visual Studio in v0.26 #1121

I'm quite enthusiastic to retain MSVC support for ancient old systems like Visual Studio 2005. These things are free downloads today, so students in India can get them for nothing and learn about Windows (they don't, but they could). I was a contractor at a small company in Silicon Valley. They had a lot of legacy code that built with DevStudio 6.0. I think you'd be surprised at how much ancient stuff is still used around the world. You've probably discovered that I am a retired Adobe Engineer. Adobe do maintain their code in pristine shape on current platforms. Smaller companies struggle with their legacy code.

One aspect of working on open-source is that I have no idea how many people use Exiv2 and for what purposes. I only hear from folks when something is broken!

RE: MSVC & Unicode - Added by Arnold Wiegert almost 3 years ago

Whether I'll be able to help with CMake will have to be seen. I don't have it installed at the moment and don't know if I can climb that hill ;-)
It so happens I do have cygwin64 installed already and so I tried a couple of things following the instructions on the wiki link you sent.

The first problem I ran into:
$ ./configure
....
checking whether to compile with video support... no
checking whether to compile with webready support... no
checking expat.h usability... no
checking expat.h presence... no
checking for expat.h... no
configure: error: either specify a valid expat installation with --with-expat=DIR or disable XMP support with --disable-xmp

To fix this, I looked for expat code within the package & found at least 3 in msvc2005, all containing only a project file
To get my compile to work I installed expat-2.1.0 in the project root and used it, but have had no luck pointing the --with-expat to the right directory; it gives me the same error as above.

$ ./configure --disable-xmp
runs to completion, but gives me:
arnold@Hummer /cygdrive/d/pkg/wx/MSVC2015/exiv2-0.25
$ make
if test "x" = "x1"; then cd xmpsdk/src && C:/gnuwin32/bin/make xmpsdk; fi;
cd src && C:/gnuwin32/bin/make
make1: Entering directory `D:/pkg/wx/MSVC2015/exiv2-0.25/src'
Xlocale" c -DEXV_BUILDING_LIB=1 -o basicio.o basicio.lo
D:\pkg\wx\MSVC2015\exiv2-0.25\libtool: compile: warning: libobj name `locale" -c -DEXV_BUILDING_LIB=1 -o basicio.o basicio.lo' may not contain shell special characters.
rm: unknown option -
c
Try 'rm --help' for more information.
D:\pkg\wx\MSVC2015\exiv2-0.25\libtool: compile: g++ -O2 -Wall -Wcast-align -Wpointer-arith -Wformat-security -Wmissing-format-attribute -Woverloaded-virtual -W -Wundef -pedantic -MMD -I../src -I../include/ -I../include/exiv2 "-DEXV_LOCALEDIR=\\/usr/local/share/locale\" -c -DEXV_BUILDING_LIB=1 -o basicio.o basicio.cpp" -DDLL_EXPORT -DPIC -o .libs/locale" -c -DEXV_BUILDING_LIB=1 -o basicio.o basicio.o
D:\pkg\wx\MSVC2015\exiv2-0.25\libtool: eval: line 991: unexpected EOF while looking for matching `"'
D:\pkg\wx\MSVC2015\exiv2-0.25\libtool: eval: line 992: syntax error: unexpected end of file
make1: * [basicio.o] Error 2
make1: Leaving directory `D:/pkg/wx/MSVC2015/exiv2-0.25/src'
make: *
[all] Error 2

RE: MSVC & Unicode - Added by Robin Mills almost 3 years ago

Ah. Build expat 2.1.0 and zlib 1.2.8 from the public tar-balls and install them.

521 rmills@rmillsmbp:~/gnu $ dirs | grep -e exiv2 -e zlib -e expat 
drwxr-xr-x+ 20 rmills  staff      680 28 Feb 20:45 exiv2/
drwxr-xr-x+ 36 rmills  staff     1224 12 Nov 17:16 gexiv2/
drwxr-xr-x+  5 rmills  staff      170  5 Oct 08:53 py3exiv2/
drwxr-xr-x+  3 rmills  staff      102  5 Aug  2013 zlib/
drwxr-xr-x+  3 rmills  staff      102  5 Aug  2013 expat/
$ cd ~/gnu/expat/expat-2.1.0 ; ./configure ; make ; sudo make install
Do the same in ~/gnu/zlib/zlib-1.2.8 . Then go to ~/gnu/exiv2/trunk and
 $ make config ; ./configure ; make ; sudo make install ; make samples ; make tests
It'll work. Trust me!

RE: MSVC & Unicode - Added by Arnold Wiegert almost 3 years ago

Reinstalled cygwin 64 just to be sure, but I get an error from make after ./configure completes

...
config.status: expat_config.h is unchanged
config.status: executing libtool commands

arnold@Hummer /cygdrive/d/pkg/wx/MSVC2015/exiv2-0.25/expat-2.1.0
$ make
C:/cygwin64/bin/sh.exe ./libtool --silent --mode=link gcc -std=gnu99 -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DHAVE_EXPAT_CONFIG_H -no-undefined -version-info 7:0:6 -rpath /usr/local/lib  -o libexpat.la lib/xmlparse.lo lib/xmltok.lo lib/xmlrole.lo
/usr/lib/gcc/x86_64-pc-msys/4.9.2/../../../../x86_64-pc-msys/bin/ld: cannot find -link
collect2: error: ld returned 1 exit status
make: *** [libexpat.la] Error 1

arnold@Hummer /cygdrive/d/pkg/wx/MSVC2015/exiv2-0.25/expat-2.1.0

The zlib part (testing with 1.2.7) gives

....
gcc -O3  -o example.exe example.o -L. libz.a
gcc -O3  -I. -c -o minigzip.o test/minigzip.c
gcc -O3  -o minigzip.exe minigzip.o -L. libz.a

arnold@Hummer /cygdrive/d/pkg/wx/MSVC2015/exiv2-0.25/zlib-1.2.7
$ sudo make install
-bash: sudo: command not found

arnold@Hummer /cygdrive/d/pkg/wx/MSVC2015/exiv2-0.25/zlib-1.2.7
$ make install
cp libz.a /usr/local/lib
cp: cannot create regular file `/usr/local/lib': No such file or directory
make: *** [install-libs] Error 1

arnold@Hummer /cygdrive/d/pkg/wx/MSVC2015/exiv2-0.25/zlib-1.2.7

RE: MSVC & Unicode - Added by Robin Mills almost 3 years ago

I think you might need to delete the files config.log and config.status. These files "remember" what happened last time you ran ./configure.

It's odd that you don't have a directory /usr/local, you better create it. And create cousins bin and include:

$ mkdir -p /usr/local/lib; mkdir -p /usr/local/bin ; mkdir -p /usr/local/include

sudo is a Linux/MacOS-X command "Super User Do". You don't need it on cygwin. Now that you've created /usr/local/lib, just do

$ make install
I've got a script /usr/local/bin/sudo
#!/bin/bash

## sudo - dummy script
"$@" 

# That's all Folks
##
This saves me thinking "do I need sudo on this platform or not?". It's just there. On Linux/MacOS-X sudo elevates the user permissions for a single command. My home-made script simply executes the arguments you pass.

RE: MSVC & Unicode - Added by Arnold Wiegert almost 3 years ago

Thanks for the sudo script.
(At one point I did dabble with Linux and still do if I have to, as well I spent a few years working with Solaris as part of one of my jobs, so sudo is not really a mystery for me; I had just assumed that perhaps cygwin might have a Win substitute, which also might elevate permissions if needed)

After all the issues I had, I tried for a clean start & reinstalled cygwin64, Then I realized I needed more than just the basic shell, because after the initial install, the scripts could not even find any compiler.
Then, to be safe I reinstalled the exiv2 package into a directory separate from my working copy for my app, just to avoid any interference.

At that point, it looked like ./configure ran to completion as did make install, but make samples does not.
Lots of:

....

ExifData::add(Exiv2::ExifKey const&, Exiv2::Value const*)'
addmoddel.o:addmoddel.cpp:(.text.startup+0x39b): undefined reference to `Exiv2::Value::Value(Exiv2::TypeId)'

....

So now I am still stuck with a number of questions:
does it matter which drive the exiv2 package is installed on?
Currently cygwin is on C: with my development code, including exiv2 on D:
Or would the exiv2 code have to be on within cygdrive C?
What sort of cygwin packages are essential for this job?
(Installing 'all' just takes up too much time and disk space :-) )

RE: MSVC & Unicode - Added by Robin Mills almost 3 years ago

There's very little wrong here. However software is a kind of dog. If you don't understand him, when you go to stroke him, he'll bite your hand off!

Is it possible screenshare on Skype 'clanmills' and I'll "talk you down" "(don't call her striker)"

RE: MSVC & Unicode - Added by Robin Mills almost 3 years ago

You shouldn't really need to worry about drive letters. Cygwin has a filesystem which starts at / c:\ => /cygdrive/c d:\ => /cygdrive/d and so on.

The messages you are getting are from the linker and effectively say "I cannot find the Exiv2 library when I'm linking the sample program addmoddel". (addmoddel means: add mod del which means add modify delete) addmoddel is used by our test suite.

The library should be build and installed in /usr/local/lib/libexiv*, the dlls and .exes in /usr/local/bin/exiv2.exe.

503 rmills@rmillsmbp-w7:/usr/local/lib $ dir | grep exiv2
-rw----r--  1 rmills rmills 5.7M Jan  7 21:43 libexiv2.a
-rw----r--  1 rmills rmills 1019 Jan  7 21:43 libexiv2.la
-rw-r--r--  1 rmills None   6.7M Dec 13 12:57 libexiv2.dll.a
504 rmills@rmillsmbp-w7:/usr/local/lib $ cd ../bin
505 rmills@rmillsmbp-w7:/usr/local/bin $ dir | grep exiv2
-rwx---r-x 1 rmills rmills 3.4M Jan  7 21:43 exiv2.exe*
-rwxr-xr-x 1 rmills None   448K Dec 13 12:57 exiv2json.exe*
-rwxr-xr-x 1 rmills None   5.0M Dec 13 12:57 cygexiv2-14.dll*
506 rmills@rmillsmbp-w7:/usr/local/bin $

If the library is there, then set a couple of environment strings and try again:

$ export LD_LIBRARY_PATH=/usr/local/lib
$ export PATH=$PATH:/usr/local/bin
$ make samples

RE: MSVC & Unicode - Added by Arnold Wiegert almost 3 years ago

Never skyped - don't have a mic nor a camera :-)

RE: MSVC & Unicode - Added by Arnold Wiegert almost 3 years ago

Here is what I get:

arnold@Hummer ~
$ cd /usr/local/lib

arnold@Hummer /usr/local/lib
$ dir |grep exiv2
libexiv2.a libexiv2.la pkgconfig

arnold@Hummer /usr/local/lib
$ cd ../bin

arnold@Hummer /usr/local/bin
$ dir |grep exiv2
exiv2.exe

arnold@Hummer /usr/local/bin
$ export LD_LIBRARY_PATH=/usr/local/lib

arnold@Hummer /usr/local/bin
$ export PATH=$PATH:/usr/local/bin

./configure
make
make install
make samples
Then we're back to the same errors

RE: MSVC & Unicode - Added by Arnold Wiegert almost 3 years ago

Had to also export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
Now make samples seems to be compiling a bunch of code

RE: MSVC & Unicode - Added by Arnold Wiegert almost 3 years ago

But:

arnold@Hummer /cygdrive/d/pkg/c++/exiv/exiv2-0.25
$ make tests
cd test && make test
/bin/sh: line 0: cd: test: No such file or directory
Makefile:74: recipe for target 'tests' failed
make: *** [tests] Error 1__

1 2 Next » (1-25/38)

Redmine Appliance - Powered by TurnKey Linux