Exiv2

r4554 Buildserver maintenance. Documentation Upate. Simplified script names. Almost all code moved from Jenkins into SVN.

I'm going to keep the Done of this issue at 100 and bump the Estimate time when I discover or modify anything. This keeps Redmine % done at 97+% and accounts for the effort I have put into this.

Applied a lot of polish and simplification to the buildserver scripts.

r4564 Added test/stdin-test. Fixed isse in with TiffImage.doWriteMetadata not respecting Image.writeXmpFromPacket()

All the buildserver maintenance stuff has been painfully slow. 5 minutes to change a script. 1.5 hours to run a build! It seems to be working OK now.

r4565. Extended stdin/stdout support to enable ICC and exv file structures to be piped.

Here are matters that I intend to investigate:
1 Build/test exiv2 on a bigEndian machine
2 Validate msvc2005/configure.py works with the trunk code
3 Some of the test suite (imagetest, crwtest, iotest, iptctest are broken on MinGW/32)
4 Known bugs #1011 and #1023
5 trunk builds broken on cygwin and mingw, yet build fine manually
6 I'll think of other matters to investigate !

Tested on MacOS-X 10.4 PPC bigEndian computer.

Only failure is:

$ exiv2 -pp test/data/imagemagick.pgf

$ 501 /Users/rmills $ cd ~/gnu/exiv2/trunk
502 /Users/rmills/gnu/exiv2/trunk $ exiv2 -pa test/data/imagemagick.pgf
Exiv2 exception in print action for file test/data/imagemagick.pgf:
Failed to read input data
503 /Users/rmills/gnu/exiv2/trunk $ exiv2 -vV -g svn -g platform -g bits -g compiler
exiv2 0.25 001900 (32 bit build)
platform=apple
compiler=G++
bits=32
svn=4571
504 /Users/rmills/gnu/exiv2/trunk $ 

This is my old Adobe laptop from 2003! My first Mac Laptop. I've had 4 new MacBook Pro laptops since then. It's clunky and slow. However it's a very good test that our code builds on elderly UNIX, bigEndian and 32bit hardware. Worth the effort.

Various minor things. See SVN log.

I'm trying again to debug the MinGW/32 CMake build. I solved this for libssh and that has given me the courage and confidence to work on this. Here's what I did with libssh:

I know what’s wrong with this. I’ve built the DLL (shared library). When I explain, you should be able to build the static library.

[ 94%] Building C object examples/CMakeFiles/exec.dir/exec.c.obj
[ 95%] Building C object examples/CMakeFiles/exec.dir/authentication.c.obj
[ 97%] Building C object examples/CMakeFiles/exec.dir/knownhosts.c.obj
[ 98%] Building C object examples/CMakeFiles/exec.dir/connect_ssh.c.obj
[100%] Linking C executable exec.exe
[100%] Built target exec
573 -32- /home/rmills/gnu/libssh/libssh-0.7.3/build> find . -name "*.a" -exec ls -alt {} \;
-rw-r--r-- 1 rmills root 11750 Sep 30 12:42 ./examples/CMakeFiles/exec.dir/objects.a
-rw-r--r-- 1 rmills root 11012 Sep 30 12:42 ./examples/CMakeFiles/libsshpp.dir/objects.a
-rw-r--r-- 1 rmills root 5990 Sep 30 12:42 ./examples/CMakeFiles/libsshpp_noexcept.dir/objects.a
-rw-r--r-- 1 rmills root 11792 Sep 30 12:42 ./examples/CMakeFiles/senddata.dir/objects.a
-rw-r--r-- 1 rmills root 651698 Sep 30 12:42 ./src/CMakeFiles/ssh_shared.dir/objects.a
-rw-r--r-- 1 rmills root 241366 Sep 30 12:42 ./src/libssh.dll.a                                            <-- GOOD
-rw-r--r-- 1 rmills root 2122 Sep 30 12:42 ./src/threads/CMakeFiles/ssh_threads_shared.dir/objects.a
-rw-r--r-- 1 rmills root 1538 Sep 30 12:42 ./src/threads/libssh_threads.dll.a                              <-- GOOD
574 -32- /home/rmills/gnu/libssh/libssh-0.7.3/build> find . -name "*.dll" -exec ls -alt {} \;
-rwxr-xr-x 1 rmills root 2291514 Sep 30 12:42 ./src/libssh.dll                                             <-- GOOD
-rwxr-xr-x 1 rmills root 41701 Sep 30 12:42 ./src/threads/libssh_threads.dll                               <-- GOOD

CMake is generating the following wrong response files:

575 -32- /home/rmills/gnu/libssh/libssh-0.7.3/build> find . -name "*.rsp" -exec ls -alt {} \;
-rw-r--r-- 1 rmills root 425 Sep 30 12:41 ./examples/CMakeFiles/exec.dir/includes_C.rsp                    <-- WRONG
-rw-r--r-- 1 rmills root 187 Sep 30 12:36 ./examples/CMakeFiles/exec.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 148 Sep 30 12:36 ./examples/CMakeFiles/exec.dir/objects1.rsp
-rw-r--r-- 1 rmills root 264 Sep 30 12:36 ./examples/CMakeFiles/libsshpp.dir/includes_CXX.rsp              <-- WRONG   
-rw-r--r-- 1 rmills root 187 Sep 30 12:36 ./examples/CMakeFiles/libsshpp.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 42 Sep 30 12:36 ./examples/CMakeFiles/libsshpp.dir/objects1.rsp
-rw-r--r-- 1 rmills root 264 Sep 30 12:36 ./examples/CMakeFiles/libsshpp_noexcept.dir/includes_CXX.rsp     <-- WRONG
-rw-r--r-- 1 rmills root 187 Sep 30 12:36 ./examples/CMakeFiles/libsshpp_noexcept.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 60 Sep 30 12:36 ./examples/CMakeFiles/libsshpp_noexcept.dir/objects1.rsp
-rw-r--r-- 1 rmills root 425 Sep 30 12:41 ./examples/CMakeFiles/senddata.dir/includes_C.rsp                <-- WRONG
-rw-r--r-- 1 rmills root 187 Sep 30 12:36 ./examples/CMakeFiles/senddata.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 168 Sep 30 12:36 ./examples/CMakeFiles/senddata.dir/objects1.rsp
-rw-r--r-- 1 rmills root 425 Sep 30 12:41 ./src/CMakeFiles/ssh_shared.dir/includes_C.rsp                   <-- WRONG
-rw-r--r-- 1 rmills root 166 Sep 30 12:36 ./src/CMakeFiles/ssh_shared.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 2137 Sep 30 12:36 ./src/CMakeFiles/ssh_shared.dir/objects1.rsp
-rw-r--r-- 1 rmills root 425 Sep 30 12:40 ./src/threads/CMakeFiles/ssh_threads_shared.dir/includes_C.rsp   <-- WRONG
-rw-r--r-- 1 rmills root 182 Sep 30 12:36 ./src/threads/CMakeFiles/ssh_threads_shared.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 49 Sep 30 12:36 ./src/threads/CMakeFiles/ssh_threads_shared.dir/objects1.rsp
576 -32- /home/rmills/gnu/libssh/libssh-0.7.3/build> 

-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/build/src/threads
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/src/threads
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/include
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/build
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3
-IC:/Qt/Qt5.6.0/Tools/mingw492_32/i686-w64-mingw32/include
-IC:/MinGW/msys/1.0/local/include 

I had to generate the build files with this command:

cmake .. -G "Unix Makefiles" -DCMAKE_C_COMPILER=$(which gcc) -DCMAKE_CXX_COMPILER=$(which g++)

I had to build and install openssl with the commands (in the openssl delivery):

$ ./config --prefix=/usr/local
$ make
$ make install

To build the static library, you’ll have to edit CMakeLists.txt. You may well find that the linker response files are not correct for static libraries.

//////////// Here are notes I made along the way to solving this ////////////////////////

I tried to build libssh this morning on MinGW/64. The experience was the same as MinGW/32.

1) I had to edit sdkddkver.h to build CMake

2) I had to bootstrap into —prefix=/usr/local (why is bootstrap not called configure?)

3) I had to use CMake with this command:

cmake .. -G "Unix Makefiles" -DCMAKE_C_COMPILER=$(which gcc) -DCMAKE_CXX_COMPILER=$(which g++)

4) Here’s what happens when I run make:

c:/MinGW64/msys/1.0/bin/cmake.exe -E cmake_progress_report ...
[  1%] Building C object src/CMakeFiles/ssh_shared.dir/auth.c.obj
cd C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/build/src && c:/TDM-GCC-64/bin/gcc.exe \
  -DLIBSSH_EXPORTS @CMakeFiles/ssh_shared.dir/includes_C.rsp   \
  -o CMakeFiles/ssh_shared.dir/auth.c.obj \
  -c C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/src/auth.c
In file included from c:/MinGW64/msys/1.0/include/sys/unistd.h:9:0,
                 from c:/MinGW64/msys/1.0/include/unistd.h:6,
                 from C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/include/libssh/libssh.h:59,
                 from C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/include/libssh/priv.h:138,
                 from C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/src/auth.c:34:
c:/MinGW64/msys/1.0/include/sys/types.h:71:18: error: conflicting types for 'time_t'
 typedef _TIME_T_ time_t; 

5) Exiv2 does not build with CMake either. Dies a terrible death instantly down inside a stack of system include files involving pthread.

6) My notes about CMake are here:
svn://dev.exiv2.org/svn/trunk/contrib/buildserver/dailyCMake.sh

Some Exiv2 users love CMake. I do not. It works well on Linux/MacOS-X/Cygwin. It doesn’t seem to work on MinGW. It’s an obstructive monster with Visual Studio. Of the 5 platforms we support on Exiv2, MinGW is the one I like least. However we build and test Exiv2 on all 5 platform on every commit. We build and test all 5 platforms and 6 editions of Visual Studio in 32 and 64 bits every night.

One of the challenges of using a build system such as CMake is to fix the build when CMake make mistakes. Because there is an abstract layer (CMakeLists.txt) between the code and the compiler, when things go wrong you have to crawl through the generated magic. This is not fun. The Linux World’s attitude “It must be the fault of Windows, because it works on Linux” is very unhelpful. You’re unlikely to get any sympathy on an opensource Forum. It’s your lucky day to have met a helpful open-source contributor who is not Windows hostile.

I’m very pleased to have worked on this. I still haven’t got Exiv2 to build using CMake on MinGW/32. However, I’m confident that I’ll fix that over the weekend.

I'm removing the 8 hours in this issue that I have spent on CMake/MinGW. That time is being spend on 1236. I started the bug hunt last week. I'll spend another week on it. So 50% done.

I'm put in an effort on contrib/buildserver/categorize.sh and rewritten in Python. Down from 25 minutes to 0.6 seconds. Much better.

I'm dragging my feet on testing msvc2005/configure.py