Bug #1230

Bug Hunt for v0.26

Added by Robin Mills 10 months ago. Updated 9 months ago.

Status:ClosedStart date:24 Sep 2016
Priority:NormalDue date:
Assignee:Robin Mills% Done:

100%

Category:testingEstimated time:40.00 hours
Target version:0.26

Description

I will have a good search in "dark holes" for mysterious issues waiting to be discovered.


Related issues

Related to Exiv2 - Bug #1234: Class pgfimage does not work on big endian hardware. Closed 27 Sep 2016

Associated revisions

Revision 4552
Added by Robin Mills 10 months ago

#1230 Test suite review and maintenance

Revision 4553
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance. Remove \r from dist\logs files

Revision 4554
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance.

Revision 4555
Added by Robin Mills 10 months ago

#1230 Buildserver Maintenance.

Revision 4556
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance.

Revision 4557
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance

Revision 4558
Added by Robin Mills 10 months ago

#1230. Correction to r4557

Revision 4559
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance.

Revision 4560
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance.

Revision 4561
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance.

Revision 4562
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance.

Revision 4563
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance.

Revision 4564
Added by Robin Mills 10 months ago

#1230 Added stdin-test and fixing issue identified by the test.

Revision 4565
Added by Robin Mills 10 months ago

#1230 Extended test/stdin-test.sh to pipe .exv and .icc file structures

Revision 4566
Added by Robin Mills 10 months ago

#1230 Correction to r4565 to fix msvc build breaker.

Revision 4567
Added by Robin Mills 10 months ago

#1230 msvc2005 ini-test.vcproj simplication and referencing correct headers.

Revision 4570
Added by Robin Mills 10 months ago

#1230. Correction to r4569. Forgot to update test/data/geotag-test.out

Revision 4575
Added by Robin Mills 10 months ago

#1199 and #1230 webpImage::printStructure(). Reporting Offset == 0 on Tiger (32bit).

Revision 4576
Added by Robin Mills 10 months ago

#1230 and #1199 Correction to r4575

Revision 4584
Added by Robin Mills 10 months ago

#1230 updated READMEs.

Revision 4586
Added by Robin Mills 10 months ago

#1230 Eliminate linux compiler warning.

Revision 4589
Added by Robin Mills 10 months ago

#1230 Work in Progress. CMake/MinGW/32.

Revision 4598
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance. Replaced categorize.sh (25 minutes) with categorize.py (0.6 second)

Revision 4599
Added by Robin Mills 10 months ago

#1230 and #1236 Buildserver maintenance. Work-in-progress fixes for CMake/MinGW32

Revision 4600
Added by Robin Mills 10 months ago

#1230 CMake build fixes.

Revision 4602
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance. Correction to r4598.

Revision 4603
Added by Robin Mills 10 months ago

#1230 CMake Maintenance. Adding EXIV2_HDR to exiv2/headers and LIBEXIV2_PRIVATE_HDR to libexiv2/headers.

Revision 4604
Added by Robin Mills 10 months ago

#1230 Adding --verbose to print action to dump more information (to debug something on the buildserver)

Revision 4605
Added by Robin Mills 10 months ago

#1230 Buildserver maintenance.

Revision 4606
Added by Robin Mills 10 months ago

#1230. Rollback 4604. I knew webp-test was sometimes failing. Fixed. Always use runTest exiv2 to be sure of executing the hot build of exiv2(.exe).

Revision 4607
Added by Robin Mills 10 months ago

#1230 Buildserver Maintenance.

Revision 4608
Added by Robin Mills 10 months ago

#1230 exiv2.cpp #include <string> breaks the build on MinGW/32.

Revision 4609
Added by Robin Mills 10 months ago

#1230 More changes to pacify MinGW (the nightmare environment).

Revision 4610
Added by Robin Mills 10 months ago

#1230 More MinGW fixes.

Revision 4611
Added by Robin Mills 10 months ago

#1230 Fix cygwin buildbreaker.

Revision 4613
Added by Robin Mills 10 months ago

#1230 Polished ascii-art in documentation.

Revision 4614
Added by Robin Mills 10 months ago

#1230 More ascii art polishing.

Revision 4615
Added by Robin Mills 10 months ago

#1230 Fix MinGW stdin-test breaker

Revision 4616
Added by Robin Mills 10 months ago

#1230 Fix MinGW test suite.

Revision 4617
Added by Robin Mills 10 months ago

#1230 Work-in-progress. Validating msvc2005/configure.py

Revision 4618
Added by Robin Mills 10 months ago

#1230 Work-in-progress to restore configure.bat/configure.py to working health (see fix_vcproj.xslt for explanation).

Revision 4629
Added by Robin Mills 9 months ago

#1230 Fix bigEndian issue in toAscii()

Revision 4636
Added by Robin Mills 9 months ago

#1230 Fixes to read tiff with web-ready. Includes the BasicIo expiremental API readUnmarked() and markRead() which I will remove later.

Revision 4640
Added by Robin Mills 9 months ago

#1230 Remove dead script categorize.sh as it has been replaced by categorize.py

Revision 4643
Added by Robin Mills 9 months ago

#1230 Fixing issues with test suite on MinGW/32.

Revision 4670
Added by Robin Mills 9 months ago

#1230 Fix msvc build breaker when building using configure -x (without xmp)

Revision 4671
Added by Robin Mills 9 months ago

#1230 Fix msvc build breaker when building using configure.bat -W (with WebReady)

Revision 4672
Added by Robin Mills 9 months ago

#1230 Fixed significant comment typo in msvc\configure.py

Revision 4673
Added by Robin Mills 9 months ago

#1230 Fix svn:eol property in msvc\configure.py

Revision 4674
Added by Robin Mills 9 months ago

#1230 Adding build/test performance information to msvc/ReadMe.txt

Revision 4686
Added by Robin Mills 8 months ago

#1230 Removing incorrect 2-byte pad following JPEG ICC_PROFILE.

History

#1 Updated by Robin Mills 10 months ago

r4552 Test suite review and maintenance. In particular, get rid of test/tmp/voodoo.out-stripped files. Update changes with $ cp test/tmp/voodoo.out test/data

#2 Updated by Robin Mills 10 months ago

r4553 Buildserver maintenance. Get rid of \r in dist/logs/build.log and dist/logs/test.log

This also required changes in Jenkins to use tr -d '$\r' on test-svn-SVN-date-YYYY-MM-DD+HH-mm-ss.txt

#3 Updated by Robin Mills 10 months ago

  • % Done changed from 10 to 100
  • Estimated time changed from 20.00 to 4.00

r4554 Buildserver maintenance. Documentation Upate. Simplified script names. Almost all code moved from Jenkins into SVN.

I'm going to keep the Done of this issue at 100 and bump the Estimate time when I discover or modify anything. This keeps Redmine % done at 97+% and accounts for the effort I have put into this.

#4 Updated by Robin Mills 10 months ago

  • Estimated time changed from 4.00 to 6.00

Applied a lot of polish and simplification to the buildserver scripts.

#5 Updated by Robin Mills 10 months ago

  • Estimated time changed from 6.00 to 12.00

r4564 Added test/stdin-test. Fixed isse in with TiffImage.doWriteMetadata not respecting Image.writeXmpFromPacket()

All the buildserver maintenance stuff has been painfully slow. 5 minutes to change a script. 1.5 hours to run a build! It seems to be working OK now.

#6 Updated by Robin Mills 10 months ago

  • Estimated time changed from 12.00 to 16.00

r4565. Extended stdin/stdout support to enable ICC and exv file structures to be piped.

Here are matters that I intend to investigate:
1 Build/test exiv2 on a bigEndian machine
2 Validate msvc2005/configure.py works with the trunk code
3 Some of the test suite (imagetest, crwtest, iotest, iptctest are broken on MinGW/32)
4 Known bugs #1011 and #1023
5 trunk builds broken on cygwin and mingw, yet build fine manually
6 I'll think of other matters to investigate !

#7 Updated by Robin Mills 10 months ago

  • Estimated time changed from 16.00 to 20.00

Tested on MacOS-X 10.4 PPC bigEndian computer.

Only failure is:

$ exiv2 -pp test/data/imagemagick.pgf
The image doesn't even open on the PPC!
$ 501 /Users/rmills $ cd ~/gnu/exiv2/trunk
502 /Users/rmills/gnu/exiv2/trunk $ exiv2 -pa test/data/imagemagick.pgf
Exiv2 exception in print action for file test/data/imagemagick.pgf:
Failed to read input data
503 /Users/rmills/gnu/exiv2/trunk $ exiv2 -vV -g svn -g platform -g bits -g compiler
exiv2 0.25 001900 (32 bit build)
platform=apple
compiler=G++
bits=32
svn=4571
504 /Users/rmills/gnu/exiv2/trunk $ 
Clearly an endian issue with src/pgfimage.cpp I've recorded an issue #1234.

This is my old Adobe laptop from 2003! My first Mac Laptop. I've had 4 new MacBook Pro laptops since then. It's clunky and slow. However it's a very good test that our code builds on elderly UNIX, bigEndian and 32bit hardware. Worth the effort.

#8 Updated by Robin Mills 10 months ago

I'm rather pleased to both discover and fix #1234 (pgfimage.cpp on bigEndian hardware). The test suite has a dependency on the utility md5sum in coreutils. The latest coreutils (8.23) does not build on Tiger (MacOS-X 10.4.11 using GCC 4.0.1). Exiv2 does build on that platform.

I build and installed coreutils-5.2.1 which was released when Tiger was the current MacOS-X delivery.

coreutils hangs in ./configure which trying to crash cd or pwd with very long path names. I'm not going to report this to coreutils. I will enjoy knowing that Exiv2 is solid code with excellant build and test infrastructure.

#9 Updated by Robin Mills 10 months ago

  • Estimated time changed from 20.00 to 24.00

Various minor things. See SVN log.

#10 Updated by Robin Mills 10 months ago

  • Estimated time changed from 24.00 to 28.00

I'm trying again to debug the MinGW/32 CMake build. I solved this for libssh and that has given me the courage and confidence to work on this. Here's what I did with libssh:

I know what’s wrong with this. I’ve built the DLL (shared library). When I explain, you should be able to build the static library.

[ 94%] Building C object examples/CMakeFiles/exec.dir/exec.c.obj
[ 95%] Building C object examples/CMakeFiles/exec.dir/authentication.c.obj
[ 97%] Building C object examples/CMakeFiles/exec.dir/knownhosts.c.obj
[ 98%] Building C object examples/CMakeFiles/exec.dir/connect_ssh.c.obj
[100%] Linking C executable exec.exe
[100%] Built target exec
573 -32- /home/rmills/gnu/libssh/libssh-0.7.3/build> find . -name "*.a" -exec ls -alt {} \;
-rw-r--r-- 1 rmills root 11750 Sep 30 12:42 ./examples/CMakeFiles/exec.dir/objects.a
-rw-r--r-- 1 rmills root 11012 Sep 30 12:42 ./examples/CMakeFiles/libsshpp.dir/objects.a
-rw-r--r-- 1 rmills root 5990 Sep 30 12:42 ./examples/CMakeFiles/libsshpp_noexcept.dir/objects.a
-rw-r--r-- 1 rmills root 11792 Sep 30 12:42 ./examples/CMakeFiles/senddata.dir/objects.a
-rw-r--r-- 1 rmills root 651698 Sep 30 12:42 ./src/CMakeFiles/ssh_shared.dir/objects.a
-rw-r--r-- 1 rmills root 241366 Sep 30 12:42 ./src/libssh.dll.a                                            <-- GOOD
-rw-r--r-- 1 rmills root 2122 Sep 30 12:42 ./src/threads/CMakeFiles/ssh_threads_shared.dir/objects.a
-rw-r--r-- 1 rmills root 1538 Sep 30 12:42 ./src/threads/libssh_threads.dll.a                              <-- GOOD
574 -32- /home/rmills/gnu/libssh/libssh-0.7.3/build> find . -name "*.dll" -exec ls -alt {} \;
-rwxr-xr-x 1 rmills root 2291514 Sep 30 12:42 ./src/libssh.dll                                             <-- GOOD
-rwxr-xr-x 1 rmills root 41701 Sep 30 12:42 ./src/threads/libssh_threads.dll                               <-- GOOD

CMake is generating the following wrong response files:

575 -32- /home/rmills/gnu/libssh/libssh-0.7.3/build> find . -name "*.rsp" -exec ls -alt {} \;
-rw-r--r-- 1 rmills root 425 Sep 30 12:41 ./examples/CMakeFiles/exec.dir/includes_C.rsp                    <-- WRONG
-rw-r--r-- 1 rmills root 187 Sep 30 12:36 ./examples/CMakeFiles/exec.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 148 Sep 30 12:36 ./examples/CMakeFiles/exec.dir/objects1.rsp
-rw-r--r-- 1 rmills root 264 Sep 30 12:36 ./examples/CMakeFiles/libsshpp.dir/includes_CXX.rsp              <-- WRONG   
-rw-r--r-- 1 rmills root 187 Sep 30 12:36 ./examples/CMakeFiles/libsshpp.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 42 Sep 30 12:36 ./examples/CMakeFiles/libsshpp.dir/objects1.rsp
-rw-r--r-- 1 rmills root 264 Sep 30 12:36 ./examples/CMakeFiles/libsshpp_noexcept.dir/includes_CXX.rsp     <-- WRONG
-rw-r--r-- 1 rmills root 187 Sep 30 12:36 ./examples/CMakeFiles/libsshpp_noexcept.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 60 Sep 30 12:36 ./examples/CMakeFiles/libsshpp_noexcept.dir/objects1.rsp
-rw-r--r-- 1 rmills root 425 Sep 30 12:41 ./examples/CMakeFiles/senddata.dir/includes_C.rsp                <-- WRONG
-rw-r--r-- 1 rmills root 187 Sep 30 12:36 ./examples/CMakeFiles/senddata.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 168 Sep 30 12:36 ./examples/CMakeFiles/senddata.dir/objects1.rsp
-rw-r--r-- 1 rmills root 425 Sep 30 12:41 ./src/CMakeFiles/ssh_shared.dir/includes_C.rsp                   <-- WRONG
-rw-r--r-- 1 rmills root 166 Sep 30 12:36 ./src/CMakeFiles/ssh_shared.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 2137 Sep 30 12:36 ./src/CMakeFiles/ssh_shared.dir/objects1.rsp
-rw-r--r-- 1 rmills root 425 Sep 30 12:40 ./src/threads/CMakeFiles/ssh_threads_shared.dir/includes_C.rsp   <-- WRONG
-rw-r--r-- 1 rmills root 182 Sep 30 12:36 ./src/threads/CMakeFiles/ssh_threads_shared.dir/linklibs.rsp
-rw-r--r-- 1 rmills root 49 Sep 30 12:36 ./src/threads/CMakeFiles/ssh_threads_shared.dir/objects1.rsp
576 -32- /home/rmills/gnu/libssh/libssh-0.7.3/build> 
The correct code (in each of the wrong files) on my system (to use the Qt 5.6 provided compiler is):
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/build/src/threads
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/src/threads
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/include
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/build
-IC:/MinGW/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3
-IC:/Qt/Qt5.6.0/Tools/mingw492_32/i686-w64-mingw32/include
-IC:/MinGW/msys/1.0/local/include 

I had to generate the build files with this command:

cmake .. -G "Unix Makefiles" -DCMAKE_C_COMPILER=$(which gcc) -DCMAKE_CXX_COMPILER=$(which g++)

I had to build and install openssl with the commands (in the openssl delivery):

$ ./config --prefix=/usr/local
$ make
$ make install

To build the static library, you’ll have to edit CMakeLists.txt. You may well find that the linker response files are not correct for static libraries.

//////////// Here are notes I made along the way to solving this ////////////////////////

I tried to build libssh this morning on MinGW/64. The experience was the same as MinGW/32.

1) I had to edit sdkddkver.h to build CMake

2) I had to bootstrap into —prefix=/usr/local (why is bootstrap not called configure?)

3) I had to use CMake with this command:

cmake .. -G "Unix Makefiles" -DCMAKE_C_COMPILER=$(which gcc) -DCMAKE_CXX_COMPILER=$(which g++)

4) Here’s what happens when I run make:

c:/MinGW64/msys/1.0/bin/cmake.exe -E cmake_progress_report ...
[  1%] Building C object src/CMakeFiles/ssh_shared.dir/auth.c.obj
cd C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/build/src && c:/TDM-GCC-64/bin/gcc.exe \
  -DLIBSSH_EXPORTS @CMakeFiles/ssh_shared.dir/includes_C.rsp   \
  -o CMakeFiles/ssh_shared.dir/auth.c.obj \
  -c C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/src/auth.c
In file included from c:/MinGW64/msys/1.0/include/sys/unistd.h:9:0,
                 from c:/MinGW64/msys/1.0/include/unistd.h:6,
                 from C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/include/libssh/libssh.h:59,
                 from C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/include/libssh/priv.h:138,
                 from C:/MinGW64/msys/1.0/home/rmills/gnu/libssh/libssh-0.7.3/src/auth.c:34:
c:/MinGW64/msys/1.0/include/sys/types.h:71:18: error: conflicting types for 'time_t'
 typedef _TIME_T_ time_t; 

5) Exiv2 does not build with CMake either. Dies a terrible death instantly down inside a stack of system include files involving pthread.

6) My notes about CMake are here:
svn://dev.exiv2.org/svn/trunk/contrib/buildserver/dailyCMake.sh

Some Exiv2 users love CMake. I do not. It works well on Linux/MacOS-X/Cygwin. It doesn’t seem to work on MinGW. It’s an obstructive monster with Visual Studio. Of the 5 platforms we support on Exiv2, MinGW is the one I like least. However we build and test Exiv2 on all 5 platform on every commit. We build and test all 5 platforms and 6 editions of Visual Studio in 32 and 64 bits every night.

One of the challenges of using a build system such as CMake is to fix the build when CMake make mistakes. Because there is an abstract layer (CMakeLists.txt) between the code and the compiler, when things go wrong you have to crawl through the generated magic. This is not fun. The Linux World’s attitude “It must be the fault of Windows, because it works on Linux” is very unhelpful. You’re unlikely to get any sympathy on an opensource Forum. It’s your lucky day to have met a helpful open-source contributor who is not Windows hostile.

I’m very pleased to have worked on this. I still haven’t got Exiv2 to build using CMake on MinGW/32. However, I’m confident that I’ll fix that over the weekend.

#11 Updated by Robin Mills 10 months ago

  • % Done changed from 100 to 50
  • Estimated time changed from 28.00 to 40.00

I'm removing the 8 hours in this issue that I have spent on CMake/MinGW. That time is being spend on 1236. I started the bug hunt last week. I'll spend another week on it. So 50% done.

#12 Updated by Robin Mills 10 months ago

  • % Done changed from 50 to 60

I'm put in an effort on contrib/buildserver/categorize.sh and rewritten in Python. Down from 25 minutes to 0.6 seconds. Much better.

#13 Updated by Robin Mills 10 months ago

  • % Done changed from 60 to 80

#14 Updated by Robin Mills 9 months ago

  • % Done changed from 80 to 90

#15 Updated by Robin Mills 9 months ago

  • % Done changed from 90 to 80

I'm dragging my feet on testing msvc2005/configure.py

#16 Updated by Robin Mills 9 months ago

  • % Done changed from 80 to 100

#17 Updated by Robin Mills 9 months ago

  • Status changed from Assigned to Closed

Also available in: Atom PDF

Redmine Appliance - Powered by TurnKey Linux