Exiv2

Charles

This is an interesting question and I don't know the answer! However, here's what I see on Windows:

The Release .lib is a little fat guy at 28mb and his debug brother is positively obese.

C:\gnu>treesize -p | grep exiv2.lib$
36999214 .\exiv2\msvc\bin\Debug\exiv2.lib
493978 .\exiv2\msvc\bin\DebugDLL\exiv2.lib
28228268 .\exiv2\msvc\bin\Release\exiv2.lib
493978 .\exiv2\msvc\bin\ReleaseDLL\exiv2.lib
The DLL .libs are the same size because they are 'stubs' for calling the DLLs.

--- === ---

The DLLs - 1mb for Release and 3 for debug. Much better.

C:\gnu>treesize -p | grep exiv2.dll$
2727936 .\exiv2\msvc\addmoddel\DebugDLL\exiv2.dll
966656 .\exiv2\msvc\addmoddel\ReleaseDLL\exiv2.dll

--- === ---

Now let's make a product from them. Release\exiv2.exe is a respectable 1.3mb and uses most of the code. And ReleaseDLL\exiv2.exe is a positive baby at 184k (of course he needs the 966k DLL)

C:\gnu>treesize -p | grep exiv2.exe
2924544 .\exiv2\msvc\bin\Debug\exiv2.exe
282624 .\exiv2\msvc\bin\DebugDLL\exiv2.exe
1294336 .\exiv2\msvc\bin\Release\exiv2.exe
184320 .\exiv2\msvc\bin\ReleaseDLL\exiv2.exe
2924544 .\exiv2\msvc\exiv2\Debug\exiv2.exe
282624 .\exiv2\msvc\exiv2\DebugDLL\exiv2.exe
1294336 .\exiv2\msvc\exiv2\Release\exiv2.exe
184320 .\exiv2\msvc\exiv2\ReleaseDLL\exiv2.exe

C:\gnu>

So this is kind of curious. I can only speculate that exiv2.lib has a lot of duplicated code (from templates?) which is eliminated when you link an executable. Just by way of comparison, in my day job, I'm working on a product which includes the CLAPACK libraries (built for Windows/CE)

03/25/2009 09:00 1,078,926 blas.lib
03/25/2009 09:00 5,994,772 clapack.lib
03/25/2009 09:00 1,027,516 libF77.lib
03/25/2009 09:00 502,138 libI77.lib
4 File(s) 8,603,352 bytes

Again, quite fat when you consider that our executable (including 150,000 lines of our own code and the SQLlite library) weighs in at 1.2mb. Libraries appear to contain a lot of fat. Of course, we don't use very much of CLAPACK - however I believe that exiv2.exe does execute most of the exiv2.lib code.

Just to give you some statistics, Andreas has written about 33000 lines of (beautiful) C++ and about 17000 lines of header files and templates. An awesome job for a weekend/evening effort. A round of applause for our friend Andreas.

Maybe another reader can enlighten us with an insight to the structure of libraries and executables. Clearly there's a lot of condensation from one to the other during the linking process.

Robin
(windows/msvc build contributor for exiv2)

I can't answer the original question either.

For comparison, on my Debian Linux system, the size of the shared library for version 0.18 is between 1.5MB and 2MB and the static library is about 3MB.
So while the Windows DLL is much smaller than the shared library on Linux, for the static libraries it's the other way around.

Since the static library is only used during development and the resulting executable has a reasonable size as Robin showed, I wouldn't be worried too much about the size of the static library. Much more about the size of the shared library/DLL. Why is the Windows DLL so much smaller than the Linux shared library? How can I make it smaller?

One thing to note is that Exiv2 makes extensive use of the C++ standard (template) library. Could it be that different handling of templates makes a significant difference in the size of the libraries?

Finally, Robin, the compliments go to the community. Significant portions of Exiv2, both in terms of functionality and number of lines of code, were written by other authors and contributed to the project - including the MSVC build environment. And it certainly wouldn't be what it is today without the invaluable and continued feedback from users.

Andreas

Yes, Andreas. You are right. Many folks have contributed to exiv2 and they deserve to share the applause.

I've done some messing with this. Probably a bad sign that I've nothing better to do on Friday evening. I wrote a "hello world" program to call a library with 20 .cpp files.

#include "MyLib.h"
int _tmain(int argc, _TCHAR* argv[]) {
printf("hello world\n") ;

function1() ;
function2() ;
// until
function19() ;
function20() ;
}

MyLib1.cpp (and his 19 mirrors)
void function1(void) {
string hello("hello from function 1") ;
cout << hello << endl ;
}

The results are rather interesting:

C:\Robin\Projects\LibrarySizeApp>treesize -p | grep lib$
596300 .\Debug\MyLib.lib (eh?, big isn't it)
9142060 .\Release\MyLib.lib (yes 9MB for 20x2 line programs)

C:\Robin\Projects\LibrarySizeApp>treesize -p | grep obj$
28925 .\MyLib\Debug\MyLib1.obj
28936 .\MyLib\Debug\MyLib10.obj
etc
455961 .\MyLib\Release\MyLib1.obj
455967 .\MyLib\Release\MyLib10.obj
etc

It's obvious that a lib is more or less a concatenation of the object files. You can see that the Release copies of MyLibN.obj are an amazing 455k each - so 20 of them weigh 9MB. Phew - thank goodness we have terra-byte disks! The Debug versions are 25k each, so 20 of them weigh 500k.

I believe UNIX uses ar to combine the object files into a library (to which ranlib can add a dictionary/lookup-table to speed up linking). And it looks like Windows does the same:
C:\Robin\Projects\LibrarySizeApp\Debug>dmpf MyLib.lib | head
0 !<arch>./ 21 3c 61 72 63 68 3e 0a 2f 20 20 20 20 20 20 20
16 12412425 20 20 20 20 20 20 20 20 31 32 34 31 32 34 32 35
32 15 31 35 20 20 20 20 20 20 20 20 20 20 20 20 20 20
48 0 3458 30 20 20 20 20 20 20 20 33 34 35 38 20 20 20 20
64 `....U........ 20 20 60 0a 00 00 00 55 00 00 1d 08 00 00 1d 08

Following linking, everything's fine. The Release version is smaller than the Debug. The Release .exe is is thin.
C:\Robin\Projects\LibrarySizeApp>treesize -p | grep exe$
44544 .\Debug\LibrarySizeApp.exe
12288 .\Release\LibrarySizeApp.exe

I haven't bothered to build a DLL version because I believe a DLL and an EXE are very similar beasts. A Windows/DLL is equipped with startup code to begin life at DllMain(). An EXE begins life at main() - however I believe the internal and external linking of a DLL and EXE are about the same. We saw this before when size of exiv2.exe+exiv2.dll (ReleaseDLL) = exiv2.exe (Release).

--- === ---
Why the release version should be 450k for 2 lines of code is mysterious. Maybe it has in-lined most of std::string and std::cout. So I redefined the library functions as:

#include "MyLib.h"
#include <stdio.h>
void function4(void) {
char* hello = "Hello from function 4" ;
printf("%s\n",hello) ;
}

And I got two interesting results:
1) I got the same results until I eliminated include <string> and include <iostream>.
2) When I removed <string> and <iostream>, I got:
C:\Robin\Projects\LibrarySizeApp>treesize . -p | grep lib$
114758 .\Debug\MyLib.lib
182164 .\Release\MyLib.lib

Sanity almost (Release is still fatter, however both are thin). Compiling <string> and <iostream> makes the .obj/.lib fat. Templates cause fat libraries!
--- === ---

And now to look at Andreas' points about the size of the libraries - I don't see much difference. Andreas is reporting 1.5 - 2.0mb on Linux. Windows is 1mb. On my Mac, the exiv2 application is 177k and the dylib is 1.8mb. It's not so very different and nowhere close to the 26mb mentioned by Charles.

So returning to Charles' original question "Why is the library around 26mb?", I think the answer is "MSVC's handling of templates results in fat libraries." Unix and the Mac are fine.

Charles - I expect you're sorry you asked this question. However I'm delighted because you have sparked my curiosity. In the office, I had noticed Release libraries are often bigger than debug versions and never had time to investigate. Thanks for asking about this.

Interesting analysis! Thanks for taking the trouble to investigate and report the results in great detail here, Robin.

Re shared libraries. After all this we find 26MB is large and 1-2MB is not, of course. But forget about the 26MB for a moment and look at it this way: The Linux shared lib is 1.5-2x or 50-100% larger than the Windows DLL. I still think this is a lot larger.

This library size issue is something I've been trying to get a hold of for a while. One of the benefits of the recently added visibility support (#523) was advertised as reducing the size of the library. It required considerable work and eventually shaved off only about 10% of the size (but of course there were other good reasons to do this). Stripping symbols from the library also has a noticeable effect but has its disadvantages too. (That's BTW where the 1.5-2MB range comes from: different combinations of visibility support and symbol stripping.) I've further experimented with the object files and done some statistics but have yet to find other ways to reduce the library size with reasonable code re-work.

Andreas

Andreas

I suspect that MSVC builds are smaller than GCC equivalents. Here's the evidence:

Adobe Reader 9 (comparing AGM/BIB/CoolType and ACE). Even allowing that Adobe's Mac/Universal builds are 2x (they contain both PPC and i386 code), you can see that the Windows versions are consistently smaller. I also compiled httpget.exe for i386 from source which uses STL: http://www.alhem.net/project/example6/index.html

Windows                             Mac                   Linux (Acro8)
 5,712,384 AGM.dll              12,656,744  Universal        7,873,500
   102,400 BIB.dll                 348,076  Universal          294,224
 2,433,024 CoolType.dll          6,649,872  Universal        3,754,460
   785,920 ACE.dll               1,690,152  Universal          863,144

641,536  debug\httpget.exe       692,300  i386 binary (not universal)
  125,952  release\httpget.exe     319,540  i386 binary (not universal)

I could only find Adobe 8 for Linux and it's very likely that 9 for Linux is fatter as it contains more functionality. I didn't build httpget for Linux, however I think the trend is clear.

However my "killer" argument is:

576 /Y/gnu/exiv2/msvc/bin/Release> dir exiv2.exe
rwx----- 1 rmills staff 1,294,336 Mar 31 21:46 exiv2.exe (exiv2 for windows static release built with MSVC)

580 /Users/rmills/Downloads/exiv2-0> finder "*.exe" -exec ls -alt {} ";"
-rwxr-xr-x@ 1 rmills staff 4,226,547 Apr 6 00:11 ./exiv2.exe (exiv2 for windows static release built with MinGW)

MSVC executables/DLLs are more compact than GCC equivalents for unknown reasons. The ratio seems to be of the order of 1.5x to 2.5x - as you are experiencing in exiv2. However your efforts to reduce library size are a very worthwhile goal quite independant of the tools being used. It might be interesting to see how the Borland and Intel toolkits measure up. Or there again, maybe it's not interesting :-)

Robin

Are you take a look if binary generated by MSVC is not compressed using zlib equivalent method ?

Gilles Caulier

Hi Gilles

I don't believe Windows executables (the so called PE format) are compressed. The dumpbin.exe utility can dump the various sections of the binary with ease. I've written several utilities that have traversed parts of the executable (for mysterious reasons) and I don't recall it being compressed.

I think the mechanism to execute a DLL/EXE is to memory map the file (as copy on write) and off it goes! There could be some kind of run-length compression of the code itself, I suppose, however I don't believe there is. I can say with 100% certainty that strings from your code (char* message = "hello world";) are NOT compressed - utilities such as "strings.exe" are knuckle-headed scanners! (I've written one of those as well). If you use od.exe (octal dump), you'll see the strings in the raw.

Robin

drag an image into an image resizer vb.nethttp://www.rasteredge.com/how-to/vb-net-imaging/resize-image/ and click the option in the sotware. this is a simple way to make image size smaller.

The image being discussed here is the size of the executable file on disk. Software folks refer to .exe and .dll files as "images". In this thread we are discussing software images and not graphical images.

So you guys are just talk about .exe and .dll files as "images"? But I am quite interesting to the graphical images resizing processing toolkits. And I am testing the following plugins to resize images these days?
Image Resizing on VB.NET: http://www.pqscan.com/convert-pdf/to-image-size-vb.html
Image Resizing on C#.NET: http://www.pqscan.com/convert-pdf/to-image-size-csharp.html
Any other suggestion will also be appreciated. Thanks in advance.