Exiv2

See also downstream report of pyexiv2 failures and nonexported/hidden ...::typeId methods due to exiv2's use of -fvisibility-inlines-hidden.

Ok, seen this: https://bugzilla.redhat.com/show_bug.cgi?id=496050

Was this intentional or oversight?

Public methods are supposed to be usable by applications, whether inlined or not.
It seems the way that pyexiv2/python access the library is different from how a "normal" application accesses a shared library - there are tons of inlined functions and that compiler switch has been there since 0.18-pre1, and yet the problem has not been reported until now.
Put differently, isn't that function compiled into pyexiv2 itself where needed (since it's inlined), why does python look for it in the library?
Or could it possibly be a distribution specific issue? IIRC there is also a Debian version of the new pyexiv2 - but I've not heard of that not working yet. (Can't try right now.)

If oversight, attaching a patch dropping the use of -fvisibility-inlines-hidden

Thanks for the patch. I think there is still some investigation required to decide what to do with this.

Andreas

Rex, any new insights into this?

On my Debian testing system, with exiv2 0.18.1 and pyexiv2 0.1.3 compiled from source (and boost 1.37 from source, the rest from the distro), I can not reproduce this problem, pyexiv2 seems to work fine. Exiv2 was built with just ./configure; make; make install.

Andreas

No news other than my ability to reproduce the problem on both Fedora 10 with gcc-4.3 and Fedora 11 with gcc-4.4. Maybe a visibility bug with gcc? I'll see if I can narrow this down to a smaller test-case.

Reading gcc.info docs on -fvisibility-inlines-hidden makes me thing that fedora's gcc here is doing per documented behavior:
You may mark a method as having a visibility explicitly to negate
the effect of the switch for that method. For example, if you do
want to compare pointers to a particular inline method, you might
mark it as having default visibility. Marking the enclosing class
with explicit visibility will have no effect.

The last line sums up my take here, that if you want to use -fvisibility-inlines-hidden with inlined class items, you need to explictly mark them visible (which isn't done in exiv2 source, as far as I can tell).

if you want to use -fvisibility-inlines-hidden with inlined class items, you need to explictly mark them visible

Only if I want to compare pointers to a particular inline method, as the documentation says. But that's not something I'm interested in and neither is pyexiv2 doing this.

This explanation from http://gcc.gnu.org/wiki/Visibility says it better:

"Lastly, there's one other new command line switch: -fvisibility-inlines-hidden. This causes all inlined class member functions to have hidden visibility, causing significant export symbol table size & binary size reductions though not as much as using -fvisibility=hidden. However, -fvisibility-inlines-hidden can be used with no source alterations, unless you need to override it for inlines where address identity is important either for the function itself or any function local static data."

In order to find out why it works here but not on Fedora, here some info about the symbols I have in my libraries - 0.18 with and without visibility support: Exifdatum::typeId is present in both cases.

ahuggel@mowgli> readelf -W -s libexiv2.so.5.2.0-noviz | grep -i exifdatum.*typeid
   531: 00083a60    23 FUNC    WEAK   DEFAULT   11 _ZNK5Exiv29Exifdatum6typeIdEv
  1862: 00083a60    23 FUNC    WEAK   DEFAULT   11 _ZNK5Exiv29Exifdatum6typeIdEv
ahuggel@mowgli> readelf -W -s libexiv2.so.5.2.0-viz | grep -i exifdatum.*typeid
  1386: 00057ec0    23 FUNC    LOCAL  HIDDEN   11 _ZNK5Exiv29Exifdatum6typeIdEv

Comments 2 and 3 in the downstream bugreport say that there is no such symbol, not even a hidden one, in the Exiv2 library from the RPM, but a hidden symbol is present if the library is compiled locally (on a Fedora system I presume), consistent with what I have here.

That would be a real difference to what I get here. Could that cause the issue? And why should the symbol be stripped from the RPM library?

Thanks for the clarification, and corrections for my botched interpretations. With your hints, I'll do some more investigating.

Looks like fedora's pyexiv2 maintainer patched it to build with the distro optimization flags, -O2 in particular, and this makes pyexiv2 fail to have/find the symbol. Building pyexiv2 with -O0 works.

I can confirm that our (unpatched) exiv2 builds do include the _ZNK5Exiv29Exifdatum6typeIdEv symbol you mentioned, so by all accounts, so far, this indeed looks to be not an issue here in exiv2, but elsewhere (in pyexiv2 or gcc).

Would be interesting if you could reproduce the problem (or not) building pyexiv2 with optimization > -O0

OK, pardon the spam, but now I'm thoroughly confusing msyelf. You mention in comment #6:
1386: 00057ec0 23 FUNC LOCAL HIDDEN 11 _ZNK5Exiv29Exifdatum6typeIdEv
But shouldn't that symbol not be hidden, since it's part of a non-hidden/exported class?

Indeed, I can reproduce the problem with pyexiv2 built with -O2.

If I then make Exifdatum::typeId and Iptcdatum::typeId non-inlined members and keep the -fvisibility-inlines-hidden flag, pyexiv2 (-O2) works again. Note that these are virtual functions (implementing Metadatum::typeId).

Re comment #9: The symbol is hidden because of -fvisibility-inlines-hidden. (As you may notice, I'm trying to navigate around a discussion of what that flag is actually good for and why it does what we observe... ;)

Bottomline for me

• The use of -fvisibility-inlines-hidden in exiv2 is debatable as it doesn't make much of a difference (library size, number of symbols). On the other hand, I don't think it is the root cause of this problem. I don't plan to remove it.
• The thing to change in exiv2 is that virtual functions should not be inlined. I'll open another issue for this. (I seem to remember obscure issues with such cases in the past.) But as no compiler actually complains about this, I don't see it as the root cause of this issue either. Maybe a gcc person could explain what is really happening here.

With this, I'm closing this issue ("won't fix"), feel free to reopen it if you disagree.

Not sure how to re-open, but my own common sense tells me ignoring the issue isn't going to fix it. I've just recently confirmed that 0.18.2 has the same problem.

Here's a recent comment in the downstream bug that may shed some additional light,
https://bugzilla.redhat.com/show_bug.cgi?id=496050#c26
------------------------------------
Hmmm... some observations.

• Just tested exiv2-0.18.2-2.fc11 and it works around the problem.
  The import succeeds with it and fails with original 0.18.2-1.f11

• I think that the different behavior with -O0 is related to the
  fact that g++ does NOT inline without optimization turned on.
  (look for -fno-default-inline in gcc info docs)

• As a result, with O0 we simply force g++ to compile a copy
  of the function into the object file (no inline) and therefore
  the -fvisibility-inlines-hidden does not affect it anymore.
  Problem "solved" ;)

• It looks like an upstream bug to me:
  - If Exifdatum::typeId() is an officially exported API, they should
  have marked it like their other methods with EXIV2API macro
  - This macro is expanded to EXV_EXPORT in the exv_conf.h header
  - The EXV_EXPORT sets the visibility attribute to "default"
  in the same header. This would make it visible again.

OK, thanks for the clarification.

I'm happy with limping along with the current hackish workaround, until 0.19 lands.