Bug #627

typeId methods not available, -fvisibility-inlines-hidden

Added by Rex Dieter over 8 years ago. Updated over 7 years ago.

Status:ClosedStart date:17 Apr 2009
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:build
Target version:0.19

Description

See also downstream report of pyexiv2 failures and nonexported/hidden ...::typeId methods due to exiv2's use of -fvisibility-inlines-hidden.

Was this intentional or oversight?

If oversight, attaching a patch dropping the use of -fvisibility-inlines-hidden

exiv2-0.18.1-visibility.patch Magnifier - patch to use only -fvisibility=hidden. you're likely only interested in the configure.ac bits (1.34 KB) Rex Dieter, 17 Apr 2009 13:08


Related issues

Related to Exiv2 - Feature #523: C++ symbol visibility support for gcc builds Closed
Related to Exiv2 - Bug #629: Virtual functions should not be inlined Closed 30 Apr 2009

History

#1 Updated by Andreas Huggel over 8 years ago

See also downstream report of pyexiv2 failures and nonexported/hidden ...::typeId methods due to exiv2's use of -fvisibility-inlines-hidden.

Ok, seen this: https://bugzilla.redhat.com/show_bug.cgi?id=496050

Was this intentional or oversight?

Public methods are supposed to be usable by applications, whether inlined or not.
It seems the way that pyexiv2/python access the library is different from how a "normal" application accesses a shared library - there are tons of inlined functions and that compiler switch has been there since 0.18-pre1, and yet the problem has not been reported until now.
Put differently, isn't that function compiled into pyexiv2 itself where needed (since it's inlined), why does python look for it in the library?
Or could it possibly be a distribution specific issue? IIRC there is also a Debian version of the new pyexiv2 - but I've not heard of that not working yet. (Can't try right now.)

If oversight, attaching a patch dropping the use of -fvisibility-inlines-hidden

Thanks for the patch. I think there is still some investigation required to decide what to do with this.

Andreas

#2 Updated by Andreas Huggel over 8 years ago

  • Target version deleted (0.18.1)

#3 Updated by Andreas Huggel over 8 years ago

Rex, any new insights into this?

On my Debian testing system, with exiv2 0.18.1 and pyexiv2 0.1.3 compiled from source (and boost 1.37 from source, the rest from the distro), I can not reproduce this problem, pyexiv2 seems to work fine. Exiv2 was built with just ./configure; make; make install.

Andreas

#4 Updated by Rex Dieter over 8 years ago

No news other than my ability to reproduce the problem on both Fedora 10 with gcc-4.3 and Fedora 11 with gcc-4.4. Maybe a visibility bug with gcc? I'll see if I can narrow this down to a smaller test-case.

#5 Updated by Rex Dieter over 8 years ago

Reading gcc.info docs on -fvisibility-inlines-hidden makes me thing that fedora's gcc here is doing per documented behavior:
You may mark a method as having a visibility explicitly to negate
the effect of the switch for that method. For example, if you do
want to compare pointers to a particular inline method, you might
mark it as having default visibility. Marking the enclosing class
with explicit visibility will have no effect.

The last line sums up my take here, that if you want to use -fvisibility-inlines-hidden with inlined class items, you need to explictly mark them visible (which isn't done in exiv2 source, as far as I can tell).

#6 Updated by Andreas Huggel over 8 years ago

if you want to use -fvisibility-inlines-hidden with inlined class items, you need to explictly mark them visible

Only if I want to compare pointers to a particular inline method, as the documentation says. But that's not something I'm interested in and neither is pyexiv2 doing this.

This explanation from http://gcc.gnu.org/wiki/Visibility says it better:

"Lastly, there's one other new command line switch: -fvisibility-inlines-hidden. This causes all inlined class member functions to have hidden visibility, causing significant export symbol table size & binary size reductions though not as much as using -fvisibility=hidden. However, -fvisibility-inlines-hidden can be used with no source alterations, unless you need to override it for inlines where address identity is important either for the function itself or any function local static data."

In order to find out why it works here but not on Fedora, here some info about the symbols I have in my libraries - 0.18 with and without visibility support: Exifdatum::typeId is present in both cases.

ahuggel@mowgli> readelf -W -s libexiv2.so.5.2.0-noviz | grep -i exifdatum.*typeid
   531: 00083a60    23 FUNC    WEAK   DEFAULT   11 _ZNK5Exiv29Exifdatum6typeIdEv
  1862: 00083a60    23 FUNC    WEAK   DEFAULT   11 _ZNK5Exiv29Exifdatum6typeIdEv
ahuggel@mowgli> readelf -W -s libexiv2.so.5.2.0-viz | grep -i exifdatum.*typeid
  1386: 00057ec0    23 FUNC    LOCAL  HIDDEN   11 _ZNK5Exiv29Exifdatum6typeIdEv

Comments 2 and 3 in the downstream bugreport say that there is no such symbol, not even a hidden one, in the Exiv2 library from the RPM, but a hidden symbol is present if the library is compiled locally (on a Fedora system I presume), consistent with what I have here.

That would be a real difference to what I get here. Could that cause the issue? And why should the symbol be stripped from the RPM library?

#7 Updated by Rex Dieter over 8 years ago

Thanks for the clarification, and corrections for my botched interpretations. With your hints, I'll do some more investigating.

#8 Updated by Rex Dieter over 8 years ago

Looks like fedora's pyexiv2 maintainer patched it to build with the distro optimization flags, -O2 in particular, and this makes pyexiv2 fail to have/find the symbol. Building pyexiv2 with -O0 works.

I can confirm that our (unpatched) exiv2 builds do include the _ZNK5Exiv29Exifdatum6typeIdEv symbol you mentioned, so by all accounts, so far, this indeed looks to be not an issue here in exiv2, but elsewhere (in pyexiv2 or gcc).

Would be interesting if you could reproduce the problem (or not) building pyexiv2 with optimization > -O0

#9 Updated by Rex Dieter over 8 years ago

OK, pardon the spam, but now I'm thoroughly confusing msyelf. You mention in comment #6:
1386: 00057ec0 23 FUNC LOCAL HIDDEN 11 _ZNK5Exiv29Exifdatum6typeIdEv
But shouldn't that symbol not be hidden, since it's part of a non-hidden/exported class?

#10 Updated by Andreas Huggel over 8 years ago

Indeed, I can reproduce the problem with pyexiv2 built with -O2.

If I then make Exifdatum::typeId and Iptcdatum::typeId non-inlined members and keep the -fvisibility-inlines-hidden flag, pyexiv2 (-O2) works again. Note that these are virtual functions (implementing Metadatum::typeId).

Re comment #9: The symbol is hidden because of -fvisibility-inlines-hidden. (As you may notice, I'm trying to navigate around a discussion of what that flag is actually good for and why it does what we observe... ;)

Bottomline for me

  • The use of -fvisibility-inlines-hidden in exiv2 is debatable as it doesn't make much of a difference (library size, number of symbols). On the other hand, I don't think it is the root cause of this problem. I don't plan to remove it.
  • The thing to change in exiv2 is that virtual functions should not be inlined. I'll open another issue for this. (I seem to remember obscure issues with such cases in the past.) But as no compiler actually complains about this, I don't see it as the root cause of this issue either. Maybe a gcc person could explain what is really happening here.

With this, I'm closing this issue ("won't fix"), feel free to reopen it if you disagree.

#11 Updated by Andreas Huggel over 8 years ago

  • Status changed from New to Resolved

#12 Updated by Andreas Huggel over 8 years ago

  • Status changed from Resolved to Closed

#13 Updated by Rex Dieter about 8 years ago

Not sure how to re-open, but my own common sense tells me ignoring the issue isn't going to fix it. I've just recently confirmed that 0.18.2 has the same problem.

Here's a recent comment in the downstream bug that may shed some additional light,
https://bugzilla.redhat.com/show_bug.cgi?id=496050#c26
------------------------------------
Hmmm... some observations.

  • Just tested exiv2-0.18.2-2.fc11 and it works around the problem.
    The import succeeds with it and fails with original 0.18.2-1.f11
  • I think that the different behavior with -O0 is related to the
    fact that g++ does NOT inline without optimization turned on.
    (look for -fno-default-inline in gcc info docs)
  • As a result, with O0 we simply force g++ to compile a copy
    of the function into the object file (no inline) and therefore
    the -fvisibility-inlines-hidden does not affect it anymore.
    Problem "solved" ;
    )
  • It looks like an upstream bug to me:
    - If Exifdatum::typeId() is an officially exported API, they should
    have marked it like their other methods with EXIV2API macro
    - This macro is expanded to EXV_EXPORT in the exv_conf.h header
    - The EXV_EXPORT sets the visibility attribute to "default"
    in the same header. This would make it visible again.

#14 Updated by Andreas Huggel about 8 years ago

  • Status changed from Closed to Feedback

Not sure how to re-open

Setting status to "Feedback" now. Need additional explanations for the proposed fix.

but my own common sense tells me ignoring the issue isn't going to fix it. I've just recently
confirmed that 0.18.2 has the same problem.

See note 10 above. Bug #629 was opened and resolved. But it touches the API so that change didn't make it into 0.18.2. It will be in 0.19.

Here's a recent comment in the downstream bug that may shed some additional light,
https://bugzilla.redhat.com/show_bug.cgi?id=496050#c26

[...]

  • It looks like an upstream bug to me:
    - If Exifdatum::typeId() is an officially exported API, they should
    have marked it like their other methods with EXIV2API macro

By convention, the whole class is marked with EXIV2API, not individual methods. Why is there a need to mark the method itself in this case?

#15 Updated by Rex Dieter about 8 years ago

OK, thanks for the clarification.

I'm happy with limping along with the current hackish workaround, until 0.19 lands.

#16 Updated by Andreas Huggel over 7 years ago

  • Status changed from Feedback to Resolved
  • Target version set to 0.19
  • % Done changed from 0 to 100

Fixed in #629

#17 Updated by Andreas Huggel over 7 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF

Redmine Appliance - Powered by TurnKey Linux