Bug #1228
Crash in Windows when reading metadata in parallel
0%
Description
I've encountered a crash while reading metadata from a bunch of .jpg images (13 items)
Windows 10 x64, exiv2 0.25
I use 1 thread per image for debugging purposes of my own code.
Crashed thread: #41 (full stacktrace of all threads is in the .txt attachment.
Would be great to know is it known/fixed.
Locals
childName 0x234e9a82380 "xmp:Rating" char *
childOptions 0 unsigned long
isArrayItem false bool
isTopLevel true bool
isValueNode false bool
newChild 0xcccccccccccccccc XMP_Node *
value 0x7ff8ed1b7625 "" char *
xmlNode 0x234e9a82340 XML_Node *
xmpParent 0x234e97d3a90 XMP_Node *
- 41 Id: 3b00.3e7c Suspend: 1 Teb: 00000091`afeef000 Unfrozen "QThread"
Child-SP RetAddr Call Site
00000091`b2efc690 00007ff9`23e481c0 KERNELBASE!RaiseException+0x68
00000091`b2efc770 00007ff8`ed02b167 VCRUNTIME140D!_CxxThrowException(void * pExceptionObject = 0x00000091`b2efc8d0, struct s_ThrowInfo * pThrowInfo = 0x00007ff8`ed223f88)+0x130 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp136]405]
00000091`b2efc820 00007ff8`ed029f3f libexiv2!AddChildNode(class XMP_Node * xmpParent = 0x000001db`ce4f1ce0, class XML_Node * xmlNode = 0x000001db`ce3d7f20, char * value = 0x00007ff8`ed1b7625 "", bool isTopLevel = true)+0x277 [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\xmpsdk\src\parserdf.cpp
00000091`b2efc9c0 00007ff8`ed02983e libexiv2!RDF_LiteralPropertyElement(class XMP_Node * xmpParent = 0x000001db`ce46e640, class XML_Node * xmlNode = 0x000001db`ce3d7f20, bool isTopLevel = true)+0x5f [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\xmpsdk\src\parserdf.cpp1015]915]
00000091`b2efcb10 00007ff8`ed0293f3 libexiv2!RDF_PropertyElement(class XMP_Node * xmpParent = 0x000001db`ce46e640, class XML_Node * xmlNode = 0x000001db`ce3d7f20, bool isTopLevel = true)+0x40e [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\xmpsdk\src\parserdf.cpp
00000091`b2efcc30 00007ff8`ed028fcc libexiv2!RDF_PropertyElementList(class XMP_Node * xmpParent = 0x000001db`ce46e640, class XML_Node * xmlParent = 0x000001db`ce394770, bool isTopLevel = true)+0x113 [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\xmpsdk\src\parserdf.cpp796]707]
00000091`b2efccd0 00007ff8`ed028eaf libexiv2!RDF_NodeElement(class XMP_Node * xmpParent = 0x000001db`ce46e640, class XML_Node * xmlNode = 0x000001db`ce394770, bool isTopLevel = true)+0xdc [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\xmpsdk\src\parserdf.cpp
00000091`b2efcd30 00007ff8`ed028dc6 libexiv2!RDF_NodeElementList(class XMP_Node * xmpParent = 0x000001db`ce46e640, class XML_Node * xmlParent = 0x000001db`ce484460, bool isTopLevel = true)+0xcf [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\xmpsdk\src\parserdf.cpp672]651]
00000091`b2efcdc0 00007ff8`ed028d43 libexiv2!RDF_RDF(class XMP_Node * xmpTree = 0x000001db`ce46e640, class XML_Node * xmlNode = 0x000001db`ce484460)+0x76 [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\xmpsdk\src\parserdf.cpp
00000091`b2efce00 00007ff8`ed00314d libexiv2!ProcessRDF(class XMP_Node * xmpTree = 0x000001db`ce46e640, class XML_Node * rdfNode = 0x000001db`ce484460, unsigned long options = 0)+0x43 [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\xmpsdk\src\parserdf.cpp629]492]
00000091`b2efce30 00007ff8`ecfcb755 libexiv2!XMPMeta::ParseFromBuffer(char * buffer = 0x000001db`ce6e8090 " 1256]
00000091`b2efd000 00007ff8`ecfbf320 libexiv2!WXMPMeta_ParseFromBuffer_1(struct XMPMeta * xmpRef = 0x000001db`ce46e630, char * buffer = 0x000001db`ce6e8090 " 1274]
00000091`b2efd0a0 00007ff8`ecfb99e5 libexiv2!TXMPMeta<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >::ParseFromBuffer(char * buffer = 0x000001db`ce6e8090 " 903]
00000091`b2efd130 00007ff8`ecfb0f1d libexiv2!TXMPMeta<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >::TXMPMeta<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >(char * buffer = 0x000001db`ce6e8090 " 169]
00000091`b2efd170 00007ff8`ecec8c12 libexiv2!Exiv2::XmpParser::decode(class Exiv2::XmpData * xmpData = 0x000001db`ce35e458, class std::basic_string<char,std::char_traits<char>,std::allocator<char> > * xmpPacket = 0x000001db`ce35e4a0)+0x16d [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\src\xmp.cpp
00000091`b2efdf60 00007ff7`e6985e8a libexiv2!Exiv2::JpegBase::readMetadata(void)+0x942 [c:\users\takushni\projects\other\xpiks-deps\exiv2-0.25\src\jpgimage.cpp @ 390]
Files
Related issues
History
Updated by Robin Mills about 5 years ago
- Status changed from New to Assigned
- % Done changed from 0 to 50
- Estimated time set to 2.00 h
Good to hear from you, Taras. I hope you are well.
I think this is a known issue. XMPsdk is not totally thread safe. You should explicitly call Exiv2::XmpParser::initialize() before spinning up threads. In the investigation of #1207, the user (digiKam) thought that your rwlock was essential to get their multithreading to work. I added a new sample app samples/mt-test.cpp which you may find useful for testing v0.25.
I'm in the end game of v0.26. I am feature complete and investigating 2 remaining known bugs. I believe your rwlock code is working fine on all platforms.
I hope calling Exiv2::XmpParser::initialize() fixes this matter and I can close this issue. If this issue requires more investigation, it will have to wait as I'm already working about 12 hours/day on the release.
Updated by Taras Kushnir about 5 years ago
I'm calling Exiv2::XmpParser::initialize() in the beginning. Threads who are reading metadata (and using exiv2) are being spawned way more later. So it's not the case.
P.S. Yes, I'm following your marathon to 0.26 and looking forward to seeing it. Great job anyway!
Updated by Robin Mills about 5 years ago
- Target version changed from 0.26 to 0.28
- % Done changed from 50 to 0
- Estimated time deleted (
2.00 h)
I'm going change the target to 0.27. Multi-threading is a complex subject. Can I encourage you to play with mt-test.cpp on trunk/v0.26 to see if you can reproduce your situation and trouble.
If you don't make progress, let's talk on Skype mid-October and we'll make progress together.
Updated by Robin Mills about 5 years ago
Niels stepped into #1222 and #1227 For sure, there's something nasty happening and it's going to take some time to discover what's at the bottom of this.
When I investigated #1207 for Gilles on MacOS-X, he asked me to do "a lot more threads than 3". So I did. I got messages from the OS that I've never seen before about "resource not available". I subsequently learnt that MacOS-X has limits of about 256 open files per process. Those limits can be changed by the user, however that is not a useful fix. It's important that multi-threaded application control/limit the number of threads opening files. I think we'll learn that Windows requires similar treatment.
The Windows platform is also challenged by invisible processes such as Virus Checkers, Dropbox and other agents who may be locking files. We are not alone. Something's going on.
I have no time at the moment to investigate this. However I think mt-test.cpp could be a useful framework from which to discover the truth.
Updated by Taras Kushnir about 5 years ago
Robin,
Next week I'll take a look at mt-test.cpp - I'll try to create you small reproducible crash with particular files and particular code so it will be easier for you to debug later.
Updated by Taras Kushnir about 5 years ago
- File exiv2_bug_qt.zip exiv2_bug_qt.zip added
Hi Robin
Sorry for the delay. Actually I tried to repro the crash that next week as I promised, but I failed to repro using just plain C++. My try was here https://gist.github.com/Ribtoks/6bc894e20946073317a339ba17a06464
After that I tried to repro using same code in Qt and it is 100% reproducible. Unfortunately, to repro this you will need to have Qt framework installed. Currently I can't fix the issue by myself, because it will take too much time to understand all the relationships so maybe it will be easier for you.
So my setup:
Windows 10 x64
Qt 5.6.0 for Visual Studio 2015
exiv2 v0.25 (+ my fixes for threading)
Application I've created (in the attachment) takes directory with images as 1st argument and creates 1 thread per image which reads XMP metadata.
Exact images I've used are here https://drive.google.com/file/d/0B05eQJybUAszdVppU1JYcXlqRUE/view?usp=sharing
Crash (as in description of this issue) is 100% reproducible in Qt Creator if you "Start Debugging" the application, when specifying input parameter via Project's -> Run tab as directory path to the images I've mentioned before.
(to make it work, you will need to fix include path to exiv2 sources in the .pro file)
If anything, just a bit patched exiv2 sources I've used are in here https://bitbucket.org/ribtoks/xpiks-deps/src in exiv2-0.25 dir.
Updated by Taras Kushnir about 5 years ago
Qt 5.6.0 can be found here http://download.qt.io/official_releases/qt/5.6/5.6.0/
Updated by Taras Kushnir about 5 years ago
I've used qt-opensource-windows-x86-msvc2015_64-5.6.0.exe but I think it will repro on any of 5.6.0
Updated by Robin Mills about 5 years ago
Taras
This will have to wait for v0.27. I'm in the final push to get v0.26 finished. In August/September I invested lots of effort into Qt/MinGW/32 with Qt 5.6.0 and now build/test/publish daily builds for that platform. We have been publishing daily builds of VS2015 since December 2015. I've run out of time, energy, and enthusiasm to dive into Qt/VS2015/Multi-threading at the moment.
If you are confident about your patch, I am willing to submit it to v0.26 provided:
- You patch does not break our test suite.
- You send a patch file which I can easily and review and submit to our repository.
However, if I have to spend more than a couple of hours on this, I must and will defer.