cannot open file on CIFS mount folder

Added by A B 30 days ago

Hello, my 1st post here, since I see some similar issues reported back in 2011 without resolution.

I'm happily running exiv2 (0.25 001900 32bit) for a couple of years on raspbian stretch mainly to extract nightly exif info from IR camera feeds.

Recently I was forced to use a CIFS mounted (via autofs) drive to store the images and despite all attempts to resolve 'usual' permission' issues (password, file_modes, uid/giu, nounix etc etc) I cannot get exiv2 command to work on any mounted images, but when copied onto any other linux system it works as expected and indeed other file manipulations (cp, display/edit) works.

I suspect it may not be a CIFS/Linux permissions issue so:-

Example:
$ exiv2 /media/300gb/RPi1-Pictures/test1.jpg
/media/300gb/RPi1-Pictures/test1.jpg: Failed to open the file

Permissions:
$ ls -l /media/300gb/RPi1-Pictures/test1.jpg
-rwxrwxrwx 1 pi pi 319705 Nov 10 10:51 /media/300gb/RPi1-Pictures/test1.jpg

Autofs Mount;
RPi1-Pictures fstype=cifs,rw,rsize=165000,wsize=165000,vers=1.0,username=pi,password=,uid=1000,gid=1000,file_mode=0777,dir_mode=0777 ://192.168.1.170/Elements1Tb/RPi1-Pictures

Could anyone point me towards a solution?

Thanks
Andrew


Replies (9)

RE: cannot open file on CIFS mount folder - Added by Robin Mills 29 days ago

I can't think of any reason why exiv2 would "fail to open file" on a remote drive.

Two questions:
1) Can you edit/update ascii files on the remote drive with your favourite text editor?
2) Can you run regular linux utilities on the file. For example:

$ od -a /media/300gb/RPi1-Pictures/test1.jpg | head -4 

I have a Linux VM on my MacBook-pro running Parallels. Parallels installs networking magic that exposes the Mac's ~ directory as /media/psf/Home. I can seamlessly update files on the Mac from Linux. For example (using my favourite test file which you can download from https://clanmills.com/Stonehenge.jpg.

568 rmills@rmillsmbp:~/bin $ exiv2 -pa --grep datetime/i ~/Stonehenge.jpg 
Exif.Image.DateTime                          Ascii      20  2015:07:16 20:25:28   <-- 20:25
Exif.Photo.DateTimeOriginal                  Ascii      20  2015:07:16 15:38:54
Exif.Photo.DateTimeDigitized                 Ascii      20  2015:07:16 15:38:54
On Linux:
rmills@rmillmbp-ubuntu:~$ od -a  /media/psf/Home/Stonehenge.jpg | head -4
0000000 del   X del   a   ;   8   E   x   i   f nul nul   I   I   * nul
0000020  bs nul nul nul  vt nul  si soh stx nul dc2 nul nul nul dc2 nul
0000040 nul nul dle soh stx nul  ff nul nul nul   $ nul nul nul dc2 soh
0000060 etx nul soh nul nul nul soh nul nul nul sub soh enq nul soh nul
rmills@rmillsmbp-ubuntu:~$ exiv2 -pa --grep datetime/i  /media/psf/Home/Stonehenge.jpg 
Exif.Image.DateTime                          Ascii      20  2015:07:16 20:25:28
Exif.Photo.DateTimeOriginal                  Ascii      20  2015:07:16 15:38:54
Exif.Photo.DateTimeDigitized                 Ascii      20  2015:07:16 15:38:54
rmills@rmillsmbp-ubuntu:~$ rmills@rmillmbp-ubuntu:~$ exiv2 -a -00:01:00 /media/psf/Home/Stonehenge.jpg # Adjust age by negative one minute
rmills@rmillsmbp-ubuntu:~$ exiv2 -pa --grep datetime/i  /media/psf/Home/Stonehenge.jpg 
Exif.Image.DateTime                          Ascii      20  2015:07:16 20:24:28
Exif.Photo.DateTimeOriginal                  Ascii      20  2015:07:16 15:37:54
Exif.Photo.DateTimeDigitized                 Ascii      20  2015:07:16 15:37:54
rmills@rmillmbp-ubuntu:~$ 
Back on the Mac, Stonehenge is a minute older!
569 rmills@rmillsmbp:~/bin $ exiv2 -pa --grep datetime/i ~/Stonehenge.jpg 
Exif.Image.DateTime                          Ascii      20  2015:07:16 20:24:28 <-- 20:24
Exif.Photo.DateTimeOriginal                  Ascii      20  2015:07:16 15:37:54
Exif.Photo.DateTimeDigitized                 Ascii      20  2015:07:16 15:37:54
570 rmills@rmillsmbp:~/bin $ 
I vaguely remember a discussion about Exiv2 being unable to correctly access CIFS file-system files. I think CIFS is SMB, is it not? I use SMB to connect my MacOS-X machines. Since the last couple of upgrades to the OS (High Sierra and Mojave), AFS doesn't seem to work well for any purpose and this has nothing to do with Exiv2 - it's system wide.

I have thoughts about this matter and I'm sure we'll discuss this more in the days ahead. Together we'll solve this. I'm confident of our success in discovering the reason and remedy.

RE: cannot open file on CIFS mount folder - Added by Gilles Caulier 29 days ago

I can confirm this problem with CIFS file system mounted under Linux :

https://bugs.kde.org/show_bug.cgi?id=338462

The digiKam console trace indicate that image editor stop while metadata are writen in remote file when image is saved after modification.

note : and yes, CIFS is Samba stuff...

Gilles Caulier

RE: cannot open file on CIFS mount folder - Added by Robin Mills 29 days ago

Thanks for this response, Gilles.

I'll look at this next week (starting Monday 2018-11-19). I'm on vacation (long weekend running in Wales) and when I get home, my priority is Exiv2 v0.27 RC2 for Friday 2018-11-16. Incidentally, we're aiming for RC3 on 2018-12-07 and GM on 2018-12-28. I hope there will be no code changes between RC3 and GM. Thank You very much for your input on Localisation. As always, Luis is doing very good work.

RE: cannot open file on CIFS mount folder - Added by A B 29 days ago

Thanks Robin and merci Gilles,

Yes indeed my 'nas' router (linux / opensource) choose Samba/CIFs to share files using USB port.

Happy to say I kept my Digikam and Piwigo images and db on linux/NFS.

I needed to keep recoding hedgehog hibernation activity so last night I switched my images recording away from the CIF/Samba and confirm my metadata is back! So it seems the issue lies with the CIFs/Samba handling of metadata.

In response to Robin's question all seems OK:-

1. can edit simple text file

andrew@VESA-ubuntu:/nfs/all/USB1Tb$ touch test.txt
andrew@VESA-ubuntu:/nfs/all/USB1Tb$ nano test.txt
andrew@VESA-ubuntu:/nfs/all/USB1Tb$ cat test.txt
hello
andrew@VESA-ubuntu:/nfs/all/USB1Tb$ ls -l test.txt
-rwxrwx--- 1 andrew andrew 6 nov.  11 13:54 test.txt
andrew@VESA-ubuntu:/nfs/all/USB1Tb$ 

2. od utility

andrew@VESA-ubuntu:/nfs/all/USB1Tb$ od -a test.txt
0000000   h   e   l   l   o  nl
0000006

With this success I tried a manual simulation of my workflow using exiv2 on a single image, captured on raspberrypi (raspbian) copied to CIFs/Samba and this seems to work:-

andrew@VESA-ubuntu:/mnt/RPi3-home-pi/RPi1-Pictures$ exiv2 -g DateTime -pt 154192999801.jpg
Exif.Image.DateTime                          Ascii      20  2018:11:11 10:53:18
Exif.Photo.DateTimeOriginal                  Ascii      20  2018:11:11 10:53:18

andrew@VESA-ubuntu:/mnt/RPi3-home-pi/RPi1-Pictures$ cp 154192999801.jpg /nfs/all/USB1Tb/
andrew@VESA-ubuntu:/mnt/RPi3-home-pi/RPi1-Pictures$ cd /nfs/all/USB1Tb/

andrew@VESA-ubuntu:/nfs/all/USB1Tb$ exiv2 -g DateTime -pt 154192999801.jpg
Exif.Image.DateTime                          Ascii      20  2018:11:11 10:53:18
Exif.Photo.DateTimeOriginal                  Ascii      20  2018:11:11 10:53:18

andrew@VESA-ubuntu:/nfs/all/USB1Tb$ exiv2 -g DateTime ex -f 154192999801.jpg
andrew@VESA-ubuntu:/nfs/all/USB1Tb$ ls -l 154192999801.*
-rwxrwx--- 1 andrew andrew   181 nov.  11 14:03 154192999801.exv
-rwxrwx--- 1 andrew andrew 61249 nov.  11 14:02 154192999801.jpg

andrew@VESA-ubuntu:/nfs/all/USB1Tb$ exiv2 -g DateTime in -f 154192999801.JPG
andrew@VESA-ubuntu:/nfs/all/USB1Tb$ exiv2 -g DateTime -pt 154192999801.JPG
Exif.Image.DateTime                          Ascii      20  2018:11:11 10:53:18
Exif.Photo.DateTimeOriginal                  Ascii      20  2018:11:11 10:53:18

The difference from my failed workflow is that the image is recorded directly to the CIFs/samba from the Raspberrypi. I dare not switchback to confirm this since the hedgehog are busy. But once everything calms down and if you need it I can check.

AB
PS . Thanks again for your work.

RE: cannot open file on CIFS mount folder - Added by Robin Mills 29 days ago

I suspect this has something to do with memory-mapping the file.

Exiv2 uses BasicIo to perform I/O and has sub-classes such as FileIo , MemIo, HttpIo and others to use images on different media. On the desktop, exiv2(.exe) usually uses memory-mapping and perhaps that's not valid for CIFS mounted files.

I'll look at this next week as releasing RC2 is the priority for this week. I'm not sure how to set up CIFS from Linux as I've never given that matter any thought. I'll send you some code to implement a sub-set of the utility od which I'll implement with/without memory mapping to validate/confirm/deny my hunch.

RE: cannot open file on CIFS mount folder - Added by Robin Mills 29 days ago

Andrew

We don't want to disturb the hedgehogs. Gosh no. And I'm in Wales to run in the Mountains. And apparently this issue has been present for years. So, this isn't urgent. Interesting and curious. If we need a code change, it'd be great to get it into exiv2 v0.27 RC3 (scheduled 2018-12-07).

I'm curious about the relationship with the rasperry π

I thought your issue is that the exiv2 command line program will not read (nor write) on the nas storage connected to Linux by a USB cable and mounted using CIFS.

So why discuss the π ? Are you saying that if you mount the nas using NFS, exiv2 works? Or am I confused?

RE: cannot open file on CIFS mount folder - Added by A B 28 days ago

Thanks Robin, no rush for sure. Sorry for confusion.

The camera was back recording last night, the hedgehogs are close to shutdown anyhow, despite global warming..... which is part of this study but don't let me start on that.....

So the RaspberryPi is the IR camera, I just gave example of command line to demo the 'bug' in actual workflow which is:-

1. Raspberrypi (Zero W in field with IR cam) captures 2 stream of jpgs, the low-res has DateTime stamp.
2. Until last week the stream was saved over WiFi using NFSv4 to another Raspberrypi3 server.
3. At dawn the Raspberrypi3 runs FFmpeg and exiv2 on the jpgs to create timestamped video of action for research/archive.

This has worked well for 2 years or more. Then due to router upgrade, we decided to the router USB3 NAS to store the jpg stream thinking it would be easier/larger etc. But we lost exiv2 functions.

So, yes if we mount the NAS (USB HDD) using NFS exiv2 works as expected.
My manual attempt to run exiv2 on 11th also seemed to work via CIF storage also seems to work, which is confusing.

I'll double check everything and report back if you think it might help you, but on my side I'll stick with original Linux based NFS4 EXT4 storage.

RE: cannot open file on CIFS mount folder - Added by Robin Mills 28 days ago

Andrew

Thanks for the clarification. So, my task is to un-puzzle exiv2/CIFS refusing to open files. How odd! I'll investigate next week. When I get home from Wales, I want to focus my attention on Exiv2 v0.27 RC2 and to release that this week.

Robin

RE: cannot open file on CIFS mount folder - Added by Robin Mills 16 days ago

Andrew and Gilles

I haven't had time to look into this issue this week. However it isn't forgotten. Luis and I are both working hard to get the Exiv2 v0.27 GM by 28 December 2018. We hoped to arrive at zero open issues for v0.27 today. I suspect this CIFS issue has been in the code for years and doesn't impact every Samba mount. If I can't investigate this next week, I'll open an issue on GitHub to be sure it's never forgotten.

Robin

(1-9/9)

Redmine Appliance - Powered by TurnKey Linux