Midnight Commander loses content of the current directory

[memy]

When I am browsing a mounted SMB directory Midnight Commander sometimes loses directory content.
When I hit Ctrl-r the content of the directory reappears.

It started today after upgrading of the kernel and samba.
All other means of browsing file system work perfectly fine: shell, Thunar, File Open dialogs, etc.
I have shares mounted on different servers:

1. Synology NAS
2. Raspberry Pi 4.19.42-1-ARCH #1 SMP PREEMPT Wed May 15 00:39:47 UTC 2019 armv7l GNU/Linux
3. Dell mini 5.1.2-arch1-1-ARCH #1 SMP PREEMPT Wed May 15 00:09:47 UTC 2019 x86_64 GNU/Linux

MC demonstrates the same behavior on all of the shares.

When I browse to a directory on a mounted SMB share MC occasionally shows empty directory. Re-read immediately restores the panel. The same happens when I am returning back to the parent directory.

The shares are mounted at boot time using systemd like this but the problem persists with any other way of mounting the SMB share:

[Unit]
Description=Mount Share at boot
Requires=network-online.target
After=network-online.target
Wants=network-online.target

[Mount]
What=//host/share
Where=/mnt/SMB/share
Options=uid=<my-uid>,credentials=/home/username/secret.txt,iocharset=utf8,rw,x-systemd.automount
Type=cifs
TimeoutSec=30

[Install]
WantedBy=multi-user.target

I run Arch Linux.
Kernel version: 5.1.2-arch1-1-ARCH #1 SMP PREEMPT Wed May 15 00:09:47 UTC 2019 x86_64 GNU/Linux
samba version: 4.10.3-1 upgraded from 4.10.2-1

LC_MESSAGES=C mc -V

GNU Midnight Commander 4.8.22
Built with GLib 2.58.3
Using the S-Lang library with terminfo database
With builtin Editor
With subshell support as default
With support for background operations
With mouse support on xterm and Linux console
With support for X11 events
With internationalization support
With multiple codepages support
Virtual File Systems: cpiofs, tarfs, sfs, extfs, ftpfs, sftpfs, fish, smbfs
Data types: char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

LC_MESSAGES=C mc -F

Home directory: /home/username
Profile root directory: /home/username

[System data]
    Config directory: /etc/mc/
    Data directory:   /usr/share/mc/
    File extension handlers: /usr/lib/mc/ext.d/
    VFS plugins and scripts: /usr/lib/mc/
	extfs.d:        /usr/lib/mc/extfs.d/
	fish:           /usr/lib/mc/fish/

[User data]
    Config directory: /home/username/.config/mc/
    Data directory:   /home/username/.local/share/mc/
	skins:          /home/username/.local/share/mc/skins/
	extfs.d:        /home/username/.local/share/mc/extfs.d/
	fish:           /home/username/.local/share/mc/fish/
	mcedit macros:  /home/username/.local/share/mc/mc.macros
	mcedit external macros: /home/username/.local/share/mc/mcedit/macros.d/macro.*
    Cache directory:  /home/username/.cache/mc/

mc --configure-options

 '--prefix=/usr' '--libexecdir=/usr/lib' '--sysconfdir=/etc' '--enable-vfs-smb' '--with-screen=slang' '--with-x' 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt' 'LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now' 'CPPFLAGS=-D_FORTIFY_SOURCE=2'

[memy]

Something interesting happened.
I continued experimenting and all of the sudden everything started to work as expected.
I immediately opened another instance of MC in a new terminal. That new instance did not work though.
So now I have two instances side by side and one of them works consistently while another occasionally shows me empty directory.
I am doing very simple test. I have "Lynx-like motion" option turned on, so I constantly pressing left and right arrow keys.
One instance of MC works as a clockwork, while another shows empty directory from time to time.
Unfortunately I don't know what "fixed" the first instance. I tried to repeat in the second instance the actions I remember doing in the first one, but the second instance continues to fail.

[ravenexp]

I have the same problem with SMB shares mounted via systemd mount units.
It also happened after the recent kernel and samba updates.

Other file managers like Gnome Nautilus do not suffer from this issue.

[memy]

After downgrading linux kernel to 5.0.13 everything went back to normal.

[memy]

I wonder why this ticket has been closed?
The problem still persists while running Linux 5.1+
This is obviously a MC bug, because all other ways to get directory content work flawlessly (ls, other file managers, open and save dialogs, etc).
Also, "Blocked By" field may be incorrect. I am using SMB shares mounted by systemd/mount, not SMB links so I suspect that built-in SMB library has nothing to do with the problem.

Please advise how could I provide more information to help with bug triaging.

The problem is really annoying, I have to hit Ctrl-r almost every time I descend to a folder.

[memy]

I finally found some time to debug the problem.
I found out that vfs/local/local.c/local_readdir()/readdir() sometimes returns error EINTR while reading directories of the mounted SMB share.
Retrying the filemanager/dir.c/dir_list_load() helps to solve this problem.

[memy]

Replying to ravenexp:

I have the same problem with SMB shares mounted via systemd mount units.
It also happened after the recent kernel and samba updates.

Other file managers like Gnome Nautilus do not suffer from this issue.

If you are still interested, I have implemented quick and dirty patch: ​https://github.com/myelsukov/mc/commit/66016a03ddeba115444c41783b30f92719c28b46
It works fine for me even though it is not a most beautiful piece of code :)

[andrew_b]

Replying to memy:

If you are still interested, I have implemented quick and dirty patch:

Why you patch dir_list_load() instead of local_readdir() or even mc_readdir()?

[memy]

Replying to andrew_b:

Why you patch dir_list_load() instead of local_readdir() or even mc_readdir()?

My experiments (including several months of using mc with kernel 5.1+) showed that EINTR was raised by the very first readdir(). In other words, I was always getting empty directory list not an incomplete directory list.
I started with retrying readdir() in the local_readdir(). Unfortunately, readdir() would stop (NULL result with errno == 0) after returning "." and ".." directory entries in that case. This left me with a single choice: retry the whole dir_list_load().

I have no idea why no other program (ls, find, grep, different "File Open" and "File Save" dialogs, numerous file managers) demonstrated this "bad" behavior though...

[andrew_b]

Ticket #4074 has been marked as a duplicate of this ticket.

[trtracer]

This is still open, right? I have the same problem with 4.8.25

[andrew_b]

Replying to trtracer:

This is still open, right?

No. It's closed as invalid.

I can't say why readdir(3) sometimes returns NULL instead of next file.
Reading of directory twice each time is not an our way.

[zaytsev]

Why wouldn't you guys take it up with kernel developers?

[trtracer]

I took a closer look at it and it seems a part of getdents (the syscall behind readdir) can be interrupted by SIGCHLD and this does not get restarted - even if you specify SA_RESTART when calling sigaction().

See this BT from GDB:

Catchpoint 1 (signal SIGCHLD), __getdents64 (fd=12, buf=buf@entry=0x7ffff7106040 "", nbytes=1048576) at ../sysdeps/unix/sysv/linux/getdents64.c:27
27      ../sysdeps/unix/sysv/linux/getdents64.c: No such file or directory.
(gdb) bt
#0  __getdents64 (fd=12, buf=buf@entry=0x7ffff7106040 "", nbytes=1048576) at ../sysdeps/unix/sysv/linux/getdents64.c:27
#1  0x00007ffff7844ea4 in __GI___readdir64 (dirp=0x7ffff7106010) at ../sysdeps/posix/readdir.c:65
#2  0x000055555560390a in local_readdir (data=0x555555709660) at local.c:111
#3  0x0000555555588a8f in mc_readdir (dirp=dirp@entry=0x55555573bd30) at interface.c:500
[...]

The documentation of signal(7) does not mention getdents. Neither that it eventually would be restarted nor that it wouldn't. But maybe because this only happens when accessing SMB-Shared, the getdents syscall (or a part of it) falls in the category of network functions that are never restarted.

Anyway, why not simply cover this condition by changing local_readdir like this:

local_readdir (void *data)
{
    void *dir;

    do {
        errno=0;
        dir = readdir (*(DIR **) data);
    }while (dir == NULL && errno == EINTR);
    return dir;
}

This works for me and I don't see this has a negative impact.

[memy]

Replying to trtracer:

Anyway, why not simply cover this condition by changing local_readdir like this:

I tried this and it didn't work for me, readdir always returned null after the EINTR. In my experiments I had to call opendir again to recover.
May be something had changed in the kernel after that...

[drolevar]

Replying to andrew_b:

No. It's closed as invalid.

I can't say why readdir(3) sometimes returns NULL instead of next file.
Reading of directory twice each time is not an our way.

Well, apparently it is a change introduced in the kernel here:
​https://github.com/torvalds/linux/commit/b30c74c73c787d853ecb9fcf5c59511a09a4ec59

And I see it's being fixed in a variety of other projects, like Go, for example:
​https://github.com/golang/go/issues/38836

The way it was fixed is exactly repeating it if EINTR is returned.
That's the way it's implemented in the kernel since 5.1 and I think it's by design.

[zaytsev]

​https://github.com/MidnightCommander/mc/pull/157/files

[andrew_b]

​https://patch-diff.githubusercontent.com/raw/MidnightCommander/mc/pull/157.patch

[andrew_b]

Branch: 3987_smb_dir_content
changeset:0343e7b65092921ffd47554c7f1d1aaaf222bc97

Please test.

[andrew_b]

Merged to master: [52be505d69b123b41354ec8118b5ad95fad8a40a].

[zaytsev]

Topic comes up again:

​https://lists.midnight-commander.org/pipermail/mc/2023-July/thread.html

Apparently there are still some problems with rewinddir with some VFS or kernel combinations.