macOS Sierra 10.12.6 non-ascii characters issue

[nebm51]

Hello,

I'm experiencing strange behavior as all non-ascii characters are shown as question marks (in file panels) or as question marks and other non-letter symbols in view. Sample screenshots are in attachement. Display codepage is set to UTF-8. Issue appears both in Terminal.app and in iTerm2

 locale
LANG="ru_RU.UTF-8"
LC_COLLATE="ru_RU.UTF-8"
LC_CTYPE="UTF-8"
LC_MESSAGES="ru_RU.UTF-8"
LC_MONETARY="ru_RU.UTF-8"
LC_NUMERIC="ru_RU.UTF-8"
LC_TIME="ru_RU.UTF-8"
LC_ALL=

 mc -V
GNU Midnight Commander 4.8.19
Built with GLib 2.52.2
Using the S-Lang library with terminfo database
With builtin Editor
With subshell support as default
With support for background operations
With mouse support on xterm
With internationalization support
With multiple codepages support
Virtual File Systems: cpiofs, tarfs, sfs, extfs, ftpfs, sftpfs, fish
Data types: char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

 mc -F
Home directory: /Users/jim
Profile root directory: /Users/jim

[System data]
    Config directory: /usr/local/Cellar/midnight-commander/4.8.19/etc/mc/
    Data directory:   /usr/local/Cellar/midnight-commander/4.8.19/share/mc/
    File extension handlers: /usr/local/Cellar/midnight-commander/4.8.19/libexec/mc/ext.d/
    VFS plugins and scripts: /usr/local/Cellar/midnight-commander/4.8.19/libexec/mc/
	extfs.d:        /usr/local/Cellar/midnight-commander/4.8.19/libexec/mc/extfs.d/
	fish:           /usr/local/Cellar/midnight-commander/4.8.19/libexec/mc/fish/

[User data]
    Config directory: /Users/jim/.config/mc/
    Data directory:   /Users/jim/.local/share/mc/
	skins:          /Users/jim/.local/share/mc/skins/
	extfs.d:        /Users/jim/.local/share/mc/extfs.d/
	fish:           /Users/jim/.local/share/mc/fish/
	mcedit macros:  /Users/jim/.local/share/mc/mc.macros
	mcedit external macros: /Users/jim/.local/share/mc/mcedit/macros.d/macro.*
    Cache directory:  /Users/jim/.cache/mc/

mc --configure-options
 '--disable-debug' '--disable-dependency-tracking' '--disable-silent-rules' '--prefix=/usr/local/Cellar/midnight-commander/4.8.19' '--without-x' '--with-screen=slang' '--enable-vfs-sftp' 'CC=clang' 'PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig:/usr/local/opt/pcre/lib/pkgconfig:/usr/local/opt/glib/lib/pkgconfig:/usr/local/opt/openssl/lib/pkgconfig:/usr/local/opt/libpng/lib/pkgconfig:/usr/local/opt/s-lang/lib/pkgconfig:/usr/local/opt/libssh2/lib/pkgconfig' 'PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.12'

[zaytsev]

Looks like your locale is set incorrectly. Try

LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 mc

and if this works, then maybe you are missing Russian locale files or something...

[nebm51]

Thank you for the hint.

I've tried different combinations and found, that issue appears if LC_ALL is not set. E.g. when I set LC_ALL to ru_RU.UTF-8 or en_US.UTF-8 non-ascii symbols are shown correctly. If I unset this variable not touching other - issue appears again.

Further investigation showed, that issue is related to LC_CTYPE variable. By default it's set to UTF-8, if I add lang specification like en_US.UTF-8 or ru_RU.UTF-8 then symbols start to work as they should be. So

This does not work for mc

$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

This works for mc

 locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

Curious thing here is that LC_TYPE=UTF-8 does not produce wrong symbols representation for other console apps, like vim, emacs or nano.

[zaytsev]

Ok, so closing this as not a bug in mc.

I'm not sure how you are setting your locale, but every Unix has it's own way of doing this right, and as far as I remember, to set it correctly in Terminal.app, there are some options buried deep down in the settings dialog...

You need to consult system documentation, or seek help elsewhere, we are not able to assist, as we are not using macOS ourselves.

[egmont]

On Linux, LC_CTYPE needs to contain a locale (just as the others), that is, language, country, charset.

I vaguely recall that on Mac it can contain simply a charset. (I'm not sure though.) Maybe that's not the case with mc itself, or maybe with slang, one of them probably requires a Linux-style value.