mc is not using graphical characters when running on osx (ssh connections)

[sorin]

I discovered that when I do ssh to a machine using OS X 10.6 and use mc I do not see the graphical line drawing characters.

This does not happen if I open terminal and start mc.

I'm connecting using putty configured to use xterm-color, configuraton that works just fine if I do ssh to a linux machine.

The mc from OS X is version 4.7.0 (installed using macports).

The environment shows TERM=xterm-color in both cases Terminal.app and ss but mc looks different.

The only difference I was able to identify between the ssh connection and the local Terminal.app was regarding what locale returns:

On ssh (when mc fails to draw the graphical characters)
LANG=
LC_COLLATE="C"
LC_CTYPE="C" <== on Terminal.app this is LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

This looks like a bug in mc.

[sorin]

Additional info:
ssh: mc > display bits shows: 7-bit ASCII (changing does not help, it defaults to the same value)
Terminal.app: mc > display bits shows: UTF-8

[zaytsev]

This looks like putty configuration bug to me. Set the encoding to UTF-8 and see if it is any different.

And, by the way, a screenshot would be nice.

[sorin]

Putty is already configured to use UTF-8 encoding and it has not problems when doing ssh to other platforms, like Ubuntu or CentOS.

I hope the screenshots are ok. I know that I can start mc with mc -a to improve the experience a little bit but it's harder.

[zaytsev]

The fact that it shows 7-bit ASCII is correct, but it should normally draw the characters anyway.

Can you try running >>> LC_CTYPE="UTF-8" mc <<< in Putty to see if it works? Just curious.

[sorin]

LC_TYPE is not an environment variable so you cannot change it this way.

Instead I found a way of changing it by doing export LC_ALL=en_US.UTF-8 - see ​http://jonkinney.com/articles/2009/11/23/change-locale-on-os-x-snow-leopard-for-freetds-functionality

This changed all locale and now the mc runs properly.

Now I have an workaround but we we need to find a solution to make this work by default for all users.

Any ideas? I would consider as an acceptable solution even if mc will fallback to the "mc -a" mode when the LC_CTYPE="C".

Also we should remember updating the FAQ regarding this subject.

[zaytsev]

The point is, that on my platforms, LC_ALL=C mc does draw the borders correctly. Maybe single-byte people such as andrew_b can reproduce this bug.

And, by the way, can you upgrade to the latest version of mc? It might be that this bug is no longer valid.

[andrew_b]

Replying to zaytsev:

The point is, that on my platforms, LC_ALL=C mc does draw the borders correctly. Maybe single-byte people such as andrew_b can reproduce this bug.

I don't have any OS X installation in my neighbourhood.

[sorin]

I managed to reproduce this bug the other way, by setting the LC_ALL="C" on another machine (linux) and the behavior was the same: bug appeared.

Can you confirm if newer versions of mc do not reproduce the bug when you set LC_ALL="C"?

I did not had enough time to compile mc on OSX due to some problems related to mac-ports.

Instead I compiled latest release (4.7.0.8) on Ubuntu and when I set LC_ALL="C" the bug reproduces.

In this case the real issue is when LC_ALL is set to "C", on all platforms.

[zaytsev]

I can't reproduce this on Ubuntu Hardy, 4.7.0.8 package from my PPA. Everything is drawn correctly. mc correctly detects it as 7bits ASCII.

zyv@mypride:~/Desktop$ LC_ALL="C" LANG="C" mc

zyv@mypride:~/Desktop$ locale
LANG=C
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C

Are you still running mc from putty on Ubuntu? What is your echo $TERM? Can you try TERM=xterm mc?

[sorin]

You are right regarding that mc does switch to 7-bits ASCII.

My terminal is using xterm-color in all cases (putty or Terminal.app), and changing it to xterm or color_xterm does not produce any change in behavior.

But I think we have a different expectations regarding the result: I would expect to see graphical characters or ASCII ones but never characters like "lqxmwv" instead of graphical characters.

I really do not understand why there are *3* modes of displaying the graphical characters. It does make sense to see only two: ASCII (mc -a) and the graphical ones.

On OS X (10.6) the default locale is "C" *but* the console is really using UTF-8. I tested using the Terminal.app and by doing ssh to the same machine. On both cases running cat file_with_utf8_content proved me that the console is fully supporting UTF-8 (the test file contained a wide range on unicode characters).

Considering that the console is already supporting UTF-8 (even if you use LC_ALL=C) probably mc could properly use graphical characters always. If this is not possible for other reasons, like portability, it should use ascii at least.

One thing is sure: the doc/faq is wrong about a the causes of displaying the graphical characters as lowercase Latin letters.

By the way, what kind of platform do you use? Setting LC_ALL=C did reproduce the bug for me on all platforms I tested: centos, ubuntu and osx.

[zaytsev]

You don't seem to understand what I am saying. I am saying that for me it draws the lines correctly using pseudo-graphics even if the locale is set to C (which is exactly what one would expect to see if the terminal correctly supports the whole set of ASCII characters).

Also, you didn't answer my question, whether you always use Putty as a terminal emulator when connecting to these machines or not.

I am running wide range of Ubuntus and CentOSes, all fully UTF-8 enabled with gnome-terminal as the terminal emulator. I can't reproduce your problem.

[mwisnicki]

Same thing happens with mc-4.7.2 on FreeBSD-8.1 when using Putty.

Putty does not ​support VT100 line drawing characters in UTF-8 mode.

You can work around it by forcing ncurses to use Unicode line drawing characters with environment variable NCURSES_NO_UTF8_ACS=1.

I don't know why it does not happen on linux servers.

[zaytsev]

Wow, dude, you've almost got it. From the manual page:

Specifically, when running in a UTF-8 locale, 
the Linux console emulator and the GNU screen 
program ignore these.

That's why it doesn't happen on Linux and under screen.

OP, I think you should file a bug against ncurses to ask Thomas to add Putty to the NCURSES_NO_UTF8_ACS blacklist.

Does this happen with slang by the way?

Either way, sounds like it's not an mc bug.

[mwisnicki]

Linux console emulator (TERM=linux) is used only on system console (i.e. outside of X).
Putty by default pretends to be xterm (TERM=xterm), you can configure putty to set terminal type (Connection\Data\Terminal-type string) of linux but then it's not xterm and you lose mouse and certain key codes.

I don't think there is a way to detect if we are running in putty. It's just putty that's broken, if it wants to pretend to be xterm it should behave like one.

What I don't understand is that when googling for solution I've found many bugs in bug trackers of various linux distributions and while some of them acknowledged the problem and recommended setting NCURSES_NO_UTF8_ACS when using Putty, there were also those where people said that it works for them out of the box.

[zaytsev]

Excellent analysis! Many thanks for your help. Closing as not an mc bug.

[mwisnicki]

In case someone found this bug when troubleshooting putty:

I've created a modified version of PuTTY with support for ACS linedrawing in UTF-8 (as an option), fixed UTF-8 window titles and more here:

• ​http://github.com/mwisnicki/putty/downloads

BTW supporting ACS is against UTF-8 standard so it's the whole world (various xterms, ncurses) that's broken while PuTTY (and Terminal.app) actually conform to standards.

[zaytsev]

Let's put it this way: we have nothing to do with line drawing. If something is broken, it must be ncurses, which exhibits incorrect behavior by default, but again, this can be changed using an environment variable, which, I guess, you can put into your Putty profile to be set on log in.