Compile failure on OS X 10.9

[kroleg]

I was unable to compile MC on OS X 10.9. The error message is the following:

5 warnings generated.
warning: unknown warning option '-Wfloat-conversion' [-Wunknown-warning-option]
warning: unknown warning option '-Wformat-signedness'; did you mean '-Wformat-non-iso'?
      [-Wunknown-warning-option]
warning: unknown warning option '-Wmaybe-uninitialized'; did you mean '-Wuninitialized'?
      [-Wunknown-warning-option]
warning: unknown warning option '-Wmissing-parameter-type' [-Wunknown-warning-option]
warning: unknown warning option '-Wunused-but-set-variable'; did you mean '-Wunused-const-variable'?
      [-Wunknown-warning-option]
5 warnings generated.
duplicate symbol _menu_map in:
    ./.libs/libinternal.a(keybind-defaults.o)
    ./.libs/libinternal.a(menu.o)
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [mc] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
bash-4.4$ 

The full log is attached.

[andrew_b]

Branch: 4035_menu_map_dup
changeset:093571938a6c31706f918b011335b8ab30602336

[kroleg]

Replying to andrew_b:

Branch: 4035_menu_map_dup
changeset:093571938a6c31706f918b011335b8ab30602336

из 093571938a6c31706f918b011335b8ab30602336 собирается нормально
из ​git://github.com/MidnightCommander/mc.git после замены 4-х файлов - тоже, нормально
из mc-4.8.23 , после замены 4-х файлов, сборка завершается с ошибкой

warning: unknown warning option '-Wunused-but-set-variable'; did you mean '-Wunused-const-variable'?
      [-Wunknown-warning-option]
midnight.c:918:38: error: too many arguments to function call, expected 2, have 5
    the_menubar = menubar_new (0, 0, COLS, NULL, menubar_visible);
                  ~~~~~~~~~~~        ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../../lib/tty/tty-slang.h:27:15: note: expanded from macro 'COLS'
#define COLS  SLtt_Screen_Cols
              ^
../../lib/widget/menu.h:53:1: note: 'menubar_new' declared here
WMenuBar *menubar_new (GList * menu, gboolean visible);
^
5 warnings and 1 error generated.
make[3]: *** [midnight.lo] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
bash-4.4$ 

[andrew_b]

Replying to kroleg:

из 093571938a6c31706f918b011335b8ab30602336 собирается нормально
из ​git://github.com/MidnightCommander/mc.git после замены 4-х файлов - тоже, нормально

It's ok. All fixes are made for the current master.

из mc-4.8.23 , после замены 4-х файлов, сборка завершается с ошибкой

Since there are some differences between 4.8.23 and current master, patches for master might not be applicable for 4.8.23. 4.8.24 release is expected before end of year. Alternatively, you should build snapshot from master, or patch 4.8.23 yourself.

Please write in English.

[kroleg]

Replying to andrew_b:

Replying to kroleg:

из 093571938a6c31706f918b011335b8ab30602336 собирается нормально
из ​git://github.com/MidnightCommander/mc.git после замены 4-х файлов - тоже, нормально

It's ok. All fixes are made for the current master.

из mc-4.8.23 , после замены 4-х файлов, сборка завершается с ошибкой

Since there are some differences between 4.8.23 and current master, patches for master might not be applicable for 4.8.23. 4.8.24 release is expected before end of year. Alternatively, you should build snapshot from master, or patch 4.8.23 yourself.

Please write in English.

Pardon me. Thank you!

[kroleg]

After applying mc-4.8.23.patch, compile again fails:

5 warnings generated.
  CCLD     libinternal.la
copying selected object files to avoid basename conflicts...
  CCLD     mc
Making all in doc
Making all in hints
Making all in l10n
make[4]: Nothing to be done for `all'.
make[4]: Nothing to be done for `all-am'.
Making all in man
Making all in es
MAN_FILE=mc.1.in; MAN_DATE=$(LC_ALL=es_ES.UTF-8 /opt/local/bin/perl -MPOSIX -e ' @fi=lstat("'${MAN_FILE}'"); print POSIX::strftime("%B de %Y", localtime($fi[9]));' 2>/dev/null); \
	sed -e "s/%DATE_OF_MAN_PAGE%/${MAN_DATE}/g" -e "s/%DISTR_VERSION%/4.8.23/g" -e "s{%prefix%{/usr/local{g" -e "s{%sysconfdir%{/usr/local/etc{g" -e "s{%libexecdir%{/usr/local/libexec{g" mc.1.in > mc.1
Making all in hu
MAN_FILE=mc.1.in; MAN_DATE=$(LC_ALL=hu_HU.UTF-8 /opt/local/bin/perl -MPOSIX -e ' @fi=lstat("'${MAN_FILE}'"); print POSIX::strftime("%Y. %B", localtime($fi[9]));' 2>/dev/null); \
	sed -e "s/%DATE_OF_MAN_PAGE%/${MAN_DATE}/g" -e "s/%DISTR_VERSION%/4.8.23/g" -e "s{%prefix%{/usr/local{g" -e "s{%sysconfdir%{/usr/local/etc{g" -e "s{%libexecdir%{/usr/local/libexec{g" mc.1.in > mc.1
sed: RE error: illegal byte sequence
make[4]: *** [mc.1] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
bash-4.4$

[andrew_b]

Replying to andrew_b:

Branch: 4035_menu_map_dup
changeset:093571938a6c31706f918b011335b8ab30602336

Merged to master: [23f259fde327934f6a120fb195bc4a75b4190d2b].

[andrew_b]

Replying to kroleg:

After applying mc-4.8.23.patch, compile again fails:

Making all in hu
MAN_FILE=mc.1.in; MAN_DATE=$(LC_ALL=hu_HU.UTF-8 /opt/local/bin/perl -MPOSIX -e ' @fi=lstat("'${MAN_FILE}'"); print POSIX::strftime("%Y. %B", localtime($fi[9]));' 2>/dev/null); \
	sed -e "s/%DATE_OF_MAN_PAGE%/${MAN_DATE}/g" -e "s/%DISTR_VERSION%/4.8.23/g" -e "s{%prefix%{/usr/local{g" -e "s{%sysconfdir%{/usr/local/etc{g" -e "s{%libexecdir%{/usr/local/libexec{g" mc.1.in > mc.1
sed: RE error: illegal byte sequence
make[4]: *** [mc.1] Error 1
make[3]: *** [all-recursive] Error 1

Can't say anything about HU locale on the target platform.

@egmont?

[egmont]

Manually executing this command on Linux:

LC_ALL=hu_HU.UTF-8 perl -MPOSIX -e ' @fi=lstat("/etc/services"); print POSIX::strftime("%Y. %B", localtime($fi[9]));'

which is the same as in the log excerpt, except for a hardwired filename, prints incorrect accent. You can try with hu_HU.UTF-8 and months 1-7 and 10, or de_DE.UTF-8 and month 3, or plenty of other combos.

If the intended string isn't representable in Latin-1, e.g. with ru_RU.UTF-8, the output appears correctly but you also get a warning "Wide character in print at -e line 1".

This is because perl's stdout is not set to UTF-8. You need one of the working solutions from ​https://stackoverflow.com/q/15210532 or ​https://stackoverflow.com/q/627661, such as e.g. "perl -CS" or "binmode(STDOUT, ":utf8");".

[egmont]

On my Ubuntu 19.10, if this manual page is installed to a system location and the "man" command uses its search path to locate it (as opposed to passing it a full path to the file), an incorrect UTF-8 sequence triggers the entire manual page to be interpreted in ISO-8859-2.

In other words, after installing mc, the command "LC_ALL=hu_HU.UTF-8 man mc" gives an unreadably broken manual page if and only if the timestamp of mc.1.in happens to be in a month with an accent in its name.

We caught a pretty nasty bug. :)

[kroleg]

Error with the Hungarian manual decided:

cat /dev/null > ./doc/man/hu/mc.1.in

Building mc-4.8.23 completed successfully. Thank you all.

[andrew_b]

This is dirty hack.

If you want skip HU docs, then following patch can be used:

doc/man/Makefile.am

@@ -1 @@
-DIST_SUBDIRS = es hu it pl ru sr
+DIST_SUBDIRS = es it pl ru sr
 if USE_NLS
 SUBDIRS = $(DOC_LINGUAS)
 endif

[kroleg]

Replying to andrew_b:

This is dirty hack.

My stupid mistake.
I opened ./doc/man/hu/mc.1.in in "mcedit" to edit and saved it without changes.
The compilation completed successfully.

[zaytsev]

So let's just replace perl -MPOSIX -e with perl -CS -MPOSIX -e -- thanks Egmont for the analysis!

[kroleg]

Replying to zaytsev:

So let's just replace perl -MPOSIX -e with perl -CS -MPOSIX -e -- thanks Egmont for the analysis!

I added in patch file of your proposal. The compilation completed successfully.

[andrew_b]

Branch: 4035_man_perl_unicode.
changeset:b263f2ae3561144dfaff9d3508070187722b390a

[andrew_b]

Merged to master: [e12f563fcfbfc4323ca50a62d2ec112174a15b5f].

Thanks everybody!

[egmont]

On a side note, taking the date to be embedded in the manual page from the file's timestamp is I'm afraid against the great goal of reproducible builds (see e.g. #4031).

If someone operates from the official tarballs, I'm not sure whether a difference can be hit on rare circumstances. I'm not sure if tarballs contain Unix timestamps (hopefully) or local time (would be damn ugly). The perl command looks at the month according to the local time. If the file's timestamp is near a month boundary, the generated manual page may depend on the timezone.

Even worse when operating on git versions. Upon a git clone, all the files receive the current time as timestamp (and not some timestamp stored in the repository – perhaps this metadata isn't even stored there). Upon a git pull, files that actually change will have their timestamps updated, again, to the current time. So the resulting manual page depends on when the tree was git cloned originally, and git pulled subsequently.

If a distribution patches a manual page, the embedded date will be the date of compiling.

I don't think it's wise for the result to depend on metadata such as timestamps. I'm just not sure what would be better, and still reasonable. (In fact, I'm wondering whether the timestamp in the manual page is supposed to refer to the software version, or when the manual was last updated.) Either maintain the timestamp manually, or if it's too much trouble (sure it is) then maybe just don't put a date there at all? :) I'm wondering what other projects do here...

[jpew]

Replying to egmont:

On a side note, taking the date to be embedded in the manual page from the file's timestamp is I'm afraid against the great goal of reproducible builds (see e.g. #4031).

If someone operates from the official tarballs, I'm not sure whether a difference can be hit on rare circumstances. I'm not sure if tarballs contain Unix timestamps (hopefully) or local time (would be damn ugly). The perl command looks at the month according to the local time. If the file's timestamp is near a month boundary, the generated manual page may depend on the timezone.

Even worse when operating on git versions. Upon a git clone, all the files receive the current time as timestamp (and not some timestamp stored in the repository – perhaps this metadata isn't even stored there). Upon a git pull, files that actually change will have their timestamps updated, again, to the current time. So the resulting manual page depends on when the tree was git cloned originally, and git pulled subsequently.

If a distribution patches a manual page, the embedded date will be the date of compiling.

I don't think it's wise for the result to depend on metadata such as timestamps. I'm just not sure what would be better, and still reasonable. (In fact, I'm wondering whether the timestamp in the manual page is supposed to refer to the software version, or when the manual was last updated.) Either maintain the timestamp manually, or if it's too much trouble (sure it is) then maybe just don't put a date there at all? :) I'm wondering what other projects do here...

General consensus for reproducible builds is to respect the $SOURCE_DATE_EPOCH environment variable (if set) when encoding timestamps. The build system will set this to an appropriate value before building your package. Many tools like tar have added options like --clamp-mtime to limit the latest timestamp they encode to a specific value, so if you're creating an archive, you do something like tar --clamp-mtime=$SOURCE_DATE_EPOCH ...

[jpew]

See ​https://reproducible-builds.org/docs/source-date-epoch/

[egmont]

Thanks for sharing this knowledge! As far as I take it, the desired thing for mc would be to do something like this (totally pseudocode):

perl -e 'strftime(localtime( env('SOURCE_DATE_EPOCH') or stat(file)[9] ))'

Am I right? I like this approach, would be glad to see in mc.