Ticket #4173 (closed defect: fixed)

Opened 4 months ago

Last modified 2 months ago

[patch] libtool copies objects for unification which slows down build

Reported by: psprint Owned by:
Priority: major Milestone: 4.8.27
Component: mc-core Version: master
Keywords: Cc:
Blocked By: Blocking:
Branch state: no branch Votes for changeset:

Description

Hi,
I've had enough straying at the make message:

copying selected object files to avoid basename conflicts...

and I've decided to fix it – i.e.: to make the names of objects unique – patch is attached. It renames things such as lib/util.c and src/util.c so that their basename is unique (lib/sub-util.c in this case, for subroutine library…).

However, this is not enough to make the message (and the pause) go away. I've added following snippet into mc/libtool at line 11122 to get the names of the objects that still conflict:

            for obj in $oldobjs
	    do
	      func_basename "$obj"
	      $ECHO "$func_basename_result"
	    done | sort | sort >/tmp/libtool-info.txt 2>&1

and it turned out that there are plenty of such files. It looks like if some internal static library has been linked twice. I cannot go further than this point. Could anyone help?

I've tried removing libmc.la from:

– src/diffviewer/Makefile.am,
– src/filemanager/Makefile.am,
– src/viewer/Makefile.am,

the only places where it is (excessively, apparently) added to the linked libs but the effect is the same and the libtool-info.txt list doesn't change. Funny, isn't it?

Also, it looks like the main libmc.la reference is also not needed somehow, because changing:

mc_LDADD = \
	libinternal.la \
	$(top_builddir)/lib/libmc.la

to:

mc_LDADD = \
	libinternal.la

doesn't result in any build or runtime error. Funny again, isn't it?

From where do the duplicates come from? Has anyone some idea?

Attachments

0001-Resolve-file-name-base-conflicts-to-avoid-libtool-co.patch (70.5 KB) - added by psprint 4 months ago.
a patch that makes all conflicting source files unique
libtool-info.txt (3.5 KB) - added by psprint 4 months ago.
libtool trace on the list of objects that ·should· be unique
NoObjectCopyingWhenLinking_v3.patch (71.4 KB) - added by psprint 4 months ago.

Change History

Changed 4 months ago by psprint

a patch that makes all conflicting source files unique

Changed 4 months ago by psprint

libtool trace on the list of objects that ·should· be unique

comment:1 Changed 4 months ago by andrew_b

I don't think non-unique file names is a problem.
Apparently close as wontfix.

comment:2 Changed 4 months ago by psprint

Little regret, though… The filenames should be unique, because otherwise libtool is required to ·copy· the conflicting files under unique temporary names. Not sure, if this is only copying the names that conflict or other too, however an error in the Makefile.am files is making libmc to be linked three times so a large number of files are required to be copied anyway, and this adds 5-10 seconds to the build process (the copying object files to avoid basename conflicts… stage), which is a serious inconvenience (especially when the build is a simple non-header file change, inducing only a single file compilation and then an immediate link…)… The above patch resolves the real (not from linking multiple times) base name conflicts, so IMO it could/should be merged. Then the error in Makefile.am files could be tracked down.

PS. I've created a 100 points bounty for this problem on SO, maybe this will help solving the issue?

Last edited 4 months ago by psprint (previous) (diff)

comment:3 Changed 4 months ago by psprint

I've revised the problem and was able to successfully get rid of the "Copying object … avoid conflicts" stage of the linking! Time of the build dropped from 2:50 seconds to 2:34 seconds meaning that the time of build after basic single .c file change will be shorter by 16 seconds :) Please merge, the patch is safe as it doesn't do any logic changes, it only renames the files and – crucial change: – removes libmc.a from libdiffviewer.a and libmcviewer.a, because they are an unneeded, excessive linkage that were causing the objects of libmc.a to be linked 3 times… Without it and with the few renames, the linking is free of basename conflicts and can go without the copy stage.

Changed 4 months ago by psprint

comment:4 Changed 3 months ago by psprint

Ping on this important patch, which shortens build time. Will it be merged? It's completely safe patch – no logic is being introduced by it.

comment:5 Changed 3 months ago by andrew_b

Libraries linkage is fixed in the 4179_cleanup branch (current changeset:[95f92ece391000e88d2c65019a889dfa2c2f8136).
This branch will be merged to master after a while.
Files will not be renamed.

comment:6 Changed 2 months ago by andrew_b

  • Status changed from new to closed
  • Resolution set to fixed
  • Milestone changed from Future Releases to 4.8.27
Note: See TracTickets for help on using tickets.