Use off_t for free space calculation (free space on filesystems >2TB not properly displayed)

[swegener]

The free space calculation currently uses ints, which are just 32bit on plain x86 systems and result in free space on filesystems >2TB not being properly displayed. See attached patch.

[swegener]

This is needed to show correct free space information for todays large filesystems on 32bit systems.

[andrew_b]

Thanks!

Do we really have use signed off_t instead of unsigned uintmax_t?

Generally, we have 3 different types for FS description: my_statfs (mountlist.h:26), fs_usage (fsusage.h:36) and again fs_usage (mountlist.c:139). I think all that structures should be merged into single one.

[swegener]

Replying to andrew_b:

Do we really have use signed off_t instead of unsigned uintmax_t?

I've taken off_t, because when showing these values in src/info.c, we pass them through size_trunc_len(), which takes an off_t. In the end any integer that is 64bit on 32bit systems is fine, but we should be consistent.

[mak]

since i run into this bug daily, i was wondering if there has been any progress on the issue?

in the current version(4.7.5.3) i see this, when browsing my 6TB RAID 5:

mc: 793G/1493G (53%)
os: 700G/5.5T 13%

[andrew_b]

Branch: 2338_free_space_calculation (parent: master).
changeset:8846ecde8be2c656bab23c24cbbf822b7a352b3e
Please review.

[mak]

because i'm unable to build the git-tree(some stuff missing on osx apparently, as autogen.sh fails), i applied the patch from git to 4.7.5.3. patch worked, however now i'm seeing:

mc: 2024G/1493G (135%) on my 5.5tb disc.
os: 1517G/4072G 28%

[andrew_b]

Replying to mak:

mc: 2024G/1493G (135%) on my 5.5tb disc.
os: 1517G/4072G 28%

Would you show exact that values in bytes?

[mak]

• mc

Free space: 1982G/1493G (132%)

• disk utility

Capacity : 6 TB (6,000,699,228,160 Bytes)
Free Space : 4.33 TB (4,327,139,147,776 Bytes)
Used : 1.67 TB (1,673,560,080,384 Bytes)

• df (1k blocks)

5860057840 1634336016 4225721824 28%

…hope this helps.

[andrew_b]

Thanks! Would you attach your config.h here?

[mak]

config.h while builing with brew

[andrew_b]

Hmm, the _FILE_OFFSET_BITS macro is not defined in your config.h. It should be

#define _FILE_OFFSET_BITS 64

which is provided by AC_SYS_LARGEFILE macro. If you run configure without --disable-largefile option, _FILE_OFFSET_BITS should be defined.

BTW, what system do you use?

[mak]

hmm. looks like another perk of running os x(10.6.8 and 10.7.1).

getting this while building with --enable-largefile

checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no

[zap]

I also hit the same problem on a x86_64 FreeBSD system, mc 4.7.5.4.

df output:

Filesystem 1K-blocks        Used      Avail Capacity  Mounted on
/dev/da0   7802994930 1643615732 6159379198    21%    /

Midnight shows: 1778G/16Z

Applied the above git patch and recompiled.

Now Midnight shows: 5874G/7442G

[andrew_b]

Please test 2338_free_space_calculation branch. If you cannot get this branch from our repo, you can get the following snapshot: ​http://repo.or.cz/w/midnight-commander.git/snapshot/947be78a1b4070a35461b23ffc7c8bad32740afb.tar.gz

[mak]

sadly i'm not able to build the snapshot.
this, however has nothing to do with the code, it's simply impossible to run autogen.sh successfully. lets call it another limitation of osx.

[andrew_b]

Replying to mak:

it's simply impossible to run autogen.sh successfully.

I've created ​tarball for you.

[mak]

thx for the tarball.

when building i get this error:

make[5]: * No rule to make target changesetfs', needed by all-am'. Stop.
make[4]: * [all-recursive] Error 1
make[3]: * [all-recursive] Error 1
make[2]: * [all-recursive] Error 1
make[1]: * [all-recursive] Error 1
make: * [all] Error 2

i also attached the config.h file.

[andrew_b]

Ah, sorry. This bug was fixed in master after created branch. I think, build with --disable-vfs-extfs option as a quick workaround can help you.
I've created ​another tarball. Please test it.

[mak]

build worked, but after starting it crashed instantly:

mc(61865,0x7fff70040cc0) malloc: * error for object 0x10201d048: pointer being freed was not allocated
* set a breakpoint in malloc_error_break to debug
Abort trap

[andrew_b]

Replying to mak:

build worked, but after starting it crashed instantly:

mc(61865,0x7fff70040cc0) malloc: * error for object 0x10201d048: pointer being freed was not allocated
* set a breakpoint in malloc_error_break to debug
Abort trap

Would you get a backtrace using following ​instructions?

[mak]

del

[andrew_b]

There is no debug info here. If you use gcc, run

make CFLAGS='-g -ggdb3'

[mak]

sorry 'bout that:

bt

#0  0x00007fff8044b0b6 in __kill ()
#1  0x00007fff804eb9f6 in abort ()
#2  0x00007fff80403195 in free ()
#3  0x0000000100093cf7 in free_mount_entry (me=0x101301490) at mountlist.c:295
#4  0x00000001000940c7 in free_my_statfs () at mountlist.c:1387
#5  0x00000001000940f4 in init_my_statfs () at mountlist.c:1400
#6  0x0000000100086694 in show_free_space (panel=0x10180f200) at panel.c:1032
#7  0x0000000100086ce1 in show_dir (panel=0x10180f200) at panel.c:1123
#8  0x00000001000872d9 in paint_frame (panel=0x10180f200) at panel.c:1306
#9  0x000000010008aa7f in panel_callback (w=0x10180f200, msg=WIDGET_DRAW, parm=0) at panel.c:3003
#10 0x0000000100046e39 in send_message (w=0x10180f200, msg=WIDGET_DRAW, parm=0) at widget-common.h:132
#11 0x0000000100046e07 in dlg_broadcast_msg_to (h=0x101301190, msg=WIDGET_DRAW, reverse=1, flags=0) at dialog.c:145
#12 0x0000000100048316 in dlg_broadcast_msg (h=0x101301190, msg=WIDGET_DRAW, reverse=1) at dialog.c:901
#13 0x0000000100048717 in dlg_redraw (h=0x101301190) at dialog.c:1060
#14 0x00000001000487f1 in init_dlg (h=0x101301190) at dialog.c:1097
#15 0x0000000100048942 in run_dlg (h=0x101301190) at dialog.c:1154
#16 0x00000001000806c6 in create_panels_and_run_mc () at midnight.c:897
#17 0x0000000100081f0a in do_nc () at midnight.c:1649
#18 0x0000000100007aa1 in main (argc=1, argv=0x7fff5fbfeb38) at main.c:510

bt full

#0  0x00007fff8044b0b6 in __kill ()
No symbol table info available.
#1  0x00007fff804eb9f6 in abort ()
No symbol table info available.
#2  0x00007fff80403195 in free ()
No symbol table info available.
#3  0x0000000100093cf7 in free_mount_entry (me=0x101301490) at mountlist.c:295
No locals.
#4  0x00000001000940c7 in free_my_statfs () at mountlist.c:1387
	next = (struct mount_entry *) 0x1013014c0
#5  0x00000001000940f4 in init_my_statfs () at mountlist.c:1400
No locals.
#6  0x0000000100086694 in show_free_space (panel=0x10180f200) at panel.c:1032
	rpath = "??_?\000\000\000?30\001\001\000\000\000\000TR\000\001\000\000\000\000\020R\000\001\000\000\000\b", '\0' <repeats 15 times>, "`?_?\000\000 \r@??\000\000??_?\000\000??\003\000\001\000\000\000????\000\000\000\000?\000\000\000\000\000\000\000\001\000\000\000\b\000\000\000\001\000\000\000\001\000\000\000\001\000\000\000\000\000\000\000??\003\000 \000\001\000P?_?\000\000m?\003\000\001\000\000\000\001\000\000\000?\000\000\000?", '\0' <repeats 16 times>, "\020R\000\001\000\000\000?E\003p?\000\000\000\000\000\000\000\000\000\000??_?\000\000?\t@??\000\000??"...
	vpath = (vfs_path_t *) 0x101303090
	myfs_stats = {
  type = -1, 
  typename = 0x101819d38 "hfs", 
  mpoint = 0x1013013b0 "/Volumes/data", 
  device = 0x1013013a0 "/dev/disk1s2", 
  avail = 98268964, 
  total = 732238672, 
  nfree = 24567241, 
  nodes = 183059666
}
	old_cwd = 0x101301620 "/Users/mak/_get"
#7  0x0000000100086ce1 in show_dir (panel=0x10180f200) at panel.c:1123
	tmp = (gchar *) 0x101303340 "\n"
#8  0x00000001000872d9 in paint_frame (panel=0x10180f200) at panel.c:1306
	side = 0
	width = 0
	format_txt = (GString *) 0x1a00621b00
#9  0x000000010008aa7f in panel_callback (w=0x10180f200, msg=WIDGET_DRAW, parm=0) at panel.c:3003
	panel = (WPanel *) 0x10180f200
	bb = (WButtonBar *) 0x101301190
#10 0x0000000100046e39 in send_message (w=0x10180f200, msg=WIDGET_DRAW, parm=0) at widget-common.h:132
No locals.
#11 0x0000000100046e07 in dlg_broadcast_msg_to (h=0x101301190, msg=WIDGET_DRAW, reverse=1, flags=0) at dialog.c:145
	w = (Widget *) 0x10180f200
	p = (GList *) 0x10061f2e0
	first = (GList *) 0x1006212a0
#12 0x0000000100048316 in dlg_broadcast_msg (h=0x101301190, msg=WIDGET_DRAW, reverse=1) at dialog.c:901
No locals.
#13 0x0000000100048717 in dlg_redraw (h=0x101301190) at dialog.c:1060
No locals.
#14 0x00000001000487f1 in init_dlg (h=0x101301190) at dialog.c:1097
No locals.
#15 0x0000000100048942 in run_dlg (h=0x101301190) at dialog.c:1154
No locals.
#16 0x00000001000806c6 in create_panels_and_run_mc () at midnight.c:897
No locals.
#17 0x0000000100081f0a in do_nc () at midnight.c:1649
	midnight_colors = {9, 9, 9, 9, 9}
#18 0x0000000100007aa1 in main (argc=1, argv=0x7fff5fbfeb38) at main.c:510
	error = (GError *) 0x0
	isInitialized = 1

[andrew_b]

Thanks! Please test ​this tarball.

[mak]

thx again.

build worked and i'm seeing the correct values in the bottom bar.

mc:
521G/5589G (9%)

os:
521G/5589G 91%

[andrew_b]

Initial changeset: b9f03c63d80445c4b74dd0c52c2a9a07ee547f36

[andrew_b]

Merged to master.
changeset:d7ed6615ef980daf84d715d3ea32137f9c9007fd

git log --pretty=oneline bbbb918..d7ed661