Ticket #3250: mc-3250-viewer-rewrite-v1.patch

diff --git a/AUTHORS b/AUTHORS
index bb85c83..60ef7f7 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -64,6 +64,7 @@ Egmont Koblinger <egmont@gmail.com>
         Support of extended mouse clicks beyond 223 column
         Support of bracketed paste mode of xterm
                 (http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#Bracketed%20Paste%20Mode)
+        Rewritten viewer
 
 Erwin van Eijk <wabbit@corner.iaf.nl>
 
diff --git a/src/viewer/Makefile.am b/src/viewer/Makefile.am
index 53bc7a4..0602084 100644
--- a/src/viewer/Makefile.am
+++ b/src/viewer/Makefile.am
@@ -3,6 +3,7 @@ noinst_LTLIBRARIES = libmcviewer.la
 
 libmcviewer_la_SOURCES = \
        actions_cmd.c \
+       ascii.c \
        coord_cache.c \
        datasource.c \
        dialogs.c \
@@ -16,7 +17,6 @@ libmcviewer_la_SOURCES = \
        mcviewer.h \
        move.c \
        nroff.c \
-       plain.c \
        search.c
 
 AM_CPPFLAGS = -I$(top_srcdir) $(GLIB_CFLAGS) $(PCRE_CPPFLAGS)
diff --git a/src/viewer/actions_cmd.c b/src/viewer/actions_cmd.c
index c4d52b7..44c70cd 100644
--- a/src/viewer/actions_cmd.c
+++ b/src/viewer/actions_cmd.c
@@ -512,6 +512,8 @@ mcview_execute_cmd (mcview_t * view, unsigned long command)
         break;
     case CK_Bookmark:
         view->dpy_start = view->marks[view->marker];
+        view->dpy_paragraph_skip_lines = 0;  // TODO: remember this value in the marker???
+        view->dpy_wrap_dirty = TRUE;
         view->dirty++;
         break;
 #ifdef HAVE_CHARSET
@@ -594,6 +596,7 @@ mcview_adjust_size (WDialog * h)
     widget_set_size (WIDGET (view), 0, 0, LINES - 1, COLS);
     widget_set_size (WIDGET (b), LINES - 1, 0, 1, COLS);
 
+    view->dpy_wrap_dirty = TRUE;
     mcview_compute_areas (view);
     mcview_update_bytes_per_line (view);
 }
diff --git a/src/viewer/ascii.c b/src/viewer/ascii.c
new file mode 100644
index 0000000..ea83ab1
--- /dev/null
+++ b/src/viewer/ascii.c
@@ -0,0 +1,899 @@
+/*
+   Internal file viewer for the Midnight Commander
+   Function for plain view
+
+   Copyright (C) 1994-2014
+   Free Software Foundation, Inc.
+
+   Written by:
+   Miguel de Icaza, 1994, 1995, 1998
+   Janne Kukonlehto, 1994, 1995
+   Jakub Jelinek, 1995
+   Joseph M. Hinkle, 1996
+   Norbert Warmuth, 1997
+   Pavel Machek, 1998
+   Roland Illig <roland.illig@gmx.de>, 2004, 2005
+   Slava Zanko <slavazanko@google.com>, 2009
+   Andrew Borodin <aborodin@vmail.ru>, 2009-2014
+   Ilia Maslakov <il.smind@gmail.com>, 2009
+   Rewritten almost from scratch by:
+   Egmont Koblinger <egmont@gmail.com>, 2014
+
+   This file is part of the Midnight Commander.
+
+   The Midnight Commander is free software: you can redistribute it
+   and/or modify it under the terms of the GNU General Public License as
+   published by the Free Software Foundation, either version 3 of the License,
+   or (at your option) any later version.
+
+   The Midnight Commander is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+   ------------------------------------------------------------------------------------------------
+
+   The viewer is implemented along the following design principles:
+
+   Terminology: a "paragraph" is the text between two adjacent newline characters. A "line" or
+   "row" is a visual row on the screen. In wrap mode, the viewer formats a paragraph into one or
+   more lines.
+
+   The parser-formatter is designed to be stateless across paragraphs. This is so that we can walk
+   backwards without having to reparse the whole file (although we still need to reparse and
+   reformat the whole paragraph, but it's a lot better).
+
+   The parser-formatter, however, needs to carry a state across lines. Currently this state
+   contains:
+
+    - The logical column (as if we didn't wrap). This is used for handling TAB characters after a
+      wordwrap consistently with less.
+
+    - Whether the last nroff character was bold or underlined. This is used for displaying the
+      ambiguous _\b_ sequence consistently with less.
+
+    - Whether the desired way of displaying a lonely combining accent or spacing mark is to place
+      it over a dotted circle (we do this at the beginning of the paragraph of after a TAB), or to
+      ignore the combining char and show replacement char for the spacing mark (we do this if e.g.
+      too many of these were encountered and hence we don't glue them with their base character).
+
+    - (This state needs to be expanded if e.g. we decide to print verbose replacement characters
+      (e.g. "<U+0080>") and allow these to wrap around lines.)
+
+   The state also contains the file offset, as it doesn't make sense to ever
+   know the state without knowing the corresponding offset.
+
+   The state depends on various settings (viewer width, encoding, nroff mode, charwrap or wordwrap
+   mode (if we'll have that one day) etc., needs to be recomputed if any of these changes.
+
+   Walking forwards is usually relatively easy both in the file and on the screen. Walking
+   backwards within a paragraph would only be possible in some special cases and even then it would
+   be painful, so we always walk back to the beginning of the paragraph and reparse-reformat from
+   there.
+
+   (Walking back within a line in the file would have at least the following difficulties: handling
+   the parser state; processing invalid UTF-8; processing invalid nroff (e.g. what is "_\bA\bA"?).
+   Walking back on the display: we wouldn't know where to display the last line of a paragraph, or
+   where to display a line if its following line starts with a wide (CJK or Tab) character. Long
+   story short: just forget this approach.)
+
+   Most important variables:
+
+    - dpy_start: Both in unwrap and wrap modes this points to the beginning of the topmost
+      displayed paragraph.
+
+    - dpy_text_column: Only in unwrap mode, an additional horizontal scroll.
+
+    - dpy_paragraph_skip_lines: Only in wrap mode, an additional vertical scroll (the number of
+      lines that are scrolled off at the top from the topmost paragraph).
+
+    - dpy_state_top: Only in wrap mode, the offset and parser-formatter state at the line where
+      displaying the file begins is cached here.
+
+    - dpy_wrap_dirty: If some parameter has changed that makes it necessary to reparse-redisplay
+      the topmost paragraph.
+
+   In wrap mode, the three variables "dpy_start", "dpy_paragraph_skip_lines" and "dpy_state_top"
+   are kept consistent. Think of the first two as the ones describing the position, and the third
+   as a cached value for better performance so that we don't need to wrap the invisible beginning
+   of the topmost paragraph over and over again. The third value needs to be recomputed each time a
+   parameter that influences parsing or displaying the file (e.g. width of screen, encoding, nroff
+   mode) changes, this is signaled by "dpy_wrap_dirty" to force recomputing "dpy_state_top" (and
+   clamp "dpy_paragraph_skip_lines" if necessary).
+
+   blahblahblah...
+
+   Goals: Always display simple scripts, double wide (CJK), combining accents and spacing marks
+   (often used e.g. in Devanagari) perfectly. Make the arrow keys always work correctly.
+
+   Absolutely non-goal: RTL.
+
+   ------------------------------------------------------------------------------------------------
+
+   Help integration
+
+   I'm planning to port the help viewer to this codebase.
+
+   Splitting at sections would still happen in the help viewer. It would either copy a section, or
+   set force_max and a similar force_min to limit displaying to one section only.
+
+   Parsing the help format would go next to the nroff parser. The colors, alternate character set,
+   and emitting the version number would go to the "state". (The version number would be
+   implemented by emitting remaining characters of a buffer in the "state" one by one, without
+   advancing in the file position.)
+
+   The active link would be drawn similarly to the search highlight. Other than that, the viewer
+   wouldn't care about links (except for their color). help.c would keep track of which one is
+   highlighted, how to advance to the next/prev on an arrow, how the scroll offset needs to be
+   adjusted when moving, etc.
+
+   Add wrapping at word boundaries to where wrapping at char boundaries happen now.
+ */
+
+#include <config.h>
+#include <wchar.h>   // shouldn't depend on this
+
+#include "lib/global.h"
+#include "lib/tty/tty.h"
+#include "lib/skin.h"
+#include "lib/util.h"           /* is_printable() */
+#ifdef HAVE_CHARSET
+#include "lib/charsets.h"
+#endif
+
+#include "src/setup.h"          /* option_tab_spacing */
+
+#include "internal.h"
+
+/*** global variables ****************************************************************************/
+
+/*** file scope macro definitions ****************************************************************/
+
+#define BASE_CHARACTER_FOR_LONELY_COMBINING 0x25CC  /* dotted circle */
+#define MAX_COMBINING_CHARS 4  /* both slang and ncurses support exactly 4 */
+
+// I think space looks better than arrows. Still, use arrows while developing for better debugging.
+// Final version could take it from the skin.
+#define PARTIAL_CJK_AT_LEFT_MARGIN  0x25C2  // ' '  /* '<' doesn't look that good */
+#define PARTIAL_CJK_AT_RIGHT_MARGIN 0x25B8  // ' '  /* '>' doesn't look that good */
+
+/*
+ * Wrap mode: This is for safety so that jumping to the end of file (which already includes
+ * scrolling back by a page) and then walking backwards is reasonably fast, even if the file is
+ * extremely large and consists of maybe full zeros or something like that. If there's no newline
+ * found within this limit, just start displaying from there and see what happens. We might get
+ * some displaying parameteres (most importantly the columns) incorrect, but at least will show the
+ * file without spinning the CPU for ages. When scrolling back to that point, the user might see a
+ * garbled first line (even starting with an invalid partial UTF-8), but then walking back by yet
+ * another line should fix it.
+ *
+ * Unwrap mode: This is not used, we wouldn't be able to do anything reasonable without walking
+ * back a whole paragraph (well, view->data_area.height paragraphs actually).
+ */
+#define MAX_BACKWARDS_WALK_IN_PARAGRAPH (100 * 1000)
+
+/*** file scope type declarations ****************************************************************/
+
+/*** file scope variables ************************************************************************/
+
+/*** file scope functions ************************************************************************/
+
+// TODO: These methods shouldn't be necessary, see ticket 3257
+
+static int
+mcview_wcwidth (const mcview_t * view, int c)
+{
+#ifdef HAVE_CHARSET
+       if (view->utf8) {
+               if (g_unichar_iswide(c))
+                       return 2;
+               if (g_unichar_iszerowidth(c))
+                       return 0;
+       }
+#endif /* HAVE_CHARSET */
+       return 1;
+}
+
+static gboolean
+mcview_ismark (const mcview_t * view, int c)
+{
+#ifdef HAVE_CHARSET
+       if (view->utf8)
+               return g_unichar_ismark(c);
+#endif /* HAVE_CHARSET */
+       return FALSE;
+}
+
+/* actually is_non_spacing_mark_or_enclosing_mark */
+static gboolean
+mcview_is_non_spacing_mark (const mcview_t * view, int c)
+{
+#ifdef HAVE_CHARSET
+       if (view->utf8) {
+               GUnicodeType type = g_unichar_type(c);
+               return type == G_UNICODE_NON_SPACING_MARK || type == G_UNICODE_ENCLOSING_MARK;
+       }
+#endif /* HAVE_CHARSET */
+       return FALSE;
+}
+
+#if 0
+static gboolean
+mcview_is_spacing_mark (const mcview_t * view, int c)
+{
+#ifdef HAVE_CHARSET
+       if (view->utf8) {
+               return g_unichar_type(c) == G_UNICODE_SPACING_MARK;
+       }
+#endif /* HAVE_CHARSET */
+       return FALSE;
+}
+#endif /* 0 */
+
+static gboolean
+mcview_isprint (const mcview_t * view, int c)
+{
+#ifdef HAVE_CHARSET
+       if (!view->utf8)
+               c = convert_from_8bit_to_utf_c ((unsigned char) c, view->converter);
+       return g_unichar_isprint(c);
+#endif /* HAVE_CHARSET */
+       // TODO this is very-very buggy by design: ticket 3257 comments 0-1
+       return is_printable (c);
+}
+
+static int
+mcview_char_display (const mcview_t * view, int c, char *s)
+{
+#ifdef HAVE_CHARSET
+       if (mc_global.utf8_display) {
+               if (!view->utf8)
+                       c = convert_from_8bit_to_utf_c ((unsigned char) c, view->converter);
+               if (!g_unichar_isprint (c))
+                       c = '.';
+               return g_unichar_to_utf8(c, s);
+       } else if (view->utf8) {
+               if (g_unichar_iswide(c)) {
+                       s[0] = s[1] = '.';
+                       return 2;
+               }
+               if (g_unichar_iszerowidth(c))
+                       return 0;
+               // TODO the is_printable check below will be broken for this
+               c = convert_from_utf_to_current_c (c, view->converter);
+       } else {
+               // TODO the is_printable check below will be broken for this
+               c = convert_to_display_c (c);
+       }
+#endif /* HAVE_CHARSET */
+       // TODO this is very-very buggy by design: ticket 3257 comments 0-1
+       if (!is_printable (c))
+               c = '.';
+       *s = c;
+       return 1;
+}
+
+/* --------------------------------------------------------------------------------------------- */
+
+/*
+ * Just for convenience, a common interface in front of mcview_get_utf and mcview_get_byte, so that
+ * the caller doesn't have to care about utf8 vs 8-bit modes.
+ *
+ * Normally: stores c, updates state, returns TRUE.
+ * At EOF: state is unchanged, c is undefined, returns FALSE.
+ *
+ * Also, temporary hack: handle force_max here.
+ * TODO: move it to lower layers (datasource.c)?
+ */
+static gboolean
+mcview_get_next_char (mcview_t * view, mcview_state_machine_t * state, int *c)
+{
+       gboolean result;
+       int bytes_consumed;
+
+       /* Pretend EOF if we reached force_max */
+       if (view->force_max >= 0 && state->offset >= view->force_max) {
+               return FALSE;
+       }
+#ifdef HAVE_CHARSET
+        if (view->utf8)
+        {
+            *c = mcview_get_utf (view, state->offset, &bytes_consumed, &result);
+            if (!result)
+               return FALSE;
+            /* Pretend EOF if we crossed force_max */
+            if (view->force_max >= 0 && state->offset + bytes_consumed > view->force_max) {
+                return FALSE;
+            }
+            state->offset += bytes_consumed;
+            return TRUE;
+        }
+#endif /* HAVE_CHARSET */
+        if (!mcview_get_byte (view, state->offset, c))
+           return FALSE;
+        state->offset++;
+        return TRUE;
+}
+
+/*
+ * This function parses the next nroff character and gives it to you along with its desired color,
+ * so you never have to care about nroff again.
+ *
+ * The nroff mode does the backspace trick for every single character (Unicode codepoint). At least
+ * that's what the GNU groff 1.22 package produces, and that's what less 458 expects. For
+ * double-wide characters (CJK), still only a single backspace is emitted. For combining accents
+ * and such, the print-backspace-print step is repeated for the base character and then for each
+ * accent separately.
+ *
+ * So, the right place for this layer is after the bytes are interpreted in UTF-8, but before
+ * joining a base character with its combining accents.
+ *
+ * Normally: stores c and color, updates state, returns TRUE.
+ * At EOF: state is unchanged, c and color are undefined, returns FALSE.
+ *
+ * color can be null if the caller doesn't care.
+ */
+static gboolean
+mcview_get_next_maybe_nroff_char (mcview_t * view, mcview_state_machine_t * state, int *c, int *color)
+{
+       mcview_state_machine_t state_after_nroff;
+       int c2, c3;
+
+       if (color) *color = VIEW_NORMAL_COLOR;
+
+       if (!view->text_nroff_mode)
+               return mcview_get_next_char (view, state, c);
+
+       if (!mcview_get_next_char (view, state, c))
+               return FALSE;
+       /* Don't allow nroff formatting around CR, LF, TAB or other special chars */
+       if (!mcview_isprint(view, *c))
+               return TRUE;
+
+       state_after_nroff = *state;
+
+       if (!mcview_get_next_char (view, &state_after_nroff, &c2))
+               return TRUE;
+       if (c2 != '\b')
+               return TRUE;
+
+       if (!mcview_get_next_char (view, &state_after_nroff, &c3))
+               return TRUE;
+       if (!mcview_isprint(view, c3))
+               return TRUE;
+
+       if (*c == '_' && c3 == '_') {
+               *state = state_after_nroff;
+               if (color) *color = state->nroff_underscore_is_underlined ? VIEW_UNDERLINED_COLOR : VIEW_BOLD_COLOR;
+               return TRUE;
+       } else if (*c == c3) {
+               *state = state_after_nroff;
+               state->nroff_underscore_is_underlined = FALSE;
+               if (color) *color = VIEW_BOLD_COLOR;
+               return TRUE;
+       } else if (*c == '_') {
+               *c = c3;
+               *state = state_after_nroff;
+               state->nroff_underscore_is_underlined = TRUE;
+               if (color) *color = VIEW_UNDERLINED_COLOR;
+               return TRUE;
+       } else {
+               return TRUE;
+       }
+}
+
+/*
+ * Get one base character, along with its combining or spacing mark characters.
+ *
+ * (A spacing mark is a character that extends the base character's width 1 into a combined
+ * character of width 2, yet these two character cells should not be separated. E.g. Devanagari
+ * <U+0939><U+094B>.)
+ *
+ * This method exists mainly for two reasons. One is to be able to tell if we fit on the current
+ * line or need to wrap to the next one. The other is that both slang and ncurses seem to require
+ * that the character and its combining marks are printed in a single call (or is it just a
+ * limitation of mc's wrapper to them?).
+ *
+ * For convenience, this method takes care of converting CR or CR+LF into LF.
+ * TODO this should probably happen later, when displaying the file?
+ *
+ * Normally: stores cs and color, updates state, returns >= 1 (entries in cs).
+ * At EOF: state is unchanged, cs and color are undefined, returns 0.
+ *
+ * @param view ...
+ * @param state the parser-formatter state machine's state, updated
+ * @param cs store the characters here
+ * @param clen the room available in cs (that is, at most clen-1 combining marks are allowed), must
+ *   be at least 2
+ * @param color if non-NULL, store the color here, taken from the first codepoint's color
+ * @return the number of entries placed in cs, or 0 on EOF
+ */
+static int
+mcview_next_complex_char (mcview_t * view, mcview_state_machine_t * state, int *cs, int clen, int *color)
+{
+       int i = 1;
+       mcview_state_machine_t state_after_combining;
+
+       if (!mcview_get_next_maybe_nroff_char (view, state, cs, color))
+               return 0;
+
+       /* Process \r and \r\n newlines. */
+       if (cs[0] == '\r') {
+               int cnext;
+               mcview_state_machine_t state_after_crlf = *state;
+               if (mcview_get_next_maybe_nroff_char (view, &state_after_crlf, &cnext, NULL) && cnext == '\n')
+                       *state = state_after_crlf;
+               cs[0] = '\n';
+               return 1;
+       }
+
+       /* We don't want combining over non-printable characters. This includes '\n' and '\t' too. */
+       if (!mcview_isprint(view, cs[0]))
+               return 1;
+
+       if (mcview_ismark(view, cs[0])) {
+               if (!state->print_lonely_combining) {
+                       /* First character is combining. Either just return it, ... */
+                       return 1;
+               } else {
+                       /* or place this (and subsequent combining ones) over a dotted circle. */
+                       cs[1] = cs[0];
+                       cs[0] = BASE_CHARACTER_FOR_LONELY_COMBINING;
+                       i = 2;
+               }
+       }
+
+       if (mcview_wcwidth(view, cs[0]) == 2) {
+               /* Don't allow combining or spacing mark for wide characters, is this okay? */
+               return 1;
+       }
+
+       /* Look for more combining chars. Either at most clen-1 zero-width combining chars,
+        * or at most 1 spacing mark. Is this logic correct? */
+       for (; i < clen; i++) {
+               state_after_combining = *state;
+               if (!mcview_get_next_maybe_nroff_char (view, &state_after_combining, &cs[i], NULL))
+                       return i;
+               if (!mcview_ismark(view, cs[i]) || !mcview_isprint(view, cs[i]))
+                       return i;
+               if (g_unichar_type(cs[i]) == G_UNICODE_SPACING_MARK) {  // is this the right check?
+                       /* Only allow as the first combining char. Stop processing in either case. */
+                       if (i == 1) {
+                               *state = state_after_combining;
+                               i++;
+                       }
+                       return i;
+               }
+               *state = state_after_combining;
+       }
+       return i;
+}
+
+/*
+ * Parse, format and possibly display one visual line of text.
+ *
+ * Formatting starts at the given "state" (which encodes the file offset and parser and formatter's
+ * internal state). In unwrap mode, this should point to the beginning of the paragraph with the
+ * default state, the additional horizontal scrolling is added here. In wrap mode, this should
+ * point to the beginning of the line, with the proper state at that point.
+ *
+ * In wrap mode, if a line ends in a newline, it is consumed, even if it's exactly at the right
+ * edge. In unwrap mode, the whole remaining line, including the newline is consumed. Displaying
+ * the next line should start at "state"'s new value, or if we displayed the bottom line then
+ * state->offset tells the file offset to be shown in the top bar.
+ *
+ * If "row" is offscreen, don't actually display the line but still update "state" and return the
+ * proper value. This is used by mcview_wrap_move_down to advance in the file.
+ *
+ * @param view ...
+ * @param state the parser-formatter state machine's state, updated
+ * @param row print to this row
+ * @param paragraph_ended store TRUE if paragraph ended by newline or EOF, FALSE if wraps to next
+ *   line
+ * @return the number of rows, that is, 0 if we were already at EOF, otherwise 1
+ */
+static int
+mcview_display_line (mcview_t * view, mcview_state_machine_t * state, int row, gboolean *paragraph_ended)
+{
+       const screen_dimen left = view->data_area.left;
+       const screen_dimen top = view->data_area.top;
+       const screen_dimen width = view->data_area.width;
+       const screen_dimen height = view->data_area.height;
+       off_t dpy_text_column = view->text_wrap_mode ? 0 : view->dpy_text_column;  // actually maybe we shouldn't allow view->data_area.left to be any different
+       screen_dimen col = 0;
+       int color;
+       int cs[1 + MAX_COMBINING_CHARS];
+       int n;
+       char str[(1 + MAX_COMBINING_CHARS) * UTF8_CHAR_LEN + 1];
+       int charwidth;
+       int i, j;
+       mcview_state_machine_t state_saved;
+
+       if (paragraph_ended) *paragraph_ended = TRUE;
+
+       if (!view->text_wrap_mode && col >= dpy_text_column + width) {
+               /* Optimization: Fast forward to the end of the line, rather than carefully
+                * parsing and then not actually displaying it. */
+               off_t eol = mcview_eol(view, state->offset, mcview_get_filesize (view));
+               int retval = eol > state->offset ? 1 : 0;
+               mcview_state_machine_init (state, eol);
+               return retval;
+       }
+
+       while (1) {
+               state_saved = *state;
+               n = mcview_next_complex_char (view, state, cs, 1 + MAX_COMBINING_CHARS, &color);
+               if (n == 0)
+                       return col > 0 ? 1 : 0;
+
+               if (view->search_start <= state->offset && state->offset < view->search_end)
+                       color = SELECTED_COLOR;
+
+               if (cs[0] == '\n') {
+                       /* New line: reset all formatting state for the next paragraph. */
+                       mcview_state_machine_init (state, state->offset);
+                       return 1;
+               }
+
+               if (mcview_is_non_spacing_mark(view, cs[0])) {
+                       /* Lonely combining character. Probably leftover after too many combining chars. Just ignore. */
+                       continue;
+               }
+
+               /* Nonprintable, or lonely spacing mark */
+               if ((!mcview_isprint(view, cs[0]) || mcview_ismark(view, cs[0])) && cs[0] != '\t')
+                       cs[0] = '.';
+
+               charwidth = 0;
+               for (i = 0; i < n; i++) {
+                       charwidth += mcview_wcwidth(view, cs[i]);
+               }
+
+               /* Adjust the width for TAB. It's handled below along with the normal characters,
+                * so that it's wrapped consistently with them, and is painted with the proper
+                * attributes (although currently it can't have a special color). */
+               if (cs[0] == '\t') {
+                       charwidth = option_tab_spacing - state->unwrapped_column % option_tab_spacing;
+                       state->print_lonely_combining = TRUE;
+               } else {
+                       state->print_lonely_combining = FALSE;
+               }
+
+               /* In wrap mode only: We're done with this row if the complex character wouldn't
+                * fit. Except if at the first column, because then it wouldn't fit in the next row
+                * either. In this extreme case let the unwrapped code below do its best to display
+                * it. */
+               if (view->text_wrap_mode && (off_t) col + charwidth > dpy_text_column + width && col > 0) {
+                       *state = state_saved;
+                       if (paragraph_ended) *paragraph_ended = FALSE;
+                       return 1;
+               }
+
+               /* Display, unless outside of the viewport. */
+               if (row >= 0 && row < (int) height) {
+                       if ((off_t) col >= dpy_text_column &&
+                           (off_t) col + charwidth <= dpy_text_column + width) {
+                               /* The complex character fits entirely in the viewport. Print it. */
+                               tty_setcolor(color);
+                               widget_move (view, top + row, left + ((off_t) col - dpy_text_column));
+                               if (cs[0] == '\t') {
+                                       for (i = 0; i < charwidth; i++)
+                                               tty_print_char(' ');
+                               } else {
+                                       j = 0;
+                                       for (i = 0; i < n; i++) {
+                                               j += mcview_char_display(view, cs[i], str + j);
+                                       }
+                                       str[j] = '\0';
+                                       /* This is probably a bug in our tty layer, but tty_print_string
+                                        * normalizes the string, whereas tty_printf doesn't. Don't normalize,
+                                        * since we handle combining characters ourselves correctly, it's
+                                        * better if they are copy-pasted correctly. Ticket 3255. */
+                                       tty_printf ("%s", str);
+                               }
+                       } else if ((off_t) col < dpy_text_column &&
+                                  (off_t) col + charwidth > dpy_text_column) {
+                               /* The complex character would cross the left edge of the viewport.
+                                * This cannot happen with wrap mode. Print replacement character(s),
+                                * or spaces with the correct attributes for partial Tabs. */
+                               tty_setcolor(color);
+                               for (i = dpy_text_column; i < (off_t) col + charwidth && i < dpy_text_column + width; i++) {
+                                       widget_move (view, top + row, left + (i - dpy_text_column));
+                                       tty_print_anychar(cs[0] == '\t' ? ' ' : PARTIAL_CJK_AT_LEFT_MARGIN);
+                               }
+                       } else if ((off_t) col < dpy_text_column + width &&
+                                  (off_t) col + charwidth > dpy_text_column + width) {
+                               /* The complex character would cross the right edge of the viewport
+                                * and we're not wrapping. Print replacement character(s),
+                                * or spaces with the correct attributes for partial Tabs. */
+                               tty_setcolor(color);
+                               for (i = col; i < dpy_text_column + width; i++) {
+                                       widget_move (view, top + row, left + (i - dpy_text_column));
+                                       tty_print_anychar(cs[0] == '\t' ? ' ' : PARTIAL_CJK_AT_RIGHT_MARGIN);
+                               }
+                       }
+               }
+
+               col += charwidth;
+               state->unwrapped_column += charwidth;
+
+               if (!view->text_wrap_mode && col >= dpy_text_column + width) {
+                       /* Optimization: Fast forward to the end of the line, rather than carefully
+                        * parsing and then not actually displaying it. */
+                       off_t eol = mcview_eol(view, state->offset, mcview_get_filesize (view));
+                       mcview_state_machine_init (state, eol);
+                       return 1;
+               }
+       }
+}
+
+/*
+ * Parse, format and possibly display one paragraph (perhaps not from the beginning).
+ *
+ * Formatting starts at the given "state" (which encodes the file offset and parser and formatter's
+ * internal state). In unwrap mode, this should point to the beginning of the paragraph with the
+ * default state, the additional horizontal scrolling is added here. In wrap mode, this may point
+ * to the beginning of the line within a paragraph (to display the partial paragraph at the top),
+ * with the proper state at that point.
+ *
+ * Displaying the next paragraph should start at "state"'s new value, or if we displayed the bottom
+ * line then state->offset tells the file offset to be shown in the top bar.
+ *
+ * If "row" is negative, don't display the first abs(row) lines and display the rest from the top.
+ * This was a nice idea but it's now unused :)
+ *
+ * If "row" is too large, don't display the paragraph at all but still return the number of lines.
+ * This is used when moving upwards.
+ *
+ * @param view ...
+ * @param state the parser-formatter state machine's state, updated
+ * @param row print starting at this row
+ * @return the number of rows the paragraphs is wrapped to, that is, 0 if we were already at EOF,
+ *   otherwise 1 in unwrap mode, >= 1 in wrap mode. We stop when reaching the bottom of the
+ *   viewport, it's not counted how many more lines the paragraph would occupy
+ */
+static int
+mcview_display_paragraph (mcview_t * view, mcview_state_machine_t * state, int row)
+{
+       const screen_dimen height = view->data_area.height;
+       int lines = 0;
+       gboolean paragraph_ended;
+
+       while (1) {
+               lines += mcview_display_line(view, state, row, &paragraph_ended);
+               if (paragraph_ended)
+                       return lines;
+
+               if (row < (int) height) {
+                       row++;
+                       /* stop if bottom of screen reached */
+                       if (row >= (int) height)
+                               return lines;
+               }
+       }
+}
+
+/*
+ * Recompute dpy_state_top from dpy_start and dpy_paragraph_skip_lines. Clamp
+ * dpy_paragraph_skip_lines if necessary.
+ *
+ * This method should be called in wrap mode after changing one of the parsing or formatting
+ * properties (e.g. window width, encoding, nroff), or when switching to wrap mode from unwrap or
+ * hex.
+ *
+ * If we stayed within the same paragraph then try to keep the vertical offset within that
+ * paragraph as well. It might happen though that the paragraph became shorter than our desired
+ * vertical position, in that case move to its last row.
+ */
+static void
+mcview_wrap_fixup (mcview_t * view)
+{
+       mcview_state_machine_t state_prev;
+       gboolean paragraph_ended;
+       int lines = view->dpy_paragraph_skip_lines;
+
+       if (!view->dpy_wrap_dirty)
+               return;
+       view->dpy_wrap_dirty = FALSE;
+
+       view->dpy_paragraph_skip_lines = 0;
+       mcview_state_machine_init (&view->dpy_state_top, view->dpy_start);
+
+       while (lines--) {
+               state_prev = view->dpy_state_top;
+               if (!mcview_display_line (view, &view->dpy_state_top, -1, &paragraph_ended))
+                       break;
+               if (paragraph_ended) {
+                       view->dpy_state_top = state_prev;
+                       break;
+               }
+               view->dpy_paragraph_skip_lines++;
+       }
+}
+
+/* --------------------------------------------------------------------------------------------- */
+/*** public functions ****************************************************************************/
+/* --------------------------------------------------------------------------------------------- */
+
+/*
+ * In both wrap and unwrap modes, dpy_start points to the beginning of the paragraph.
+ *
+ * In unwrap mode, start displaying from this position, probably applying an additional horizontal
+ * scroll.
+ *
+ * In wrap mode, an additional dpy_paragraph_skip_lines lines are skipped from the top of this
+ * paragraph. dpy_state_top contains the position and parser-formatter state corresponding to the
+ * top left corner so we can just start rendering from here. Unless dpy_wrap_dirty is set in which
+ * case dpy_state_top is invalid and we need to recompute first.
+ */
+void
+mcview_display_text (mcview_t * view)
+{
+       const screen_dimen left = view->data_area.left;
+       const screen_dimen top = view->data_area.top;
+       const screen_dimen height = view->data_area.height;
+       int row;
+       int n;
+       mcview_state_machine_t state;
+       gboolean again;
+
+       do {
+               again = FALSE;
+
+               mcview_display_clean (view);
+               mcview_display_ruler (view);
+
+               if (view->text_wrap_mode) {
+                       mcview_wrap_fixup (view);
+                       state = view->dpy_state_top;
+               } else {
+                       mcview_state_machine_init(&state, view->dpy_start);
+               }
+               row = 0;
+               while (row < (int) height) {
+                       n = mcview_display_paragraph (view, &state, row);
+                       if (n == 0) {
+                               /* This is quite ugly here. In the rare case that displaying didn't
+                                * start at the beginning of the file, yet there are some empty
+                                * lines at the bottom, scroll the file and display again. This
+                                * happens when e.g. the window is made bigger, or the file becomes
+                                * shorter due to charset change or enabling nroff.
+                                *
+                                * TODO: probably set some dirty bit when any of these changes, and
+                                * perform a blind rendering first to see if it would fit, before
+                                * talking to slang/ncurses. (Although they do buffering too, so
+                                * it's not that bad. And that approach would render twice and
+                                * hence be slower in the typical case.)
+                                */
+                               if ((view->text_wrap_mode ? view->dpy_state_top.offset : view->dpy_start) > 0) {
+                                       mcview_ascii_move_up (view, height - row);
+                                       again = TRUE;
+                               }
+                               break;
+                       }
+                       row += n;
+               }
+       } while (again);
+
+       view->dpy_end = state.offset;
+       view->dpy_state_bottom = state;
+
+       if (mcview_show_eof != NULL && mcview_show_eof[0] != '\0') {
+               while (row < (int) height) {
+                       widget_move (view, top + row, left);
+                       // TODO: should make it no wider than the viewport
+                       tty_print_string (mcview_show_eof);
+                       row++;
+               }
+       }
+}
+
+/*
+ * Move down.
+ *
+ * It's very simple. Just invisibly format the next "lines" lines, carefully carrying the formatter
+ * state in wrap mode. But before each step we need to check if we've already hit the end of the
+ * file, in that case we can no longer move. This is done by walking from dpy_state_bottom.
+ *
+ * Note that this relies on mcview_display_text() setting dpy_state_bottom to its correct value
+ * upon rendering the screen contents. So don't call this function from other functions (e.g. at
+ * the bottom of mcview_ascii_move_up()) which invalidate this value.
+ */
+void
+mcview_ascii_move_down (mcview_t * view, off_t lines)
+{
+       gboolean paragraph_ended;
+
+       while (lines--) {
+               /* See if there's still data below the bottom line. If not, we can't scroll any
+                * more. If there is, adjust dpy_state_bottom by imaginarily displaying one more
+                * line there. */
+               if (view->dpy_state_bottom.offset >= mcview_get_filesize (view))
+                       break;
+               mcview_display_line (view, &view->dpy_state_bottom, -1, &paragraph_ended);
+
+               /* Okay, there's enough data. Move by 1 row at the top, too. No need to check for
+                * EOF, that can't happen. */
+               if (!view->text_wrap_mode) {
+                       view->dpy_start = mcview_eol(view, view->dpy_start, mcview_get_filesize (view));
+                       view->dpy_paragraph_skip_lines = 0;
+                       view->dpy_wrap_dirty = TRUE;
+               } else {
+                       mcview_display_line (view, &view->dpy_state_top, -1, &paragraph_ended);
+                       if (paragraph_ended) {
+                               view->dpy_start = view->dpy_state_top.offset;
+                               view->dpy_paragraph_skip_lines = 0;
+                       } else {
+                               view->dpy_paragraph_skip_lines++;
+                       }
+               }
+       }
+}
+
+/*
+ * Move up.
+ *
+ * Unwrap mode: Piece of cake. Wrap mode: If we'd walk back more than the current line offset
+ * within the paragraph, we need to jump back to the previous paragraph and compute its height to
+ * see if we start from that paragraph, and repeat this if necessary. Once we're within the desired
+ * paragraph, we still need to format it from its beginning to know the state.
+ *
+ * See the top of this file for comments about MAX_BACKWARDS_WALK_IN_PARAGRAPH.
+ *
+ * force_max is a nice protection against the rare extreme case that the file underneath us
+ * changes, we don't want to endlessly consume a file of maybe full of zeros upon moving upwards.
+ */
+void
+mcview_ascii_move_up (mcview_t * view, off_t lines)
+{
+       int i;
+
+       if (!view->text_wrap_mode) {
+               while (lines--)
+                       view->dpy_start = mcview_bol(view, view->dpy_start - 1, 0);
+               view->dpy_paragraph_skip_lines = 0;
+               view->dpy_wrap_dirty = TRUE;
+       } else {
+               while (lines > view->dpy_paragraph_skip_lines) {
+                       /* We need to go back to the previous paragraph. */
+                       if (view->dpy_start == 0) {
+                               /* Oops, we're already in the first paragraph. */
+                               view->dpy_paragraph_skip_lines = 0;
+                               mcview_state_machine_init(&view->dpy_state_top, 0);
+                               return;
+                       }
+                       lines -= view->dpy_paragraph_skip_lines;
+                       view->force_max = view->dpy_start;
+                       view->dpy_start = mcview_bol (view, view->dpy_start - 1, view->dpy_start - MAX_BACKWARDS_WALK_IN_PARAGRAPH);
+                       mcview_state_machine_init(&view->dpy_state_top, view->dpy_start);
+                       /* This is a tricky way of denoting that we're at the end of the paragraph.
+                        * Normally we'd jump to the next paragraph and reset paragraph_skip_lines. But for
+                        * walking backwards this is exactly what we need. */
+                       view->dpy_paragraph_skip_lines = mcview_display_paragraph (view, &view->dpy_state_top, view->data_area.height);
+                       view->force_max = -1;
+               }
+
+               /* Okay, we have have dpy_start pointing to the desired paragraph, and we still need to
+                * walk back "lines" lines from the current "dpy_paragraph_skip_lines" offset. We can't do
+                * that, so walk from the beginning of the paragraph. */
+               mcview_state_machine_init(&view->dpy_state_top, view->dpy_start);
+               view->dpy_paragraph_skip_lines -= lines;
+               for (i = 0; i < view->dpy_paragraph_skip_lines; i++)
+                       mcview_display_line (view, &view->dpy_state_top, -1, NULL);
+       }
+}
+
+/* --------------------------------------------------------------------------------------------- */
+
+void
+mcview_state_machine_init (mcview_state_machine_t * state, off_t offset)
+{
+       memset(state, 0, sizeof (*state));
+       state->offset = offset;
+       state->print_lonely_combining = TRUE;
+}
+
+/* --------------------------------------------------------------------------------------------- */
diff --git a/src/viewer/datasource.c b/src/viewer/datasource.c
index 9ec5ab4..97b04c6 100644
--- a/src/viewer/datasource.c
+++ b/src/viewer/datasource.c
@@ -164,7 +164,7 @@ mcview_get_ptr_string (mcview_t * view, off_t byte_index)
 /* --------------------------------------------------------------------------------------------- */
 
 int
-mcview_get_utf (mcview_t * view, off_t byte_index, int *char_width, gboolean * result)
+mcview_get_utf (mcview_t * view, off_t byte_index, int *bytes_consumed, gboolean * result)
 {
     gchar *str = NULL;
     int res = -1;
@@ -172,7 +172,7 @@ mcview_get_utf (mcview_t * view, off_t byte_index, int *char_width, gboolean * r
     gchar *next_ch = NULL;
     gchar utf8buf[UTF8_CHAR_LEN + 1];
 
-    *char_width = 0;
+    *bytes_consumed = 0;
     *result = FALSE;
 
     switch (view->datasource)
@@ -218,7 +218,7 @@ mcview_get_utf (mcview_t * view, off_t byte_index, int *char_width, gboolean * r
     if (res < 0)
     {
         ch = *str;
-        *char_width = 1;
+        *bytes_consumed = 1;
     }
     else
     {
@@ -226,7 +226,7 @@ mcview_get_utf (mcview_t * view, off_t byte_index, int *char_width, gboolean * r
         /* Calculate UTF-8 char width */
         next_ch = g_utf8_next_char (str);
         if (next_ch)
-            *char_width = next_ch - str;
+            *bytes_consumed = next_ch - str;
         else
             return 0;
     }
diff --git a/src/viewer/display.c b/src/viewer/display.c
index 00c6ec0..b1bd390 100644
--- a/src/viewer/display.c
+++ b/src/viewer/display.c
@@ -251,10 +251,6 @@ mcview_display (mcview_t * view)
     {
         mcview_display_hex (view);
     }
-    else if (view->text_nroff_mode)
-    {
-        mcview_display_nroff (view);
-    }
     else
     {
         mcview_display_text (view);
diff --git a/src/viewer/internal.h b/src/viewer/internal.h
index f172c5b..319f77e 100644
--- a/src/viewer/internal.h
+++ b/src/viewer/internal.h
@@ -87,6 +87,18 @@ typedef struct
     coord_cache_entry_t **cache;
 } coord_cache_t;
 
+// TODO: find a better name. This is not actually a "state machine",
+// but a "state machine's state", but that sounds silly.
+// Could be parser_state, formatter_state...
+typedef struct
+{
+    off_t offset;               /* The file offset at which this is the state. */
+    off_t unwrapped_column;     /* Columns if the paragraph wasn't wrapped, */
+                                /* used for positioning TABs in wrapped lines */
+    gboolean nroff_underscore_is_underlined;  /* whether _\b_ is underlined rather than bold */
+    gboolean print_lonely_combining;   /* whether lonely combining marks are printed on a dotted circle */
+} mcview_state_machine_t;
+
 struct mcview_nroff_struct;
 
 struct mcview_struct
@@ -140,8 +152,12 @@ struct mcview_struct
 
     /* Display information */
     screen_dimen dpy_frame_size;        /* Size of the frame surrounding the real viewer */
-    off_t dpy_start;            /* Offset of the displayed data */
+    off_t dpy_start;            /* Offset of the displayed data (start of the paragraph in non-hex mode) */
     off_t dpy_end;              /* Offset after the displayed data */
+    off_t dpy_paragraph_skip_lines; /* Extra lines to skip in wrap mode */
+    mcview_state_machine_t dpy_state_top;  /* Parser-formatter state at the topmost visible line in wrap mode */
+    mcview_state_machine_t dpy_state_bottom;  /* Parser-formatter state after the bottomvisible line in wrap mode */
+    gboolean dpy_wrap_dirty;    /* dpy_state_top needs to be recomputed */
     off_t dpy_text_column;      /* Number of skipped columns in non-wrap
                                  * text mode */
     off_t hex_cursor;           /* Hexview cursor position in file */
@@ -152,6 +168,8 @@ struct mcview_struct
     struct area ruler_area;     /* Where the ruler is displayed */
     struct area data_area;      /* Where the data is displayed */
 
+    ssize_t force_max;          /* Force a max offset, or -1 */
+
     int dirty;                  /* Number of skipped updates */
     gboolean dpy_bbar_dirty;    /* Does the button bar need to be updated? */
 
@@ -219,6 +237,12 @@ cb_ret_t mcview_callback (Widget * w, Widget * sender, widget_msg_t msg, int par
 cb_ret_t mcview_dialog_callback (Widget * w, Widget * sender, widget_msg_t msg, int parm,
                                  void *data);
 
+/* ascii.c: */
+void mcview_display_text (mcview_t *);
+void mcview_state_machine_init (mcview_state_machine_t *, off_t);
+void mcview_ascii_move_down (mcview_t *, off_t);
+void mcview_ascii_move_up (mcview_t *, off_t);
+
 /* coord_cache.c: */
 coord_cache_t *coord_cache_new (void);
 void coord_cache_free (coord_cache_t * cache);
@@ -307,9 +331,7 @@ void mcview_place_cursor (mcview_t *);
 void mcview_moveto_match (mcview_t *);
 
 /* nroff.c: */
-void mcview_display_nroff (mcview_t * view);
 int mcview__get_nroff_real_len (mcview_t * view, off_t, off_t p);
-
 mcview_nroff_t *mcview_nroff_seq_new_num (mcview_t * view, off_t p);
 mcview_nroff_t *mcview_nroff_seq_new (mcview_t * view);
 void mcview_nroff_seq_free (mcview_nroff_t **);
@@ -317,10 +339,6 @@ nroff_type_t mcview_nroff_seq_info (mcview_nroff_t *);
 int mcview_nroff_seq_next (mcview_nroff_t *);
 int mcview_nroff_seq_prev (mcview_nroff_t *);
 
-
-/* plain.c: */
-void mcview_display_text (mcview_t *);
-
 /* search.c: */
 mc_search_cbret_t mcview_search_cmd_callback (const void *user_data, gsize char_offset,
                                               int *current_char);
diff --git a/src/viewer/lib.c b/src/viewer/lib.c
index c996c45..46b12c9 100644
--- a/src/viewer/lib.c
+++ b/src/viewer/lib.c
@@ -106,9 +106,8 @@ mcview_toggle_magic_mode (mcview_t * view)
 void
 mcview_toggle_wrap_mode (mcview_t * view)
 {
-    if (view->text_wrap_mode)
-        view->dpy_start = mcview_bol (view, view->dpy_start, 0);
     view->text_wrap_mode = !view->text_wrap_mode;
+    view->dpy_wrap_dirty = TRUE;
     view->dpy_bbar_dirty = TRUE;
     view->dirty++;
 }
@@ -120,6 +119,7 @@ mcview_toggle_nroff_mode (mcview_t * view)
 {
     view->text_nroff_mode = !view->text_nroff_mode;
     mcview_altered_nroff_flag = 1;
+    view->dpy_wrap_dirty = TRUE;
     view->dpy_bbar_dirty = TRUE;
     view->dirty++;
 }
@@ -144,6 +144,8 @@ mcview_toggle_hex_mode (mcview_t * view)
         widget_want_cursor (WIDGET (view), FALSE);
     }
     mcview_altered_hex_mode = 1;
+    view->dpy_paragraph_skip_lines = 0;
+    view->dpy_wrap_dirty = TRUE;
     view->dpy_bbar_dirty = TRUE;
     view->dirty++;
 }
@@ -170,6 +172,10 @@ mcview_init (mcview_t * view)
     view->coord_cache = NULL;
 
     view->dpy_start = 0;
+    view->dpy_paragraph_skip_lines = 0;
+    mcview_state_machine_init (&view->dpy_state_top, 0);
+    view->dpy_wrap_dirty = FALSE;
+    view->force_max = -1;
     view->dpy_text_column = 0;
     view->dpy_end = 0;
     view->hex_cursor = 0;
@@ -283,6 +289,7 @@ mcview_set_codeset (mcview_t * view)
             view->converter = conv;
         }
         view->utf8 = (gboolean) str_isutf8 (cp_id);
+        view->dpy_wrap_dirty = TRUE;
     }
 #else
     (void) view;
@@ -340,7 +347,7 @@ mcview_bol (mcview_t * view, off_t current, off_t limit)
         if (c == '\r')
             current--;
     }
-    while (current > 0 && current >= limit)
+    while (current > 0 && current > limit)
     {
         if (!mcview_get_byte (view, current - 1, &c))
             break;
diff --git a/src/viewer/mcviewer.c b/src/viewer/mcviewer.c
index f55eecf..0a009ba 100644
--- a/src/viewer/mcviewer.c
+++ b/src/viewer/mcviewer.c
@@ -397,6 +397,10 @@ mcview_load (mcview_t * view, const char *command, const char *file, int start_l
   finish:
     view->command = g_strdup (command);
     view->dpy_start = 0;
+    view->dpy_paragraph_skip_lines = 0;
+    mcview_state_machine_init (&view->dpy_state_top, 0);
+    view->dpy_wrap_dirty = FALSE;
+    view->force_max = -1;
     view->search_start = 0;
     view->search_end = 0;
     view->dpy_text_column = 0;
@@ -416,7 +420,10 @@ mcview_load (mcview_t * view, const char *command, const char *file, int start_l
         else
             new_offset = min (new_offset, max_offset);
         if (!view->hex_mode)
+        {
             view->dpy_start = mcview_bol (view, new_offset, 0);
+            view->dpy_wrap_dirty = TRUE;
+        }
         else
         {
             view->dpy_start = new_offset - new_offset % view->bytes_per_line;
diff --git a/src/viewer/move.c b/src/viewer/move.c
index 7cd852b..c8facc5 100644
--- a/src/viewer/move.c
+++ b/src/viewer/move.c
@@ -83,6 +83,8 @@ mcview_scroll_to_cursor (mcview_t * view)
         if (cursor < topleft)
             topleft = mcview_offset_rounddown (cursor, bytes);
         view->dpy_start = topleft;
+        view->dpy_paragraph_skip_lines = 0;
+        view->dpy_wrap_dirty = TRUE;
     }
 }
 
@@ -107,64 +109,24 @@ mcview_movement_fixups (mcview_t * view, gboolean reset_search)
 void
 mcview_move_up (mcview_t * view, off_t lines)
 {
-    off_t new_offset;
-
     if (view->hex_mode)
     {
         off_t bytes = lines * view->bytes_per_line;
         if (view->hex_cursor >= bytes)
         {
             view->hex_cursor -= bytes;
-            if (view->hex_cursor < view->dpy_start)
+            if (view->hex_cursor < view->dpy_start) {
                 view->dpy_start = mcview_offset_doz (view->dpy_start, bytes);
+                view->dpy_paragraph_skip_lines = 0;
+                view->dpy_wrap_dirty = TRUE;
+            }
         }
         else
         {
             view->hex_cursor %= view->bytes_per_line;
         }
-    }
-    else
-    {
-        off_t i;
-
-        for (i = 0; i < lines; i++)
-        {
-            if (view->dpy_start == 0)
-                break;
-            if (view->text_wrap_mode)
-            {
-                new_offset = mcview_bol (view, view->dpy_start, view->dpy_start - (off_t) 1);
-                /* check if dpy_start == BOL or not (then new_offset = dpy_start - 1,
-                 * no need to check more) */
-                if (new_offset == view->dpy_start)
-                {
-                    size_t last_row_length;
-
-                    new_offset = mcview_bol (view, new_offset - 1, 0);
-                    last_row_length = (view->dpy_start - new_offset) % view->data_area.width;
-                    if (last_row_length != 0)
-                    {
-                        /* if dpy_start == BOL in wrapped mode, find BOL of previous line
-                         * and move down all but the last rows */
-                        new_offset = view->dpy_start - (off_t) last_row_length;
-                    }
-                }
-                else
-                {
-                    /* if dpy_start != BOL in wrapped mode, just move one row up;
-                     * no need to check if > 0 as there is at least exactly one wrap
-                     * between dpy_start and BOL */
-                    new_offset = view->dpy_start - (off_t) view->data_area.width;
-                }
-                view->dpy_start = new_offset;
-            }
-            else
-            {
-                /* if unwrapped -> current BOL equals dpy_start, just find BOL of previous line */
-                new_offset = view->dpy_start - 1;
-                view->dpy_start = mcview_bol (view, new_offset, 0);
-            }
-        }
+    } else {
+        mcview_ascii_move_up (view, lines);
     }
     mcview_movement_fixups (view, TRUE);
 }
@@ -187,52 +149,14 @@ mcview_move_down (mcview_t * view, off_t lines)
         for (i = 0; i < lines && view->hex_cursor < limit; i++)
         {
             view->hex_cursor += view->bytes_per_line;
-            if (lines != 1)
+            if (lines != 1) {
                 view->dpy_start += view->bytes_per_line;
+                view->dpy_paragraph_skip_lines = 0;
+                view->dpy_wrap_dirty = TRUE;
+           }
         }
-    }
-    else
-    {
-        off_t new_offset = 0;
-
-        if (view->dpy_end - view->dpy_start > last_byte - view->dpy_end)
-        {
-            while (lines-- > 0)
-            {
-                if (view->text_wrap_mode)
-                    view->dpy_end =
-                        mcview_eol (view, view->dpy_end,
-                                    view->dpy_end + (off_t) view->data_area.width);
-                else
-                    view->dpy_end = mcview_eol (view, view->dpy_end, last_byte);
-
-                if (view->text_wrap_mode)
-                    new_offset =
-                        mcview_eol (view, view->dpy_start,
-                                    view->dpy_start + (off_t) view->data_area.width);
-                else
-                    new_offset = mcview_eol (view, view->dpy_start, last_byte);
-                if (new_offset < last_byte)
-                    view->dpy_start = new_offset;
-                if (view->dpy_end >= last_byte)
-                    break;
-            }
-        }
-        else
-        {
-            off_t i;
-            for (i = 0; i < lines && new_offset < last_byte; i++)
-            {
-                if (view->text_wrap_mode)
-                    new_offset =
-                        mcview_eol (view, view->dpy_start,
-                                    view->dpy_start + (off_t) view->data_area.width);
-                else
-                    new_offset = mcview_eol (view, view->dpy_start, last_byte);
-                if (new_offset < last_byte)
-                    view->dpy_start = new_offset;
-            }
-        }
+    } else {
+        mcview_ascii_move_down (view, lines);
     }
     mcview_movement_fixups (view, TRUE);
 }
@@ -257,7 +181,7 @@ mcview_move_left (mcview_t * view, off_t columns)
             if (old_cursor > 0 || view->hexedit_lownibble)
                 view->hexedit_lownibble = !view->hexedit_lownibble;
     }
-    else
+    else if (!view->text_wrap_mode)
     {
         if (view->dpy_text_column >= columns)
             view->dpy_text_column -= columns;
@@ -289,7 +213,7 @@ mcview_move_right (mcview_t * view, off_t columns)
             if (old_cursor < last_byte || !view->hexedit_lownibble)
                 view->hexedit_lownibble = !view->hexedit_lownibble;
     }
-    else
+    else if (!view->text_wrap_mode)
     {
         view->dpy_text_column += columns;
     }
@@ -302,6 +226,8 @@ void
 mcview_moveto_top (mcview_t * view)
 {
     view->dpy_start = 0;
+    view->dpy_paragraph_skip_lines = 0;
+    mcview_state_machine_init(&view->dpy_state_top, 0);
     view->hex_cursor = 0;
     view->dpy_text_column = 0;
     mcview_movement_fixups (view, TRUE);
@@ -331,6 +257,8 @@ mcview_moveto_bottom (mcview_t * view)
         const off_t datalines = view->data_area.height;
 
         view->dpy_start = filesize;
+        view->dpy_paragraph_skip_lines = 0;
+        view->dpy_wrap_dirty = TRUE;
         mcview_move_up (view, datalines);
     }
 }
@@ -347,6 +275,8 @@ mcview_moveto_bol (mcview_t * view)
     else if (!view->text_wrap_mode)
     {
         view->dpy_start = mcview_bol (view, view->dpy_start, 0);
+        view->dpy_paragraph_skip_lines = 0;
+        view->dpy_wrap_dirty = TRUE;
     }
     view->dpy_text_column = 0;
     mcview_movement_fixups (view, TRUE);
@@ -424,10 +354,14 @@ mcview_moveto_offset (mcview_t * view, off_t offset)
     {
         view->hex_cursor = offset;
         view->dpy_start = offset - offset % view->bytes_per_line;
+        view->dpy_paragraph_skip_lines = 0;
+        view->dpy_wrap_dirty = TRUE;
     }
     else
     {
         view->dpy_start = offset;
+        view->dpy_paragraph_skip_lines = 0;
+        view->dpy_wrap_dirty = TRUE;
     }
     mcview_movement_fixups (view, TRUE);
 }
@@ -498,9 +432,14 @@ mcview_moveto_match (mcview_t * view)
         view->hexedit_lownibble = FALSE;
         view->dpy_start = view->search_start - view->search_start % view->bytes_per_line;
         view->dpy_end = view->search_end - view->search_end % view->bytes_per_line;
+        view->dpy_paragraph_skip_lines = 0;
+        view->dpy_wrap_dirty = TRUE;
     }
-    else
+    else {
         view->dpy_start = mcview_bol (view, view->search_start, 0);
+        view->dpy_paragraph_skip_lines = 0;
+        view->dpy_wrap_dirty = TRUE;
+    }
 
     mcview_scroll_to_cursor (view);
     view->dirty++;
diff --git a/src/viewer/nroff.c b/src/viewer/nroff.c
index eb3a486..7905eb9 100644
--- a/src/viewer/nroff.c
+++ b/src/viewer/nroff.c
@@ -1,6 +1,6 @@
 /*
    Internal file viewer for the Midnight Commander
-   Function for nroff-like view
+   Functions for searching in nroff-like view
 
    Copyright (C) 1994-2014
    Free Software Foundation, Inc.
@@ -91,6 +91,7 @@ mcview_nroff_get_char (mcview_nroff_t * nroff, int *ret_val, off_t nroff_index)
 /*** public functions ****************************************************************************/
 /* --------------------------------------------------------------------------------------------- */
 
+#if 0  /* moved to ascii.c */
 void
 mcview_display_nroff (mcview_t * view)
 {
@@ -249,6 +250,7 @@ mcview_display_nroff (mcview_t * view)
     }
     view->dpy_end = from;
 }
+#endif
 
 /* --------------------------------------------------------------------------------------------- */
 
diff --git a/src/viewer/plain.c b/src/viewer/plain.c
deleted file mode 100644
index 8003f3a..0000000
--- a/src/viewer/plain.c
+++ /dev/null
@@ -1,207 +0,0 @@
-/*
-   Internal file viewer for the Midnight Commander
-   Function for plain view
-
-   Copyright (C) 1994-2014
-   Free Software Foundation, Inc.
-
-   Written by:
-   Miguel de Icaza, 1994, 1995, 1998
-   Janne Kukonlehto, 1994, 1995
-   Jakub Jelinek, 1995
-   Joseph M. Hinkle, 1996
-   Norbert Warmuth, 1997
-   Pavel Machek, 1998
-   Roland Illig <roland.illig@gmx.de>, 2004, 2005
-   Slava Zanko <slavazanko@google.com>, 2009
-   Andrew Borodin <aborodin@vmail.ru>, 2009-2014
-   Ilia Maslakov <il.smind@gmail.com>, 2009
-
-   This file is part of the Midnight Commander.
-
-   The Midnight Commander is free software: you can redistribute it
-   and/or modify it under the terms of the GNU General Public License as
-   published by the Free Software Foundation, either version 3 of the License,
-   or (at your option) any later version.
-
-   The Midnight Commander is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-   GNU General Public License for more details.
-
-   You should have received a copy of the GNU General Public License
-   along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <config.h>
-
-#include "lib/global.h"
-#include "lib/tty/tty.h"
-#include "lib/skin.h"
-#include "lib/util.h"           /* is_printable() */
-#ifdef HAVE_CHARSET
-#include "lib/charsets.h"
-#endif
-
-#include "src/setup.h"          /* option_tab_spacing */
-
-#include "internal.h"
-
-/*** global variables ****************************************************************************/
-
-/*** file scope macro definitions ****************************************************************/
-
-/*** file scope type declarations ****************************************************************/
-
-/*** file scope variables ************************************************************************/
-
-/*** file scope functions ************************************************************************/
-/* --------------------------------------------------------------------------------------------- */
-
-/* --------------------------------------------------------------------------------------------- */
-/*** public functions ****************************************************************************/
-/* --------------------------------------------------------------------------------------------- */
-
-void
-mcview_display_text (mcview_t * view)
-{
-    const screen_dimen left = view->data_area.left;
-    const screen_dimen top = view->data_area.top;
-    const screen_dimen width = view->data_area.width;
-    const screen_dimen height = view->data_area.height;
-    screen_dimen row = 0, col = 0;
-    off_t from;
-    int cw = 1;
-    int c, prev_ch = 0;
-    gboolean last_row = TRUE;
-    struct hexedit_change_node *curr = view->change_list;
-
-    mcview_display_clean (view);
-    mcview_display_ruler (view);
-
-    /* Find the first displayable changed byte */
-    from = view->dpy_start;
-    while ((curr != NULL) && (curr->offset < from))
-        curr = curr->next;
-
-    while (row < height)
-    {
-#ifdef HAVE_CHARSET
-        if (view->utf8)
-        {
-            gboolean read_res = TRUE;
-
-            c = mcview_get_utf (view, from, &cw, &read_res);
-            if (!read_res)
-                break;
-        }
-        else
-#endif
-        if (!mcview_get_byte (view, from, &c))
-            break;
-
-        last_row = FALSE;
-        from++;
-        if (cw > 1)
-            from += cw - 1;
-
-        if (c != '\n' && prev_ch == '\r')
-        {
-            if (++row >= height)
-                break;
-
-            col = 0;
-            /* tty_print_anychar ('\n'); */
-        }
-
-        prev_ch = c;
-        if (c == '\r')
-            continue;
-
-        if (c == '\n')
-        {
-            col = 0;
-            row++;
-            continue;
-        }
-
-        if (col >= width && view->text_wrap_mode)
-        {
-            col = 0;
-            if (++row >= height)
-                break;
-        }
-
-        if (c == '\t')
-        {
-            col += (option_tab_spacing - col % option_tab_spacing);
-            if (view->text_wrap_mode && col >= width && width != 0)
-            {
-                row += col / width;
-                col %= width;
-            }
-            continue;
-        }
-
-        if (view->search_start <= from && from < view->search_end)
-            tty_setcolor (SELECTED_COLOR);
-        else
-            tty_setcolor (VIEW_NORMAL_COLOR);
-
-        if (((off_t) col >= view->dpy_text_column)
-            && ((off_t) col - view->dpy_text_column < (off_t) width))
-        {
-            widget_move (view, top + row, left + ((off_t) col - view->dpy_text_column));
-
-#ifdef HAVE_CHARSET
-            if (mc_global.utf8_display)
-            {
-                if (!view->utf8)
-                    c = convert_from_8bit_to_utf_c ((unsigned char) c, view->converter);
-                if (!g_unichar_isprint (c))
-                    c = '.';
-            }
-            else if (view->utf8)
-                c = convert_from_utf_to_current_c (c, view->converter);
-            else
-            {
-                c = convert_to_display_c (c);
-                if (!is_printable (c))
-                    c = '.';
-            }
-#else /* HAVE_CHARSET */
-            if (!is_printable (c))
-                c = '.';
-#endif /* HAVE_CHARSET */
-
-            tty_print_anychar (c);
-        }
-
-        col++;
-
-#ifdef HAVE_CHARSET
-        if (view->utf8)
-        {
-            if (g_unichar_iswide (c))
-                col++;
-            else if (g_unichar_iszerowidth (c))
-                col--;
-        }
-#endif
-    }
-
-    view->dpy_end = from;
-    if (mcview_show_eof != NULL && mcview_show_eof[0] != '\0')
-    {
-        if (last_row && mcview_get_byte (view, from - 1, &c) && c != '\n')
-            row--;
-
-        while (++row < height)
-        {
-            widget_move (view, top + row, left);
-            tty_print_string (mcview_show_eof);
-        }
-    }
-}
-
-/* --------------------------------------------------------------------------------------------- */
diff --git a/tests/src/viewer/viewertest.txt b/tests/src/viewer/viewertest.txt
new file mode 100644
index 0000000..6a0a299
--- /dev/null
+++ b/tests/src/viewer/viewertest.txt
@@ -0,0 +1,76 @@
+* LF as line terminator
+This row has 79 columns:    30        40        50        60        70       79
+This row has 80 columns:    30        40        50        60        70        80
+This row has 81 columns:    30        40        50        60        70         81
+
+* CR as line terminator
This row has 79 columns:    30        40        50        60        70       79
This row has 80 columns:    30        40        50        60        70        80
This row has 81 columns:    30        40        50        60        70         81

* CR+LF as line terminator
+This row has 79 columns:    30        40        50        60        70       79
+This row has 80 columns:    30        40        50        60        70        80
+This row has 81 columns:    30        40        50        60        70         81
+
+* TAB characters of varying widths (with reference rendering above)
+88888888········7·······66······555·····4444····33333···222222··1111111·|
+88888888       7       66      555     4444    33333   222222  1111111 |
+
+* Combining accents on top of every second letter (a, c, ...)
+---------------------------------------------------|
+ÁBC̀DÉFG̀HÍJK̀LḾNÒPQ́RS̀TÚVẀXÝzỳxẃvùtśrq̀pónm̀lḱjìhǵfèdćbà|
+
+* More and more combining accents on a single character
+---  ---  ---  ---  ---  ---|
+0:a  1:à  2:à́  3:à́̂  4:à́̂̃  5:à́̂̃̄|
+0:x  1:x̀  2:x̀́  3:x̀́̂  4:x̀́̂̃  5:x̀́̂̃̄|
+
+* Combining accents at beginning of line, and after tab, with
+  reference rendering (explicit dotted circles, and spaces) above
+-       -       -       -       -|
+◌̀́       ◌̀́       ◌̀́       ◌̀́       ◌̀́|
+̀́     ̀́      ̀́      ̀́      ̀́|
+
+* Same with spacing mark
+一      一      一      一      一|
+◌ो      ◌ो      ◌ो      ◌ो      ◌ो|
+ो      ो       ो       ो       ो|
+
+* CJK, Lorem ipsum by Google translate, second line shifted by a space.
+  When wrapped, the trailing bars will not align
+のイプサム嘆き、の痛みに座るが、時折状況が労苦と痛みが彼にいくつかの大きな喜びを調達することができる起こるので。 |
+ のイプサム嘆き、の痛みに座るが、時折状況が労苦と痛みが彼にいくつかの大きな喜びを調達することができる起こるので。|
+
+* Devanagari spacing marks, with reference positions. Just as with CJK,
+  the two cells should appear/disappear together
+一 一 一 一|一  一  一  一|一   一   一   一|一    一    一    一|
+हो हि हो हि|हो  हि  हो  हि|हो   हि   हो   हि|हो    हि    हो    हि|
+
+* Thai Sara Am
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --|
+aำ bำ cำ dำ eำ fำ gำ hำ iำ jำ kำ lำ mำ nำ oำ pำ qำ rำ sำ tำ uำ vำ wำ xำ yำ zำ|
+
+* TABs mixed with other wierd characters
+-----   -       一      一一一一        --      -- --   |
+abcde          の      イプサム    à́b̀́  हो हि   |
+
+* Extreme stress test: base letter with multiple (c)ombining or (s)pacing marks
+---  -------  ----  ------------  -----------  -----------  -----------  -----------|
+c:x̀  ccccc:x̀́̂̃̄  s:xो  sssss:xोोोोो  sssccc:xोोो̀́̂  cccsss:x̀́̂ोोो  scscsc:xो̀ो́ो̂  cscscs:x̀ो́ो̂ो|
+
+* Same as above, but with CJK base char
+--一  ------一  --一-  ------一-----  -------一---  -------一---  -------一---  -------一---|
+c:の̀  ccccc:の̀́̂̃̄  s:のो  sssss:のोोोोो  sssccc:のोोो̀́̂  cccsss:の̀́̂ोोो  scscsc:のो̀ो́ो̂  cscscs:の̀ो́ो̂ो|
+
+* Nroff
+---------------  一一一一  - - - -  一 一 一 一  -----------------|
+_Hello,_World!_  のイプサ  à́ b̀́ c̀́ d̀́  हो हि हो हि  __b___u___b___u__|
+__HHeelllloo,,___W_o_r_l_d_!__  ののイイ_プ_サ  aà̀́́ bb̀̀́́ _c_̀_́ _d_̀_́  हहोो हहिि _ह_ो _ह_ि  ____bb_______u______bb_______u____|
+______ <- should be bold again
+
+* Invalid nroff (a backspace b tab backspace tab underscore backspace newline,
+  reference rendering in the first row)
+a.b     .       _.
+ab           _
+
+* Control characters (00-1F except tab/lf/cr, 7F, 80-9F), should all be replaced by dots
+@ABCDEFGH--KL-NOPQRSTUVWXYZ[\]^_|?|@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_|
+   ||€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ|
+
+* Invalid UTF-8 not tested here, use Markus Kuhn's stress test