Calculating bi-directional rendering order¶

The following process computes the rendering order of characters according to the Unicode Bi-Directional algorithm:

The parameters to unicode_bidi_calc_types() are:

The parameters to unicode_bidi_calc_levels() are:

The parameters to unicode_bidi_calc() are the same except for the unicode_bidi_type_t pointer. unicode_bidi_calc() allocates this buffer by itself and calls unicode_bidi_calc_types, and destroys the buffer before returning.

unicode_bidi_calc() and unicode_bidi_calc_levels() fill in the unicode_bidi_level_t array with the values corresponding to the embedding level of the corresponding character, according the Unicode Bidirection Algorithm (even values for left-to-right ordering, and odd values for right-to-left ordering). A value of UNICODE_BIDI_SKIP designates directional markers (from step X9).

unicode_bidi_calc() and unicode_bidi_calc_levels() return the resolved paragraph direction level, which always matches the passed in level, if specified, else it reports the derived one. These functions return a unicode_bidi_direction structure:

direction gives the paragraph embedding level, UNICODE_BIDI_LR or UNICODE_BIDI_RL. is_explicit indicates whether: the optional pointer to a UNICODE_BIDI_LR or UNICODE_BIDI_RL value was specified (and returned in direction), or whether the direction comes from an character with an explicit direction indication.

unicode_bidi_reorder() takes the actual unicode string together with the embedding values from unicode_bidi_calc or unicode_bidi_calc_levels(), then reverses the bi-directional string, as specified by step L2 of the bi-directional algorithm. The parameters to unicode_bidi_reorder() are:

A non-NULL reorder_callback gets invoked to report each reversed character range. The callback's first parameter is the index of the first reversed character, the second parameter is the number of reversed characters, starting at the given index of the Unicode string. The third parameter is the arg passthrough parameter.

unicode_bidi_reorder modifies its string and levels. reorder_callback gets invoked after reversing each consecutive range of values in the string and levels buffers. For example: “reorder_callback(5, 7, arg)” reports that character indexes #5 through #11 got reversed.

A NULL string pointer leaves the levels buffer unchanged, but still invokes the reorder_callback as if the character string, and their embedding values, were reversed.

The resulting string and embedding levels are in “rendering order”, but still contain bi-directional embedding, override, boundary-neutral, isolate, and marker characters. unicode_bidi_cleanup removes these characters and directional markers.

The parameters to unicode_bidi_cleanup() are:

The function pointer (if not NULL) gets invoked to report the index of each removed character. The reported index is the index from the original string, and the callback gets invoked in strict order, from the first to the last removed character (if any).

The character string and the embedding level values resulting from unicode_bidi_cleanup() with the UNICODE_BIDI_CLEANUP_CANONICAL are in “canonical rendering order”. unicode_bidi_logical_order(), unicode_bidi_needs_embed() and unicode_bidi_embed() require the canonical rendering order for their string and embedding level values.

The parameters to unicode_bidi_cleaned_size() are a pointer to the unicode string, its size, and the bitmask option to unicode_bidi_cleanup().

Embedding bi-directional markers in Unicode text strings¶

unicode_bidi_logical_order() rearranges the string from rendering to its logical order. unicode_bidi_embed() adds various bi-directional markers to a Unicode string in canonical rendering order. The resulting string is not guaranteed to be identical to the original Unicode bi-directional string. The algorithm is fairly basic, but the resulting bi-directional string produces the same canonical rendering order after applying unicode_bidi_calc() or unicode_bidi_calc_levels(), unicode_reorder() and unicode_bidi_cleanup() (with the canonical option), with the same paragraph_embedding level. unicode_bidi_needs_embed() attempts to heuristically determine whether unicode_bidi_embed() is required.

unicode_bidi_logical_order() gets called first, followed by unicode_bidi_embed() (or unicode_bidi_needs_embed() in order to determine whether bi-directional markers are required). Finally, unicode_bidi_embed_paragraph_level() optionally determines whether the resulting string's default paragraph embedding level matches the one used for the actual embedding direction, and if not returns a directional marker to be prepended to the Unicode character string, as a hint.

unicode_bidi_logical_order() factors in the characters' embedding values, and the provided paragraph embedding value (UNICODE_BIDI_LR or UNICODE_BIDI_RL), and rearranges the characters and the embedding levels in left-to-right order, while simultaneously invoking the supplied reorder_callback indicating each range of characters whose relative order gets reversed. The reorder_callback() receives, as parameters:

This specifies a consecutive range of characters (and directional embedding values) that get reversed (first character in the range becomes the last character, and the last character becomes the first character).

After unicode_bidi_logical_order(), unicode_bidi_embed() progressively invokes the passed-in callback with the contents of a bi-directional unicode string. The parameters to unicode_bidi_embed() are:

The callback receives pointers to various parts of the original string that gets passed to unicode_bidi_embed(), intermixed with bi-directional markers, overrides, and isolates. The callback's parameters are:

The assembled unicode string should produce the same canonical rendering order, for the same paragraph embedding level. unicode_bidi_embed_paragraph_level() checks if the specified Unicode string computes the given default paragraph embedding level and returns 0 if it matches. Otherwise it returns a directional marker that should be prepended to the Unicode string to allow unicode_bidi_calc's (or unicode_bidi_calc_levels()) optional paragraph embedding level pointer's value to be NULL, but derive the same default embedding level. The parameters to unicode_bidi_embed_paragraph_level() are:

unicode_bidi_needs_embed() attempts to heuristically determine whether the Unicode string, in logical order, requires bi-directional markers. The parameters to unicode_bidi_embed_paragraph_level() are:

unicode_bidi_needs_embed() returns 0 if the Unicode string does not need explicit directional markers, or 1 if it does. This is done by using unicode_bidi_calc(), unicode_bidi_reorder(), unicode_bidi_logical_order and then checking if the end result is different from what was passed in.

Combining character ranges¶

unicode_bidi_combinings() reports consecutive sequences of one or more combining marks in bidirectional text (which can be either in rendering or logical order) that have the same embedding level. It takes the following parameters:

The callback function gets invoked for every consecutive sequence of one or more characters that have a canonical combining class other than 0, and with the same embedding level. The parameters to the callback function are:

A consecutive sequence of Unicode characters with non-0 combining classes but different embedding levels gets reported individually, for each consecutive sequence with the same embedding level.

This function helps with reordering the combining characters in right-to-left-rendered text. Right-to-left text reversed by unicode_bidi_reorder() results in combining characters preceding their starter character. They get reversed no differently than any other character. The same thing also occurs after unicode_bidi_logical_order() reverses everything back. Use unicode_bidi_combinings to identify consecutive sequences of combining characters followed by their original starter.

The callback may reorder the characters identified by its third and the fourth parameters in the manner described below. unicode_bidi_reorder's parameter is pointers to a constant Unicode string; but it can modify the string (via an out-of-band mutable pointer) subject to the following conditions:

Here's an example of a callback that reverses combining characters and their immediately-following starter character:

void reorder_right_to_left_combining(unicode_bidi_level_t level,


                                     size_t level_start,


                                     size_t n_chars,


                                     size_t comb_start,


                                     size_t n_comb_chars,


                                     void *arg)
{


    /* Let's say that this is the Unicode string */


    char32_t *buf=(char32_t *)arg;


    if ((level & 1) == 0)


        return; /* Left-to-right text not reversed */


    char32_t *b=buf+comb_start;


    char32_t *e=b+n_comb_chars;


    /*


    ** Include the starter characters in the reversed range.


    ** The semantics of the combining characters with different


    ** embedding levels -- so they get reported here separately -- is


    ** not specified. This will reverse just the combining marks, and


    ** they're on their own.


    */


    if (comb_start + n_comb_chars < level_start + n_chars)


        ++e;


    while (b < e)


    {


        char32_t t;


        --e;


        t=*b;


        *b=*e;


        *e=t;


        ++b;


    }
}

Miscellaneous utility functions¶

unicode_bidi_get_direction takes a pointer to a unicode string, the number of characters in the unicode string, and determines default paragraph level level. unicode_bidi_get_direction returns a struct with the following fields:

direction

is_explicit

unicode_bidi_type looks up each character's bi-directional character type.

unicode_bidi_setbnl takes a pointer to a unicode string, a pointer to an array of enum_bidi_type_t values and the number of characters in the string and the array. unicode_bidi_setbnl replaces all paragraph separators in the unicode string with a newline character (same as the UNICODE_BIDI_CLEANUP_BNL option to unicode_bidi_cleanup.

unicode_bidi_mirror returns the glyph that's a mirror image of the parameter (i.e. an open parenthesis for a close parenthesis, and vice versa); or the same value if there is no mirror image (this is the Bidi_Mirrored=Yes property).

unicode_bidi_bracket_type looks up each bracket character and returns its opposite, or the same value if the character is not a bracket that has an opposing bracket character (this is the Bidi_Paired_Bracket_type property). A non-NULL ret gets initialized to either UNICODE_BIDI_o, UNICODE_BIDI_c or UNICODE_BIDI_n.