Liblouis User's and Programmer's Manual

6.1 YAML Tests ¶

YAML is a human readable data serialization format that allows for an easy and compact way to define tests.

A YAML file first defines which tables are to be used for the tests. Then it optionally defines flags such as the ‘testmode’. Finally all the tests are defined.

You can repeat the cycle as many times as you like (tables, optional flags, tests). You can also define several rounds of tests for any table, with or without the optional flags. Just remember that the flags are reset to their default values each time you start a new round of tests or load a new set of tables.

Let’s just look at a simple example how tests could be defined:

# comments start with '#' anywhere on a line
# first define which tables will be used for your tests
table: [unicode.dis, en-ueb-g1.ctb]

# then optionally define flags such as testmode. If no flags are
# defined forward translation is assumed

# now define the tests
tests:
  - # each test is a list.
    # The first item is the string to translate. Quoting of strings is
    # optional
    - hello
    # The second item is the expected translation
    - ⠓⠑⠇⠇⠕
  - # optionally you can define additional parameters in a third
    # item such as typeform or expected failure, etc
    - Hello
    - ⠨⠶⠠⠓⠑⠇⠇⠕⠨⠄
    - {typeform: {italic: '++++ '}, xfail: true}
  - # a simple, no-frills test
    - Good bye
    - ⠠⠛⠕⠕⠙ ⠃⠽⠑
  # same as above using "flow style" notation
  - [Good bye,  ⠠⠛⠕⠕⠙ ⠃⠽⠑]

The four basic components of a test file are as follows:

‘table’

A list containing table files, which the tests should be run against. This is usually just one file, but for some situations more than one file can be required. For example:

table: [hu-hu-g1.ctb, hyph_hu_HU.dic]

It is also possible to specify a table inline. Inline definition of tables below explains how to do this.

A third way to specify a table is by its metadata. A table query, which is essentially as list of “features”, is matched against the table metadata defined inside the tables contained in LOUIS_TABLEPATH. Only the best match is used for the test.

The syntax of the query is a variation of the syntax used for the lou_findTable function:

table:
  locale: fr
  grade: 1

‘display’

A display table, which should be used to encode braille in the test. This item is optional. It is only taken into account for the ‘forward’, ‘backward’ and ‘bothDirections’ test modes. If it is present it should be the first item of the file. If it is not present, the braille encoding of each test is determined by the table that is being tested.

The next example shows how to test the en-ueb-g1.ctb table using ASCII notation (as defined in en-ueb-g1.ctb itself):

table: [en-ueb-g1.ctb]

If you wanted to test the en-ueb-g1.ctb table using Unicode dot patterns then you would use the following definition:

display: unicode.dis
table: [en-ueb-g1.ctb]

‘flags’

The flags that apply for all tests in this file. At the moment only the ‘testmode’ flag is supported. It can have four possible values:

‘forward’

This indicates that the tests are for forward translation.

‘backward’

This indicates that the tests are for backward translation.

‘bothDirections’

Checks that forward translating the input yields the expected output, and backward translating the output yields the input again.

‘display’

Checks that a display table maps characters to the expected dot patterns. The input is a string of characters, the output are the corresponding dot patterns, as a string of Unicode braille symbols. Virtual dots in the output are ignored.

‘hyphenate’

This indicates that the tests are for hyphenation.

‘hyphenateBraille’

This indicates that the tests are for hyphenation and the input is braille.

If no flags are defined forward translation is assumed.

‘tests’

A list of tests. Each test consists of a list of two, three or in some cases four items. The first item is the test input. The second item is the expected output. Braille input and output can be either Unicode braille or an ASCII-braille like encoding, depending on the specified test mode, translation mode and display table. Quoting strings is optional. Comments can be inserted almost anywhere using the ‘#’ sign. A simple test would look at follows:

  - # a simple, no-frills test
    - Good bye
    - ⠠⠛⠕⠕⠙ ⠃⠽⠑

Using the more compact “flow style” notation it would look like the following:

  - [Good bye, ⠠⠛⠕⠕⠙ ⠃⠽⠑]

An optional third item can contain additional options for a test such as the typeform, or whether a test is expected to fail. The following shows a typical example:

  -
    - Hello
    - ⠨⠶⠠⠓⠑⠇⠇⠕⠨⠄
    - {typeform: {italic: '++++ '}, xfail: true}
  # same test more compact
  - [Hello, ⠨⠶⠠⠓⠑⠇⠇⠕⠨⠄, {typeform: {italic: '++++ '}, xfail: true}]
  # same test but is only expected to fail for backtranslation
  - [Hello, ⠨⠶⠠⠓⠑⠇⠇⠕⠨⠄, {typeform: {italic: '++++ '}, xfail: {backward: true}}]

The valid additional options for a test are as follows:

‘xfail’

Sometimes it is known that a test is failing. Maybe the table under test doesn’t handle that word correctly yet, or maybe backtranslation has not been implemented. ‘xfail’ is designed to mark tests as expected failures. There are three ways to mark a test as failing:

1. Simply Mark a test as expected failure by setting ‘xfail’ to ‘true’.

     - [Hello, ⠨, {xfail: true}]
   

2. Mark a test as expected failure and give a reason for the failure.

     - [Hello, ⠨, {xfail: Test case is not complete}]
   

3. Mark a test as expected failure just for backward or forward translation using the notation ‘{xfail: {forward: true}}’ or ‘{xfail: {backward: true}}’. If you expect both to fail use ‘{xfail: {forward: true, backward: true}}’.

   To mark just the backward translation of a test as expected failure use the following:

     - [Hello, ⠨, {xfail: {backward: true}}]
   

   Again a reason for the expected failure can be given.

     - [Hello, ⠨, {xfail: {backward: Not implemented}}]
   

   If you expect both forward and backward translation to fail set both ‘forward’ and ‘backward’ to ‘true’ (or give a reason).

     - [Hello, ⠨, {xfail: {forward: true, backward: true}}]
     # above is equivalent to
     - [Hello, ⠨, {xfail: true}]
   

‘typeform’

The typeform used for a translation. It consists of one or more emphasis specifications. For each character in the specifications that is not a space the corresponding emphasis will be set. Valid options for emphasis are ‘italic’, ‘underline’, ‘bold’, ‘computer_braille’, ‘passage_break’, ‘word_reset’, ‘script’, ‘trans_note’, ‘trans_note_1’, ‘trans_note_2’, ‘trans_note_3’, ‘trans_note_4’ or ‘trans_note_5’. The following shows an example where both ‘italic’ and ‘underline’ are specified:

  -
    - Hello
    - ⠨⠶⠠⠓⠑⠇⠇⠕⠨⠄
    - typeform:
        italic:    '++++ '
        underline: '    +'

‘inputPos’

A list of 0-based input positions, one for each output position. Useful when simulating screen reader interaction, to debug contraction and cursor behavior as in the following example. Note that all positions in this and the following examples start at 0. Also note that in these examples the additional options are not passed using the “flow style” notation.

  -
    - went
    - ⠺⠢⠞
    - inputPos: [0,1,3]

‘outputPos’

A list of 0-based output positions, one for each input position. Useful when simulating screen reader interaction, to debug contraction and cursor behavior as in the following example.

  -
    - went
    - ⠺⠢⠞
    - outputPos: [0,1,1,2]

‘cursorPos’

The cursor position for the given translation and optionally an expected cursor position where the cursor is supposed to be after the translation. Useful when simulating screen reader interaction, to debug contraction and cursor behavior:

The cursor position can take two forms: You can either specify a single number or alternatively you can give a tuple of two numbers.

single number (e.g. ‘4’)

When you simply want to specify the cursor position for the given translation you pass a number as in the following example:

  -
    - you went to
    - ⠽ ⠺⠑⠝⠞ ⠞⠕
    - mode: [compbrlAtCursor]
      cursorPos: 4

a tuple (e.g. ‘[4,2]’)

When you expect the cursor to be in a particular position after the translation and you want to check this then pass a tuple of cursor positions as in the following example:

  -
    - you went to
    - ⠽ ⠺⠑⠝⠞ ⠞⠕
    - mode: [compbrlAtCursor]
      cursorPos: [4,2]

‘mode’

A list of translation modes that should be used for this test. If not defined defaults to 0. Valid mode values are ‘noContractions’, ‘compbrlAtCursor’, ‘dotsIO’, ‘compbrlLeftCursor’, ‘ucBrl’, ‘noUndefined’ or ‘partialTrans’.

For a description of the various translation mode flags, please see the function lou_translateString.

‘maxOutputLength’

Define a maximum length of the output. This can be used to test the behavior of liblouis in the face of a limited output buffer, for example the length of the refreshable braille display.

• Optional test description
• Testing multiple tables within the same YAML test file
• Multiple test sections for each table
• Inline definition of tables
• Running the same test data on multiple tables

  -
    - Number-text-transitions with italic
    - 123abc
    - ⠼⠁⠃⠉⠨⠶⠰⠁⠃⠉⠨⠄
    - {typeform: '000111'}

table: ...
tests:
  - [..., ...]
  - [..., ...]
table: ...
tests:
  - [..., ...]
  - [..., ...]

table: ...
tests:
  - [..., ...]
  - [..., ...]

# Some more tests
  tests:
  - [..., ...]
  - [..., ...]

# Some tests for back-translation - same table
flags: {testmode: backward}
  - [..., ...]
  - [..., ...]

table: |
  sign a 1
  ...
tests:
  - ...
  - ...

table: nl-NL
table: nl-BE
tests:
  - [..., ...]
  - [..., ...]