README: Cutter Reference Manual

## README README — An introduction of Cutter, a Unit Testing Framework for C and C++ |

## Name

Cutter

## Author

### Program

- Kouhei Sutou <kou@clear-code.com>

- Hiroyuki Ikezoe <ikezoe@clear-code.com>

- Yuto Hayamizu <y.hayamizu@gmail.com>

### Kinotan Icons

- Mayu & Co.

## License

- Source: LGPLv3 or later. (detail: license/lgpl-3.txt )

- Document and kinotan icons: Triple license: LGPL, GFDL and/or CC.

  - LGPL: v3 or later. (detail: license/lgpl-3.txt )

  - GFDL: v1.3 or later. (detail: license/gfdl-1.3.txt )

  - CC: BY-SA

- Exceptions:

  - glib-compatible/glibintl.h, glib-compatible/gregex.*, glib-compatible/gscripttable.h, glib-compatible/gsequence.*, glib-compatible/gstring.*, glib-compatible/gunicode.h, glib-compatible/guniprop.c: LGPL v2.0 or later. (detail: glib-compatible/COPYING )

  - glib-compatible/pcre/: PCRE LICENSE. (detail: glib-compatible/pcre/COPYING )

  - html/blog.*, html/download.*, html/heading-mark.* html/install.*, html/readme.*, html/reference.*, html/tango-logo.png, html/tutorial.*, html/mini-*.svg: Public domain. They are deliverables by Tango Desktop Project . (Some of them are modified.)

  - html/ja.png, html/us.png, html/famfamfam-logo.png: Public domain. They are distributed by famfamfam.com .

## What's this?

Cutter is a xUnit family Unit Testing Framework for C and C++.

This is a list of features of Cutter:

- easy to write tests.

- outputs result with useful format for debugging.

- tests are built as shared libraries.

See FEATURES for more details.

## Dependency libraries

- GLib >= 2.16

## Install

See Install .

## Repository

There is the repository at clear-code/cutter on GitHub .

    % git clone https://github.com/clear-code/cutter.git

## Usage

    % cutter [OPTION ...] TEST_DIRECTORY

TEST_DIRECTORY should have test_*.so. test_*.so are searched recursively.

See cutter for more details.

## How to test

Executing flow of test is the following.

1. Write a test.

2. Compile it and build test_*.so.

3. Execute cutter. It loads test_*.so and runs them.

See a tutorial and sample/stack/.

## Test result

Here is an example test result:

    ..........F.................................................

      1) Failure: test_test_case_count
      <1 == cut_test_case_get_n_tests(test_object, NULL)>
      expected: <1>
       but was: <0>
      test/test-cut-test-case.c:143: test_test_case_count()

      Finished in 0.020857 seconds

      60 test(s), 253 assertion(s), 1 failure(s), 0 error(s), 0 pending(s), 0 notification(s)

### Progress

A part that contains "." and "F" of the test result shows test progress:

    ..........F.................................................

Each "." and "F" shows a test case (test function). "." shows a test case that is succeeded and "F" shows a test case that is failed. There are "E", "P" and "N". They shows error, pending and notification respectively. Here is a summary of test case marks:

. | A succeeded test
F | A failed test
E | A test that had an error
P | A test that is marked as pending
N | A test that had an notification

The above marks are showed after each test is finished. We can confirm the test progress from the output in testing.

### Summary of test result

Cutter outputs a summary of test result after all tests are finished. The first of a summary is a list of a detail of test result of non-succeeded test. In the example, cutter outputs a detail of test result because there is a failure.

    1) Failure: test_test_case_count
        <1 == cut_test_case_get_n_tests(test_object, NULL)>
        expected: <1>
         but was: <0>
        test/test-cut-test-case.c:143: test_test_case_count()

In the example, test_test_case_count test case is failed. We expected that cut_test_case_get_n_tests(test_object, NULL) is 1 but was 0. The failed assertion is in test_test_case_count() function in test/test-cut-test-case.c at 143th line.

Elapsed time for testing is showed after a list of a detail of test result:

    Finished in 0.020857 seconds

The last line is an summary of test result:

    60 test(s), 253 assertion(s), 1 failure(s), 0 error(s), 0 pending(s), 0 notification(s)

Here are the means of each output:

n test(s) | n test case(s) (test function(s)) are run.
n assertion(s) | n assertion(s) are passed.
n failure(s) | n assertion(s) are failed.
n error(s) | n error(s) are occurred (cut_error() is used n times)
n pending(s) | n test case(s) are pending (cut_pending() is used n times)
n notification(s) | n notification(s) are occurred (cut_notification() is used n times)

In the example, 60 test cases are run, 253 assertions are passed and an assertion is failed. There are no error, pending, notification.

### XML report

Cutter reports test result as XML format if --xml-report option is specified. A reported XML has the following structure:

    <report>
          <!-- "result" tag is generated for a test result.
                Normally, a "result" tag is generated for a test.
                If you use cut_message(), you will get two or more "result" tags
                for a test. -->
          <result>
            <test-case>
              <name>TEST CASE NAME</name>
              <description>DESCRIPTION OF TEST CASE (if exists)</description>
              <start-time>START TIME OF TEST CASE (ISO 8601 format) [e.g.: 2013-11-12T03:32:56.691676Z]</start-time>
              <elapsed>ELAPSED TIME OF TEST CASE (in seconds) [e.g.: 0.030883]</elapsed>
            </test-case>
            <test>
              <name>TEST NAME</name>
              <description>DESCRIPTION OF TEST CASE (if exists)</description>
              <start-time>START TIME OF TEST (ISO 8601 format) [e.g.: 2013-11-12T03:32:56.691823Z]</start-time>
              <elapsed>ELAPSED TIME OF TEST (in seconds) [e.g.: 0.030883]</elapsed>
              <option><!-- ATTRIBUTE INFORMATION (if exists) -->
                <name>ATTRIBUTE NAME [e.g.: bug]</name>
                <value>ATTRIBUTE VALUE [e.g.: 1234]</value>
              </option>
              <option>
                ...
              </option>
              ...
            </test>
            <status>TEST RESULT (one of them: success, notification, omission, pending, failure, error, crash)</status>
            <detail>DETAIL OF TEST RESULT (if exists)</detail>
            <backtrace><!-- BACKTRACE (if exists) -->
              <entry>
                <file>FILE NAME</file>
                <line>LINE</line>
                <info>ADDITIONAL INFORMATION</info>
              </entry>
              <entry>
                ...
              </entry>
            </backtrace>
            <start-time>START TIME OF TEST (ISO 8601 format) [e.g.: 2013-11-12T03:32:56.691823Z]</start-time>
            <elapsed>ELAPSED TIME OF TEST (in seconds) [e.g.: 0.030883]</elapsed>
          </result>
          <result>
            ...
          </result>
          ...
        </report>

### Test coverage

You can see the code coverage with Cutter if your system have LTP tools . To see the coverage, add the followling line in your configure.ac and type "make coverage".

    AC_CHECK_COVERAGE

## References

### Assertions

See cutter/cut-assertions.h .

### Attributes

You can add attributes to your test to get more useful information on failure. For example, you can add Bug ID like the following

    void attributes_invalid_input(void);
        void test_invalid_input(void);

        void
        attributes_invalid_input (void)
        {
            cut_set_attributes("bug", "123");
        }

        void
        test_invalid_input (void)
        {
            cut_assert_equal("OK", get_input());
        }

In the above example, test_invalid_input test has an attribute that the test is for Bug #123.

You need to define a function whose name is "attributes_#{TEST_NAME - 'test_' PREFIX}" to add attributes to a test. In the above example, attributes set function, "attributes_invalid_input", is defined to set "bug" attribute to "test_invalid_input" test.

### Template

The following is a template of test.

    #include <cutter.h>

        #include "HEADER_FILE_OF_YOUR_PROGRAM"

        void test_condition(void);
        void test_strstr(void);

        static int condition = 0;

        void
        cut_setup (void)
        {
            condition = 1;
        }

        void
        cut_teardown (void)
        {
            condition = 0;
        }

        void
        test_condition(void)
        {
            cut_set_message("The condition value should be set to 1 in cut_setup()");
            cut_assert_equal_int(1, condition);
          ...
        }

        void
        test_strstr(void)
        {
            cut_assert_equal_string("sub-string",
                                    strstr("string sub-string", "sub"));
            ...
        }

## Thanks

- Kazumasa Matsunaga: reported a build bug.

- Daijiro MORI: reported bugs.

- UNNO Hideyuki:

  - reported a document bug.

  - assisted Solaris install document.

- gunyara-kun: suggested API design.

- Yamakawa Hiroshi: reported works on Cygwin.

- Yoshinori K. Okuji:

  - reported locale related bugs.

  - suggested a new feature.

- Zed Shaw: reported bugs.

- Romuald Conty: reported a document bug.

- Romain Tartière:

  - reported bugs.

  - suggested improvements.

- Ilya Barygin:

  - reported bugs in test.

- Hiroaki Nakamura:

  - reported a bug on CentOS.

- Tobias Gruetzmacher:

  - fixed a GTK+ test runner bug.