# Testing libc++ using LIT

libc++ uses LIT to configure and run its tests. The primary way to run the libc++ tests is by using `make check-libcxx`. However since libc++ can be used in any number of possible configurations it is important to customize the way LIT builds and runs the tests. This guide provides information on how to use LIT directly to test libc++.

Documentation for LIT can be found here.

## Getting Started

After building libc++ use the following commands before you start using LIT to test.

- `alias lit='python path/to/llvm/utils/lit/lit.py'`
- `export LIBCXX_SITE_CONFIG=path/to/build-libcxx/test/lit.site.cfg`

You can now run the libc++ tests by running:

- `cd path/to/libcxx`
- `lit -sv ./test`

To only run a subsection of the tests use:

- `lit -sv test/std/numerics # Run only the numeric tests`

## Customization Options

libc++'s testsuite provides multiple options to configure the way the tests are build and run. To use these options you pass them on the LIT command line as `--param NAME` or `--param NAME=VALUE`. Some options have default values specified during CMake's configuration. Passing the option on the command line will override the default.

### libcxx_site_config=<path/to/lit.site.cfg>

> Specify the site configuration to use when running the tests. This option overrides the enviroment variable `LIBCXX_SITE_CONFIG`

### libcxx_headers=<path/to/headers>

> Specify the libc++ headers that are tested. By default the headers in the source tree are used.

### libcxx_library=<path/to/libc++.so>

> Specify the libc++ library that is tested. By default the library in the build directory is used. This option cannot be used when `use_system_lib` is provided.

### use_system_lib=<bool>

> Default: `False` Enable or disable testing against the installed version of libc++ library. Note: This does not use the installed headers.

### use_lit_shell=<bool>

> Enable or disable the use of LIT's internal shell in ShTests. If the enviroment variable `LIT_USE_INTERNAL_SHELL` is present then that is used as the default value. Otherwise the default value is `True` on Windows and `False` on every other platform.

### no_default_flags=<bool>

> Default: `False` Disable all default compile and link flags from being added. When this option is used only flags specified using the `compile_flags` and `link_flags` will be used.

### compile_flags="<list-of-args>"

> Specify additional compile flags as a space delimited string. Note: This options should not be used to change the standard version used.

### link_flags="<list-of-args>"

> Specify additional link flags as a space delimited string.

### std=<standard version>

> Values: `c++98, c++03, c++11, c++14, c++1z` Change the standard version used when building the tests.

### debug_level=<level>

> Values: `0, 1` Enable the use of debug mode. Level 0 enables assertions and level 1 enables assertions and debugging of iterator misuse.

### use_sanitizer=<sanitizer name>

> Values: `Memory, MemoryWithOrigins, Address, Undefined` Run the tests using the given sanitizer. If `LLVM_USE_SANITIZER` was given when building libc++ then that sanitizer will be used by default.

### color_diagnostics

> Enable the use of colorized compile diagnostics. If the `color_diagnostics` option is specified or the enviroment variable `LIBCXX_COLOR_DIAGNOSTICS` is present then color diagnostics will be enabled.