import catch2 3.8.1vendor/catch2/3.8.1vendor/catch2

1 files changed, 213 insertions, 0 deletions

diff --git a/docs/reporters.md b/docs/reporters.md
new file mode 100644
index 0000000..20ef5e5
--- /dev/null
+++ b/docs/reporters.md
@@ -0,0 +1,213 @@
+<a id="top"></a>
+# Reporters
+
+Reporters are a customization point for most of Catch2's output, e.g.
+formatting and writing out [assertions (whether passing or failing),
+sections, test cases, benchmarks, and so on](reporter-events.md#top).
+
+Catch2 comes with a bunch of reporters by default (currently 9), and
+you can also write your own reporter. Because multiple reporters can
+be active at the same time, your own reporters do not even have to handle
+all reporter event, just the ones you are interested in, e.g. benchmarks.
+
+
+## Using different reporters
+
+You can see which reporters are available by running the test binary
+with `--list-reporters`. You can then pick one of them with the [`-r`,
+`--reporter` option](command-line.md#choosing-a-reporter-to-use), followed
+by the name of the desired reporter, like so:
+
+```
+--reporter xml
+```
+
+You can also select multiple reporters to be used at the same time.
+In that case you should read the [section on using multiple
+reporters](#multiple-reporters) to avoid any surprises from doing so.
+
+
+<a id="multiple-reporters"></a>
+## Using multiple reporters
+
+> Support for having multiple parallel reporters was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 3.0.1
+
+Catch2 supports using multiple reporters at the same time while having
+them write into different destinations. The two main uses of this are
+
+* having both human-friendly and machine-parseable (e.g. in JUnit format)
+  output from one run of binary
+* having "partial" reporters that are highly specialized, e.g. having one
+  reporter that writes out benchmark results as markdown tables and does
+  nothing else, while also having standard testing output separately
+
+Specifying multiple reporter looks like this:
+```
+--reporter JUnit::out=result-junit.xml --reporter console::out=-::colour-mode=ansi
+```
+
+This tells Catch2 to use two reporters, `JUnit` reporter that writes
+its machine-readable XML output to file `result-junit.xml`, and the
+`console` reporter that writes its user-friendly output to stdout and
+uses ANSI colour codes for colouring the output.
+
+Using multiple reporters (or one reporter and one-or-more [event
+listeners](event-listeners.md#top)) can have surprisingly complex semantics
+when using customization points provided to reporters by Catch2, namely
+capturing stdout/stderr from test cases.
+
+As long as at least one reporter (or listener) asks Catch2 to capture
+stdout/stderr, captured stdout and stderr will be available to all
+reporters and listeners.
+
+Because this might be surprising to the users, if at least one active
+_reporter_ is non-capturing, then Catch2 tries to roughly emulate
+non-capturing behaviour by printing out the captured stdout/stderr
+just before `testCasePartialEnded` event is sent out to the active
+reporters and listeners. This means that stdout/stderr is no longer
+printed out from tests as it is being written, but instead it is written
+out in batch after each runthrough of a test case is finished.
+
+
+
+## Writing your own reporter
+
+You can also write your own custom reporter and tell Catch2 to use it.
+When writing your reporter, you have two options:
+
+* Derive from `Catch::ReporterBase`. When doing this, you will have
+  to provide handling for all [reporter events](reporter-events.md#top).
+* Derive from one of the provided [utility reporter bases in
+  Catch2](#utility-reporter-bases).
+
+Generally we recommend doing the latter, as it is less work.
+
+Apart from overriding handling of the individual reporter events, reporters
+have access to some extra customization points, described below.
+
+
+### Utility reporter bases
+
+Catch2 currently provides two utility reporter bases:
+
+* `Catch::StreamingReporterBase`
+* `Catch::CumulativeReporterBase`
+
+`StreamingReporterBase` is useful for reporters that can format and write
+out the events as they come in. It provides (usually empty) implementation
+for all reporter events, and if you let it handle the relevant events,
+it also handles storing information about active test run and test case.
+
+`CumulativeReporterBase` is a base for reporters that need to see the whole
+test run, before they can start writing the output, such as the JUnit
+and SonarQube reporters. This post-facto approach requires the assertions
+to be stringified when it is finished, so that the assertion can be written
+out later. Because the stringification can be expensive, and not all
+cumulative reporters need the assertions, this base provides customization
+point to change whether the assertions are saved or not, separate for
+passing and failing assertions.
+
+
+_Generally we recommend that if you override a member function from either
+of the bases, you call into the base's implementation first. This is not
+necessarily in all cases, but it is safer and easier._
+
+
+Writing your own reporter then looks like this:
+
+```cpp
+#include <catch2/reporters/catch_reporter_streaming_base.hpp>
+#include <catch2/catch_test_case_info.hpp>
+#include <catch2/reporters/catch_reporter_registrars.hpp>
+
+#include <iostream>
+
+class PartialReporter : public Catch::StreamingReporterBase {
+public:
+    using StreamingReporterBase::StreamingReporterBase;
+
+    static std::string getDescription() {
+        return "Reporter for testing TestCasePartialStarting/Ended events";
+    }
+
+    void testCasePartialStarting(Catch::TestCaseInfo const& testInfo,
+                                 uint64_t partNumber) override {
+        std::cout << "TestCaseStartingPartial: " << testInfo.name << '#' << partNumber << '\n';
+    }
+
+    void testCasePartialEnded(Catch::TestCaseStats const& testCaseStats,
+                              uint64_t partNumber) override {
+        std::cout << "TestCasePartialEnded: " << testCaseStats.testInfo->name << '#' << partNumber << '\n';
+    }
+};
+
+
+CATCH_REGISTER_REPORTER("partial", PartialReporter)
+```
+
+This create a simple reporter that responds to `testCasePartial*` events,
+and calls itself "partial" reporter, so it can be invoked with
+`--reporter partial` command line flag.
+
+
+### `ReporterPreferences`
+
+Each reporter instance contains instance of `ReporterPreferences`, a type
+that holds flags for the behaviour of Catch2 when this reporter run.
+Currently there are two customization options:
+
+* `shouldRedirectStdOut` - whether the reporter wants to handle
+   writes to stdout/stderr from user code, or not. This is useful for
+   reporters that output machine-parseable output, e.g. the JUnit
+   reporter, or the XML reporter.
+* `shouldReportAllAssertions` - whether the reporter wants to handle
+  `assertionEnded` events for passing assertions as well as failing
+   assertions. Usually reporters do not report successful assertions
+   and don't need them for their output, but sometimes the desired output
+   format includes passing assertions even without the `-s` flag.
+
+
+### Per-reporter configuration
+
+> Per-reporter configuration was introduced in Catch2 3.0.1
+
+Catch2 supports some configuration to happen per reporter. The configuration
+options fall into one of two categories:
+
+* Catch2-recognized options
+* Reporter-specific options
+
+The former is a small set of universal options that Catch2 handles for
+the reporters, e.g. output file or console colour mode. The latter are
+options that the reporters have to handle themselves, but the keys and
+values can be arbitrary strings, as long as they don't contain `::`. This
+allows writing reporters that can be significantly customized at runtime.
+
+Reporter-specific options always have to be prefixed with "X" (large
+letter X).
+
+
+### Other expected functionality of a reporter
+
+When writing a custom reporter, there are few more things that you should
+keep in mind. These are not important for correctness, but they are
+important for the reporter to work _nicely_.
+
+* Catch2 provides a simple verbosity option for users. There are three
+  verbosity levels, "quiet", "normal", and "high", and if it makes sense
+  for reporter's output format, it should respond to these by changing
+  what, and how much, it writes out.
+
+* Catch2 operates with an rng-seed. Knowing what seed a test run had
+  is important if you want to replicate it, so your reporter should
+  report the rng-seed, if at all possible given the target output format.
+
+* Catch2 also operates with test filters, or test specs. If a filter
+  is present, you should also report the filter, if at all possible given
+  the target output format.
+
+
+
+---
+
+[Home](Readme.md#top)