tests: Enforce 100% URL coverage.

We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.

If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.

With some tweaks to the output from tabbott.

Fixes #1441.
This commit is contained in:
Steve Howell
2016-11-18 16:28:28 -08:00
committed by Tim Abbott
parent d8dee522b6
commit 5f5e6b6d83
3 changed files with 24 additions and 21 deletions

View File

@@ -186,8 +186,9 @@ class Runner(DiscoverRunner):
return failed
return failed
def run_tests(self, test_labels, extra_tests=None, **kwargs):
# type: (List[str], Optional[List[TestCase]], **Any) -> bool
def run_tests(self, test_labels, extra_tests=None,
full_suite=False, **kwargs):
# type: (List[str], Optional[List[TestCase]], bool, **Any) -> bool
self.setup_test_environment()
try:
suite = self.build_suite(test_labels, extra_tests)
@@ -208,5 +209,5 @@ class Runner(DiscoverRunner):
failed = self.run_suite(suite, fatal_errors=kwargs.get('fatal_errors'))
self.teardown_test_environment()
if not failed:
write_instrumentation_reports()
write_instrumentation_reports(full_suite=full_suite)
return failed