Skip to content

Latest commit

 

History

History
144 lines (112 loc) · 5.42 KB

cEP-0027.md

File metadata and controls

144 lines (112 loc) · 5.42 KB
Raw

coala Bears Testing API

Metadata
cEP 27
Version 0.1
Title coala Bears Testing API
Authors Sangam Kumar mailto:[email protected]
Status Proposed
Type Feature

Abstract

This cEP describes the process of adding BaseTestHelper class and GlobalBearTestHelper class to improve the testing API of coala bears, as a part of the GSoC 2018 project.

Introduction

coala has its own testing API which includes LocalBearTestHelper, a helper class for testing of local bears. It does not provide support for GlobalBear. Writing tests for a GlobalBear is slightly lengthy for a developer. Example: GitCommitBearTest, VultureBearTest, etc. GlobalBearTestHelper is going to be a helper class for simplification of testing of GlobalBear.

Also, the testing API should have a base test helper class which should be inherited in all tests' helper (LocalBearTestHelper, GlobalBearTestHelper). This project introduces BaseTestHelper class, base class for all bear tests.

Proposed Approach

1. Implementation of the BaseTestHelper class

Issue: coala/coala#3676

As mentioned in introduction, BaseTestHelper class is the base class for all bear tests.

class BaseTestHelper(object):
    pass

Now, inherit the base test class in LocalBearTestHelper and GlobalBearTestBear. E.g.,

class LocalBearTestHelper(BaseTestHelper):
    pass

class GlobalBearTestHelper(BaseTestHelper):
    pass

After implementation of BaseTestHelper class, it should be ensured that all bear tests inherit from BaseTestHelper class using pytest-restrict. For example, in LocalBearTestHelperTest,

    out = self.runpytest('--restrict-types',
                         'BaseTestHelper',
                         'LocalBearTestHelper')
    out.assert_outcomes(passed=2, failed=0)

2. Implementation of GlobalBearTestHelper class

Issue: coala/coala#4884

Currently, many methods are redundant in the tests of GlobalBear. For example, all GlobalBears' tests have one method to collect absolute path of test files and then those files are checked using the bear.

  def get_absolute_test_path(file):
      return "path/of/test_file/file"

  def get_results(self, files_to_check):
      self.file_dict = [get_absolute_test_path(files_to_check)]
      self.uut = AnyGlobalBear(self.file_dict, self.section, self.queue)
      return list(self.uut.run())

  def test_results(self):
      results = self.get_results([self.test_files1, self.test_files2])
      messages = [result.message for result in results]
      assert messages == ["This is message 1.",
                          "This is message 2."]

Instead of writing these methods each time during tests of GlobalBear, it should be written once in GlobalBearTestHelper class with less flaws and better design which will be easy to reuse and less effort will be required to maintain.

Here is the prototype implementation of these repeating methods into GlobalBearTestHelper using pytest, and GlobalBear:

def get_test_path(test_dir, file):
    return os.path.join(os.path.dirname(__file__), test_dir, file)

def get_results(global_bear, test_dir, file_dict, settings={}):
    for file in file_dict:
        with get_test_path(test_dir, file) as fname,
            execute_bear(global_bear, fname, **settings) as bear_output:
            return bear_output

@contextmanager
def execute_bear(bear, *args, **kwargs):
    try:
        bear_output_generator = bear.execute(*args, **kwargs)
        assert bear_output_generator is not None, \
            'Bear returned None on execution\n'
        yield bear_output_generator
    except Exception as err:
        msg = []
        while not bear.message_queue.empty():
            msg.append(bear.message_queue.get().message)
        raise AssertionError(str(err) + ' \n' + '\n'.join(msg))
    return list(bear_output_generator)


class GlobalBearTestHelper(BaseTestHelper):
    def check_results(self, global_bear, test_dir, file_dict, results, settings={}):
        if results in [[], ()]:
            msg = ("The global bear '{}' yields a result although "
                   "it shouldn't.".format(global_bear.__class__.__name__))

        else:
            msg = ("The global bear '{}' doesn't yield the right "
                   "results.".format(global_bear.__class__.__name__))

        bear_output = get_results(global_bear,
                                  test_dir=test_dir,
                                  file_dict=file_dict,
                                  settings=settings)

        assert bear_output == results, msg=msg
        return bear_output