Release 0.1.35 (#74)

* Improved the error message when the call to the parent class constructor is missing in a test fixture * Fixing the Environment variale setting documentation for Windows Powershell * Apply suggestions from code review Co-authored-by: Omri Mendels <[email protected]> * Feature: Discover test files with '_test' suffix (#47) * Enable test discovery for test names with suffix 'test' * Combine redundant suffix tests * Remove old suffix tests * Encapsulate test name parsing and have nuttercli call test name validation from api * Have api client results call _is_valid_test_name from api Co-authored-by: Quan Nguyen <[email protected]> * Invalid State response is retriable (#49) * fixed import error and refactoring * invalid state is retriable, pull sleep is 5 seconds * Poll wait time as flag (#51) * poll wait time as flag * lint fixes * Parallel runner implementation (#59) * Toc Update (#30) * Release v0.1.34 (#52) * Improved the error message when the call to the parent class constructor is missing in a test fixture * Fixing the Environment variale setting documentation for Windows Powershell * Apply suggestions from code review Co-authored-by: Omri Mendels <[email protected]> * Feature: Discover test files with '_test' suffix (#47) * Enable test discovery for test names with suffix 'test' * Combine redundant suffix tests * Remove old suffix tests * Encapsulate test name parsing and have nuttercli call test name validation from api * Have api client results call _is_valid_test_name from api Co-authored-by: Quan Nguyen <[email protected]> * Invalid State response is retriable (#49) * fixed import error and refactoring * invalid state is retriable, pull sleep is 5 seconds * Poll wait time as flag (#51) * poll wait time as flag * lint fixes Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Update README.md * Parallel runner * Rename helper class * Fix collect results * Rename execute method * Use new execute method * Introduce add_test_fixture() method Co-authored-by: Jesus Aguilar <[email protected]> Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Fix Py4JJavaError serialization issue (#60) * Toc Update (#30) * Release v0.1.34 (#52) * Improved the error message when the call to the parent class constructor is missing in a test fixture * Fixing the Environment variale setting documentation for Windows Powershell * Apply suggestions from code review Co-authored-by: Omri Mendels <[email protected]> * Feature: Discover test files with '_test' suffix (#47) * Enable test discovery for test names with suffix 'test' * Combine redundant suffix tests * Remove old suffix tests * Encapsulate test name parsing and have nuttercli call test name validation from api * Have api client results call _is_valid_test_name from api Co-authored-by: Quan Nguyen <[email protected]> * Invalid State response is retriable (#49) * fixed import error and refactoring * invalid state is retriable, pull sleep is 5 seconds * Poll wait time as flag (#51) * poll wait time as flag * lint fixes Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Update README.md * Fix Py4JJavaError serialization issue * Test Py4JJavaError using mocks * Remove Py4J stuff * Still need to install py4j to run tests Co-authored-by: Jesus Aguilar <[email protected]> Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Add notebook params and debugger (#68) * Toc Update (#30) * Release v0.1.34 (#52) * Improved the error message when the call to the parent class constructor is missing in a test fixture * Fixing the Environment variale setting documentation for Windows Powershell * Apply suggestions from code review Co-authored-by: Omri Mendels <[email protected]> * Feature: Discover test files with '_test' suffix (#47) * Enable test discovery for test names with suffix 'test' * Combine redundant suffix tests * Remove old suffix tests * Encapsulate test name parsing and have nuttercli call test name validation from api * Have api client results call _is_valid_test_name from api Co-authored-by: Quan Nguyen <[email protected]> * Invalid State response is retriable (#49) * fixed import error and refactoring * invalid state is retriable, pull sleep is 5 seconds * Poll wait time as flag (#51) * poll wait time as flag * lint fixes Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Update README.md * Add notebook params and debugger * Add example in README.md * Have more explicit notebook_params error message * Revert "Add example in README.md" This reverts commit 1aac73c. * Add examples for notebook params Co-authored-by: Jesus Aguilar <[email protected]> Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> Co-authored-by: Neyissa Exilus <[email protected]> * upgraded python version * Feature: Discover test files with '_test' suffix (#47) * Enable test discovery for test names with suffix 'test' * Combine redundant suffix tests * Remove old suffix tests * Encapsulate test name parsing and have nuttercli call test name validation from api * Have api client results call _is_valid_test_name from api Co-authored-by: Quan Nguyen <[email protected]> * Invalid State response is retriable (#49) * fixed import error and refactoring * invalid state is retriable, pull sleep is 5 seconds * Poll wait time as flag (#51) * poll wait time as flag * lint fixes * Parallel runner implementation (#59) * Toc Update (#30) * Release v0.1.34 (#52) * Improved the error message when the call to the parent class constructor is missing in a test fixture * Fixing the Environment variale setting documentation for Windows Powershell * Apply suggestions from code review Co-authored-by: Omri Mendels <[email protected]> * Feature: Discover test files with '_test' suffix (#47) * Enable test discovery for test names with suffix 'test' * Combine redundant suffix tests * Remove old suffix tests * Encapsulate test name parsing and have nuttercli call test name validation from api * Have api client results call _is_valid_test_name from api Co-authored-by: Quan Nguyen <[email protected]> * Invalid State response is retriable (#49) * fixed import error and refactoring * invalid state is retriable, pull sleep is 5 seconds * Poll wait time as flag (#51) * poll wait time as flag * lint fixes Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Update README.md * Parallel runner * Rename helper class * Fix collect results * Rename execute method * Use new execute method * Introduce add_test_fixture() method Co-authored-by: Jesus Aguilar <[email protected]> Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Fix Py4JJavaError serialization issue (#60) * Toc Update (#30) * Release v0.1.34 (#52) * Improved the error message when the call to the parent class constructor is missing in a test fixture * Fixing the Environment variale setting documentation for Windows Powershell * Apply suggestions from code review Co-authored-by: Omri Mendels <[email protected]> * Feature: Discover test files with '_test' suffix (#47) * Enable test discovery for test names with suffix 'test' * Combine redundant suffix tests * Remove old suffix tests * Encapsulate test name parsing and have nuttercli call test name validation from api * Have api client results call _is_valid_test_name from api Co-authored-by: Quan Nguyen <[email protected]> * Invalid State response is retriable (#49) * fixed import error and refactoring * invalid state is retriable, pull sleep is 5 seconds * Poll wait time as flag (#51) * poll wait time as flag * lint fixes Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Update README.md * Fix Py4JJavaError serialization issue * Test Py4JJavaError using mocks * Remove Py4J stuff * Still need to install py4j to run tests Co-authored-by: Jesus Aguilar <[email protected]> Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Add notebook params and debugger (#68) * Toc Update (#30) * Release v0.1.34 (#52) * Improved the error message when the call to the parent class constructor is missing in a test fixture * Fixing the Environment variale setting documentation for Windows Powershell * Apply suggestions from code review Co-authored-by: Omri Mendels <[email protected]> * Feature: Discover test files with '_test' suffix (#47) * Enable test discovery for test names with suffix 'test' * Combine redundant suffix tests * Remove old suffix tests * Encapsulate test name parsing and have nuttercli call test name validation from api * Have api client results call _is_valid_test_name from api Co-authored-by: Quan Nguyen <[email protected]> * Invalid State response is retriable (#49) * fixed import error and refactoring * invalid state is retriable, pull sleep is 5 seconds * Poll wait time as flag (#51) * poll wait time as flag * lint fixes Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> * Update README.md * Add notebook params and debugger * Add example in README.md * Have more explicit notebook_params error message * Revert "Add example in README.md" This reverts commit 1aac73c. * Add examples for notebook params Co-authored-by: Jesus Aguilar <[email protected]> Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> Co-authored-by: Neyissa Exilus <[email protected]> * upgraded python version * version bump, python 3.7 * added py4j requirement * doc update * doc update Co-authored-by: RobBagby <[email protected]> Co-authored-by: Prakash Kudkuli Vishnu <[email protected]> Co-authored-by: Omri Mendels <[email protected]> Co-authored-by: quanuw <[email protected]> Co-authored-by: Quan Nguyen <[email protected]> Co-authored-by: Thomas Conté <[email protected]> Co-authored-by: Andrew Francisque <[email protected]> Co-authored-by: Neyissa Exilus <[email protected]>
microsoft · Dec 16, 2022 · 368248b · 368248b
1 parent e60be0e
commit 368248b
Show file tree

Hide file tree

Showing 13 changed files with 370 additions and 36 deletions.
diff --git a/.github/workflows/pythonpackage.yml b/.github/workflows/pythonpackage.yml
@@ -9,7 +9,7 @@ jobs:
     strategy:
       max-parallel: 4
       matrix:
-        python-version: [3.5]
+        python-version: [3.7]
 
     steps:
     - uses: actions/checkout@v1
@@ -34,5 +34,5 @@ jobs:
         flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
     - name: Test with pytest
       run: |
-        pip install pytest
+        pip install pytest py4j
         pytest
diff --git a/.vscode/example_launch.json b/.vscode/example_launch.json
@@ -0,0 +1,20 @@
+{
+    "version": "0.2.0",
+    "configurations": [
+    {
+    "name": "Python: Current File",
+    "type": "python",
+    "request": "launch",
+    "console": "integratedTerminal",
+    "python": "python3",
+    "module": "cli.nuttercli",
+    "args": [
+        "run",
+        "<Add test pattern here>",
+        "--cluster_id",
+        "<Add cluster_id here>",
+        "--notebook_params",
+        "{\"example_key_1\": \"example_value_1\", \"example_key_2\": \"example_value_2\"}"
+        ]
+    }]
+}
diff --git a/.vscode/settings.json b/.vscode/settings.json
@@ -1,9 +1,10 @@
 {
-    "python.pythonPath": "/usr/bin/python3",
+    "python.pythonPath": "/usr/bin/python3", 
     "python.testing.pytestArgs": [
         "tests"
     ],
     "python.testing.unittestEnabled": false,
     "python.testing.nosetestsEnabled": false,
-    "python.testing.pytestEnabled": true
+    "python.testing.pytestEnabled": true,
+    "python.envFile": "${workspaceFolder}/.env"
 }
diff --git a/README.md b/README.md
@@ -7,11 +7,12 @@
   * [Cluster Installation](#cluster-installation)
   * [Nutter Fixture](#nutter-fixture)
   * [Test Cases](#test-cases)
-  * [before_all and after_all](#before-all-and-after-all)
+  * [*before_all* and *after_all*](#before-all-and-after-all)
+  * [Running test fixtures in parallel](#running-test-fixtures-in-parallel)
 - [Nutter CLI](#nutter-cli)
   * [Getting Started with the Nutter CLI](#getting-started-with-the-nutter-cli)
-  * [Listing Test Notebooks](#listing-test-notebooks)
-  * [Executing Test Notebooks](#executing-test-notebooks)
+  * [Listing test Notebooks](#listing-test-notebooks)
+  * [Executing test Notebooks](#executing-test-notebooks)
   * [Run single test notebook](#run-single-test-notebook)
   * [Run multiple tests notebooks](#run-multiple-tests-notebooks)
   * [Parallel Execution](#parallel-execution)
@@ -198,6 +199,64 @@ class TestFixture(NutterFixture):
     NutterFixture.__init__(self)
 ```
 
+### Running test fixtures in parallel
+
+
+Version 0.1.35 includes a parallel runner class ```NutterFixtureParallelRunner```  that facilitates the execution of test fixtures concurrently. This approach could significantly increase the performance of your testing pipeline.
+
+The following code executes two fixtures, ```CustomerTestFixture``` and  ```CountryTestFixture``` in parallel.
+
+```Python
+from runtime.runner import NutterFixtureParallelRunner
+from runtime.nutterfixture import NutterFixture, tag
+class CustomerTestFixture(NutterFixture):
+   def run_customer_data_is_inserted(self):
+      dbutils.notebook.run('../data/customer_data_import', 600)
+
+   def assertion_customer_data_is_inserted(self):
+      some_tbl = sqlContext.sql('SELECT COUNT(*) AS total FROM customers')
+      first_row = some_tbl.first()
+      assert (first_row[0] == 1)
+
+class CountryTestFixture(NutterFixture):
+   def run_country_data_is_inserted(self):
+      dbutils.notebook.run('../data/country_data_import', 600)
+
+   def assertion_country_data_is_inserted(self):
+      some_tbl = sqlContext.sql('SELECT COUNT(*) AS total FROM countries')
+      first_row = some_tbl.first()
+      assert (first_row[0] == 1)
+
+parallel_runner = NutterFixtureParallelRunner(num_of_workers=2)
+parallel_runner.add_test_fixture(CustomerTestFixture())
+parallel_runner.add_test_fixture(CountryTestFixture())
+
+result = parallel_runner.execute()
+print(result.to_string())
+# Comment out the next line (result.exit(dbutils)) to see the test result report from within the notebook
+# result.exit(dbutils)
+
+```
+
+The parallel runner combines the test results of both fixtures in a single result.
+
+``` bash
+Notebook: N/A - Lifecycle State: N/A, Result: N/A
+Run Page URL: N/A
+============================================================
+PASSING TESTS
+------------------------------------------------------------
+country_data_is_inserted (11.446587234000617 seconds)
+customer_data_is_inserted (11.53276599000128 seconds)
+
+
+============================================================
+
+Command took 11.67 seconds -- by [email protected] at 12/15/2022, 9:34:24 PM on Foo Cluster
+```
+
+
+
 ## Nutter CLI
 
 The Nutter CLI is a command line interface that allows you to execute and list tests via a Command Prompt.
@@ -230,7 +289,7 @@ $env:DATABRICKS_TOKEN="TOKEN"
 
 __Note:__ For more information about personal access tokens review  [Databricks API Authentication](https://docs.azuredatabricks.net/dev-tools/api/latest/authentication.html).
 
-### Listing Test Notebooks
+### Listing test notebooks
 
 The following command list all test notebooks in the folder ```/dataload```
 
@@ -248,16 +307,16 @@ You can list all test notebooks in the folder structure using the ```--recursive
 nutter list /dataload --recursive
 ```
 
-### Executing Test Notebooks
+### Executing test notebooks
 
 The ```run``` command  schedules the execution of test notebooks and waits for their result.
 
 ### Run single test notebook
 
-The following command executes the test notebook ```/dataload/test_sourceLoad``` in the cluster ```0123-12334-tonedabc```.
+The following command executes the test notebook ```/dataload/test_sourceLoad``` in the cluster ```0123-12334-tonedabc``` with the notebook_param key-value pairs of ```{"example_key_1": "example_value_1", "example_key_2": "example_value_2"}``` (Please note the escaping of quotes):
 
 ```bash
-nutter run dataload/test_sourceLoad --cluster_id 0123-12334-tonedabc
+nutter run dataload/test_sourceLoad --cluster_id 0123-12334-tonedabc --notebook_params "{\"example_key_1\": \"example_value_1\", \"example_key_2\": \"example_value_2\"}"
 ```
 
 __Note:__ In Azure Databricks you can get the cluster ID by selecting a cluster name from the Clusters tab and clicking on the JSON view.
@@ -267,10 +326,10 @@ __Note:__ In Azure Databricks you can get the cluster ID by selecting a cluster
 The Nutter CLI supports the execution of multiple notebooks via name pattern matching. The Nutter CLI applies the pattern to the name of test notebook **without** the *test_* prefix. The CLI also expects that you omit the prefix when specifying the pattern.
 
 
-Say the *dataload* folder has the following test notebooks: *test_srcLoad* and *test_srcValidation*. The following command will result in the execution of both tests.
+Say the *dataload* folder has the following test notebooks: *test_srcLoad* and *test_srcValidation* with the notebook_param key-value pairs of ```{"example_key_1": "example_value_1", "example_key_2": "example_value_2"}```. The following command will result in the execution of both tests.
 
 ```bash
-nutter run dataload/src* --cluster_id 0123-12334-tonedabc
+nutter run dataload/src* --cluster_id 0123-12334-tonedabc --notebook_params "{\"example_key_1\": \"example_value_1\", \"example_key_2\": \"example_value_2\"}" 
 ```
 
 In addition, if you have tests in a hierarchical folder structure, you can recursively execute all tests by setting the ```--recursive``` flag.
@@ -316,6 +375,10 @@ FLAGS
     --max_parallel_tests   Sets the level of parallelism for test notebook execution.
     --recursive            Executes all tests in the hierarchical folder structure. 
     --poll_wait_time       Polling interval duration for notebook status. Default is 5 (5 seconds).
+    --notebook_params      Allows parameters to be passed from the CLI tool to the test notebook. From the 
+                           notebook, these parameters can then be accessed by the notebook using 
+                           the 'dbutils.widgets.get('key')' syntax.
+
 ```
 
 __Note:__ You can also use flags syntax for POSITIONAL ARGUMENTS
@@ -435,6 +498,9 @@ steps:
   condition: succeededOrFailed()
 ```
 
+### Debugging Locally
+If using Visual Studio Code, you can use the `example_launch.json` file provided, editing the variables in the `<>` symbols to match your environment. You should be able to use the debugger to see the test run results, much the same as you would in Azure Devops.
+
 ## Contributing
 
 ### Contribution Tips

diff --git a/cli/nuttercli.py b/cli/nuttercli.py
@@ -17,7 +17,7 @@
 from .reportsman import ReportWriters
 from . import reportsman as reports
 
-__version__ = '0.1.34'
+__version__ = '0.1.35'
 
 BUILD_NUMBER_ENV_VAR = 'NUTTER_BUILD_NUMBER'
 
@@ -53,22 +53,22 @@ def __init__(self, debug=False, log_to_file=False, version=False):
     def run(self, test_pattern, cluster_id,
             timeout=120, junit_report=False,
             tags_report=False, max_parallel_tests=1,
-            recursive=False, poll_wait_time=DEFAULT_POLL_WAIT_TIME):
+            recursive=False, poll_wait_time=DEFAULT_POLL_WAIT_TIME, notebook_params=None):
         try:
-            logging.debug(""" Running tests. test_pattern: {} cluster_id: {} timeout: {}
+            logging.debug(""" Running tests. test_pattern: {} cluster_id: {}  notebook_params: {} timeout: {}
                                junit_report: {} max_parallel_tests: {}
                                tags_report: {}  recursive:{} """
                           .format(test_pattern, cluster_id, timeout,
                                   junit_report, max_parallel_tests,
-                                  tags_report, recursive))
+                                  tags_report, recursive, notebook_params))
 
             logging.debug("Executing test(s): {}".format(test_pattern))
 
             if self._is_a_test_pattern(test_pattern):
                 logging.debug('Executing pattern')
                 results = self._nutter.run_tests(
                     test_pattern, cluster_id, timeout,
-                    max_parallel_tests, recursive, poll_wait_time)
+                    max_parallel_tests, recursive, poll_wait_time, notebook_params)
                 self._nutter.events_processor_wait()
                 self._handle_results(results, junit_report, tags_report)
                 return

diff --git a/common/api.py b/common/api.py
@@ -88,21 +88,21 @@ def list_tests(self, path, recursive=False):
         return tests
 
     def run_test(self, testpath, cluster_id,
-                 timeout=120, pull_wait_time=DEFAULT_POLL_WAIT_TIME):
+                 timeout=120, pull_wait_time=DEFAULT_POLL_WAIT_TIME, notebook_params=None):
         self._add_status_event(NutterStatusEvents.TestExecutionRequest, testpath)
         test_notebook = TestNotebook.from_path(testpath)
         if test_notebook is None:
             raise InvalidTestException
 
         result = self.dbclient.execute_notebook(
             test_notebook.path, cluster_id,
-            timeout=timeout, pull_wait_time=pull_wait_time)
+            timeout=timeout, pull_wait_time=pull_wait_time, notebook_params=notebook_params)
 
         return result
 
     def run_tests(self, pattern, cluster_id,
                   timeout=120, max_parallel_tests=1, recursive=False,
-                  poll_wait_time=DEFAULT_POLL_WAIT_TIME):
+                  poll_wait_time=DEFAULT_POLL_WAIT_TIME, notebook_params=None):
 
         self._add_status_event(NutterStatusEvents.TestExecutionRequest, pattern)
         root, pattern_to_match = self._get_root_and_pattern(pattern)
@@ -119,7 +119,7 @@ def run_tests(self, pattern, cluster_id,
             NutterStatusEvents.TestsListingFiltered, len(filtered_notebooks))
 
         return self._schedule_and_run(
-            filtered_notebooks, cluster_id, max_parallel_tests, timeout, poll_wait_time)
+            filtered_notebooks, cluster_id, max_parallel_tests, timeout, poll_wait_time, notebook_params)
 
     def events_processor_wait(self):
         if self._events_processor is None:
@@ -168,20 +168,20 @@ def _get_root_and_pattern(self, pattern):
         return root, valid_pattern
 
     def _schedule_and_run(self, test_notebooks, cluster_id,
-                          max_parallel_tests, timeout, pull_wait_time):
+                          max_parallel_tests, timeout, pull_wait_time, notebook_params=None):
         func_scheduler = scheduler.get_scheduler(max_parallel_tests)
         for test_notebook in test_notebooks:
             self._add_status_event(
                 NutterStatusEvents.TestScheduling, test_notebook.path)
             logging.debug(
                 'Scheduling execution of: {}'.format(test_notebook.path))
             func_scheduler.add_function(self._execute_notebook,
-                                        test_notebook.path, cluster_id, timeout, pull_wait_time)
+                                        test_notebook.path, cluster_id, timeout, pull_wait_time, notebook_params)
         return self._run_and_await(func_scheduler)
 
-    def _execute_notebook(self, test_notebook_path, cluster_id, timeout, pull_wait_time):
+    def _execute_notebook(self, test_notebook_path, cluster_id, timeout, pull_wait_time, notebook_params=None):
         result = self.dbclient.execute_notebook(test_notebook_path,
-                                                cluster_id, None, timeout, pull_wait_time)
+                                                cluster_id, timeout, pull_wait_time, notebook_params)
         self._add_status_event(NutterStatusEvents.TestExecuted,
                                ExecutionResultEventData.from_execution_results(result))
         logging.debug('Executed: {}'.format(test_notebook_path))

diff --git a/common/apiclient.py b/common/apiclient.py
@@ -56,9 +56,9 @@ def list_objects(self, path):
 
         return workspace_path_obj
 
-    def execute_notebook(self, notebook_path, cluster_id,
-                         notebook_params=None, timeout=120,
-                         pull_wait_time=DEFAULT_POLL_WAIT_TIME):
+    def execute_notebook(self, notebook_path, cluster_id, timeout=120,
+                         pull_wait_time=DEFAULT_POLL_WAIT_TIME,
+                         notebook_params=None):
         if not notebook_path:
             raise ValueError("empty path")
         if not cluster_id:
@@ -68,7 +68,7 @@ def execute_notebook(self, notebook_path, cluster_id,
                 "Timeout must be greater than {}".format(self.min_timeout))
         if notebook_params is not None:
             if not isinstance(notebook_params, dict):
-                raise ValueError("Parameters must be a dictionary")
+                raise ValueError("Parameters must be in the form of a dictionary (See #run-single-test-notebook section in README)")
         if pull_wait_time <= 1:
             pull_wait_time = DEFAULT_POLL_WAIT_TIME
 

diff --git a/common/testresult.py b/common/testresult.py
@@ -3,9 +3,13 @@
 Licensed under the MIT license.
 """
 
-from .pickleserializable import PickleSerializable
-import pickle
 import base64
+import pickle
+
+from py4j.protocol import Py4JJavaError
+
+from .pickleserializable import PickleSerializable
+
 
 def get_test_results():
     return TestResults()
@@ -30,6 +34,9 @@ def append(self, testresult):
         self.total_execution_time = total_execution_time
 
     def serialize(self):
+        for i in self.results:
+            if isinstance(i.exception, Py4JJavaError):
+                i.exception = Exception(str(i.exception))
         bin_data = pickle.dumps(self)
         return str(base64.encodebytes(bin_data), "utf-8")
 

diff --git a/requirements.txt b/requirements.txt
@@ -2,4 +2,5 @@ databricks-api
 requests
 fire
 junit_xml
+py4j
-Original file line number
+Diff line change
@@ Expand Up / @@ -2,4 +2,5 @@ databricks-api @@
     requests
     fire
     junit_xml
+    py4j