Merge pull request #38 from davidmreed/feature/update-docs

Prepare for 0.9.6

LICENSE

@@ -1,4 +1,4 @@
-Copyright 2018-2019 David Reed
+Copyright 2018-2020 David Reed
 
 Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
 

amaxa/amaxa.py

@@ -118,10 +118,6 @@ def __init__(self, connection):
         self.steps = []
         self.connection = connection
         self._bulk = None
-        self.describe_info = {}
-        self.field_maps = {}
-        self.proxy_objects = {}
-        self.key_prefix_map = None
         self.logger = logging.getLogger("amaxa")
         self.file_store = FileStore()
 

assets/test_data_transforms/about.md

@@ -2,9 +2,9 @@
 
 This test data suite is a demonstration of using Amaxa to apply both custom and built-in transformations on data during load and extract operations. A set of example data is included in CSV format and can be loaded to a Salesforce DX scratch org. It's used as part of Amaxa's automated tests, and can also be used to experiment.
 
-The `test.yml` file defines an operation that loads only one sObject (`Account`). It applies two transformations: the `Name` field is lowercased, and a custom transformation (defined in `test_transforms.py`) called `multiply` is applied to the `Description` field. `multiply` takes options in the operation definition, here an integer `count` for how many times to repeat the content of the field.
+The `test.yml` file defines an operation that loads only one sObject (`Account`). It applies two transformations: the `Name` field is lowercased, and a custom transformation (defined in `example_transforms.py`) called `multiply` is applied to the `Description` field. `multiply` takes options in the operation definition, here an integer `count` for how many times to repeat the content of the field.
 
-`test_transforms.py` is an example of how to implement your own transform logic for data operations. Note that the operation definition loads this file via the `plugin-modules` key.
+`example_transforms.py` is an example of how to implement your own transform logic for data operations. Note that the operation definition loads this file via the `plugin-modules` key.
 
 To try out loading the sample data set, spin up a Salesforce DX scratch org, sandbox, or developer org, and place the credentials in a `credentials.yml` file (an empty template is included). If using a Salesforce DX org, give the org the alias `amaxa`, and use the `credentials-sfdx.yml` file to automatically authenticate into it.
 

assets/test_data_transforms/example_transforms.py

@@ -6,7 +6,7 @@ class MultiplyTransformer(TransformProvider):
     transform_name = "multiply"
     allowed_types = ["xsd:string"]
 
-    def _get_transform(self, field_context: str, options: Dict):
+    def _get_transform(self, field_context: Dict, options: Dict):
         def multiply(x):
             return x * options["count"]
 

docs/transforms.rst

@@ -38,12 +38,14 @@ Amaxa ships with five transforms:
 Custom Transforms
 *****************
 
-Amaxa also supports custom transforms. To implement a custom transformation, create a Python module containing a class that subclasses ``amaxa.transforms.TransformProvider``. Subclasses must populate the ``transform_name`` class attribute and override two methods, ``get_transform()`` and ``get_options_schema()``.
+Amaxa also supports custom transforms. To implement a custom transformation, create a Python module containing a class that subclasses ``amaxa.transforms.TransformProvider``. Subclasses must populate the ``transform_name`` class attribute and override two methods, ``_get_transform()`` and ``get_options_schema()``.
 
-``get_transform()`` accepts the API name of the configured field and a ``dict`` containing the user-specified options, if any. It returns a callable that will be invoked with a single parameter, the field value, each time the transform is used.
+``_get_transform()`` accepts a ``dict`` representing the describe of the configured field and a ``dict`` containing the user-specified options, if any. It returns a callable that will be invoked with a single parameter, the field value, each time the transform is used.
 
 ``get_options_schema()`` returns the schema by which the user-specifiable options will be validated in the operation definition. This is a `Cerberus <https://docs.python-cerberus.org/>`_ schema that should validate a ``dict``.
 
+Transforms may optionally override ``_validate_field()``, which also accepts a ``dict`` representing the describe of the configured field. This method should raise a ``TransformException`` if the transform cannot be applied to this field for any reason. The default implementation requires that the field's ``soapType`` attribute be one of the values in the class attribute ``allowed_types``, a string list.
+
 Before being used in an operation, custom transforms must be loaded by specifying them in a ``plugin-modules`` list at the top level of the YAML configuration. Specify the name of the containing module, not the transform class; Amaxa will automatically discover transforms in those modules. Modules may be located in the current working directory or anywhere in the Python search path.
 
 A complete example of using a custom transform is included in Amaxa's repository in the ``assets/test_data_transforms`` directory.