Model Driven Application Provider (#359)

* Initial read only version * Add docs * Docs and sample update * Adding entity record * MDA Provider updates #342 * Adding custom page support * Documentation update * WIP entitylist support * Foundations for CoE Kit test * Adding TestEngine.ConsentDialog * Adding TestEngine.ConsentDialog * Description update * removing format space and restoreing launchsettings * removing white space * Update sample application --------- Co-authored-by: Sourabh Namilikonda <[email protected]>
microsoft · Aug 3, 2024 · 0f63977 · 0f63977
1 parent 6408f99
commit 0f63977
Show file tree

Hide file tree

Showing 31 changed files with 3,257 additions and 4 deletions.
diff --git a/docs/Extensions/ModelDrivenApplicationProvider/README.md b/docs/Extensions/ModelDrivenApplicationProvider/README.md
@@ -0,0 +1,134 @@
+# Overview of the Model Driven Application Web Test Provider for the Power Apps Test Engine
+
+## Purpose
+
+The Model Driven Application Provider implements a [Managed Extensibility Framework](https://learn.microsoft.comdotnet/framework/mef/) (MEF) extension enhances the Power Apps Test Engine by enabling comprehensive unit and integration testing for web-based Model Driven Applications (MDA).
+
+The provider encapsulates the XRM SDK using Playwright by allowing tests to be created with low code Power Fx commands for test for automation, ensuring a seamless and efficient testing process.
+
+This extension build on the Test Engine Yaml definitions to define and execute test suites and cases, generating test results as TRX files, which can be integrated into CI/CD processes.
+
+## Testing Model-Driven Apps with Power Apps Test Engine
+
+When testing model-driven apps with the Power Apps Test Engine, you have a spectrum of choices ranging from unit tests with mocked data operations to full integration tests against a Dataverse environment. It's important to consider several key factors to ensure robust and efficient testing.
+
+### Key Points to Consider:
+
+- **Speed of Execution**
+  - **Unit Tests with Mock Data**: Typically faster as they do not interact with the actual Dataverse environment.
+  - **Integration Tests with Dataverse**: Slower due to real-time interaction with the Dataverse environment, but provide more comprehensive validation.
+
+- **Management of Test State as Dataverse Records vs Mock Data**
+  - **Mock Data**: Easier to control and reset between tests, ensuring consistency and reliability of test results.
+  - **Dataverse Records**: Requires careful management to avoid data pollution and ensure the environment is in a known state before each test.
+
+- **Setup and Configuration of Dataverse Environment**
+  - **Test Environment Setup**: Establishing a dedicated test environment that mirrors your production setup is crucial for realistic integration testing.
+  - **Data Seeding**: Pre-loading necessary test data into the Dataverse environment can save time during test execution.
+
+- **Testing Multiple Personas and Security Roles**
+  - **Security Roles**: Ensure that different security roles are accurately represented in your tests to validate role-based access and permissions.
+  - **User Personas**: Simulate different user personas to test how varying roles and permissions affect the application's behavior.
+
+By carefully considering these factors, you can create a balanced and effective testing strategy that leverages the strengths of both unit tests and integration tests for your model-driven apps.
+
+## Key Features
+
+1. **Unit and Integration Testing**:
+   - The extension supports comprehensive unit and integration testing to ensure that both individual components and the interactions between different modules within Model Driven Applications operate correctly.
+   - Unit testing verifies the proper functioning of individual application elements.
+   - The ability to mock data connections to isolate the behavior of the application
+   - Integration testing ensures that the overall application perform as expected when connected to dependent components
+
+2. **Low Code Extensibility with Power Fx**:
+   - The provider encapsulates the XRM SDK within low code Power Fx commands, enabling users to efficiently interact with the Model Driven Application’s data and logic.
+   - This approach simplifies test scripting, making it accessible to users with limited coding experience.
+   - Power Fx commands allow testers and developers to write and execute test cases without requiring deep expertise in JavaScript.
+   - By working with the JavaScript object model tests are abstracted from changes in the Document Object Model (DOM) used to represent the application
+
+3. **Playwright for Web Automation**:
+   - The extension leverages Playwright to automate testing for web-based Model Driven Applications.
+   - Playwright enables reliable and consistent automation of the application’s user interface, verifying that UI components behave as expected under various conditions.
+   - Provide for multi browser testing
+   - ALlow for recorded videos and headless testing
+
+4. **YAML-Based Test Definition and Execution**:
+   - The extension uses YAML definitions to structure and manage test suites and test cases.
+   - This supports clear and concise test configurations, facilitating easier maintenance and updates.
+   - Tests can be executed automatically, streamlining the testing process within development pipelines.
+
+5. **Generation of TRX Files for CI/CD Integration**:
+   - After test execution, the results are generated as TRX files.
+   - These TRX files can be integrated into CI/CD pipelines, providing feedback on the quality and reliability of the application.
+   - This integration helps maintain high standards and continuous quality assurance throughout the development lifecycle.
+
+6. **Seamless Integration with Power Apps**:
+   - The MEF extension integrates smoothly with the existing Power Apps Test Engine, ensuring a cohesive testing environment.
+   - It supports a consistent testing framework across all stages of application development, from initial development to pre-deployment validation.
+
+7. **Enhanced Developer and Tester Productivity**:
+   - The low code approach with Power Fx commands and the use of Playwright for automation, combined with YAML-based definitions, enhances productivity.
+   - Developers and testers can rapidly create and execute test cases, locate issues, and iterate on solutions efficiently.
+
+## Benefits
+
+- **High-Quality Applications**: Ensures comprehensive testing of web-based Model Driven Applications, leading to reliable and robust deployments.
+- **Efficient Development and Testing**: Reduces the time and effort needed for testing, allowing teams to focus on innovation and development.
+- **Accessibility**: Low code Power Fx commands make testing accessible to a broader range of users, including non-developers.
+- **Consistency**: Provides a standardized testing framework, ensuring predictable and manageable processes.
+- **CI/CD Integration**: Enhances continuous integration and continuous deployment processes by incorporating test results seamlessly, ensuring ongoing quality assurance.
+
+## Roadmap
+
+1. **Generative AI for Test Case Creation**:
+   - Future enhancements could include the ability to use generative AI to automate the process of converting natural language descriptions into test cases and Power Fx test steps.
+   - This will significantly accelerate the creation and maintenance of tests by enabling users to describe testing scenarios in plain language.
+   - The AI will interpret these descriptions and automatically generate the corresponding test cases and steps, reducing the manual effort involved and increasing the accuracy and consistency of test scripts.
+
+2. **Enhanced Natural Language Processing (NLP)**:
+   - Improved NLP capabilities will allow for more sophisticated understanding and interpretation of natural language inputs.
+   - This will enable more complex and nuanced test scenarios to be accurately translated into executable test steps.
+   - This could include synthetic test data generation.
+
+## Examples
+
+[Basic MDA - Assert Form Properties](../../../samples/mda/README.md) and associated [testPlan.fx.yaml](../../../samples/mda/testPlan.fx.yaml)
+
+## Capabilities
+
+The following table outlines the scope of testing Model Driven Applications and current support for features in the provider
+
+| Capability               | Description                                                                                     | Supported |
+|--------------------------|-------------------------------------------------------------------------------------------------|-----------|
+| Command Bars and Actions | Ability to interact with command bars to automate and validate command bar actions and custom buttons.|
+| Forms (Get)              | Ability to read all [controls](./controls.md) as Power Fx variables.                                           | Y
+| Forms (Set)              | Ability to set Power Fx variables which change form data.       |
+| Navigation               | Provides methods for navigating forms and items in Model Driven Apps.                           |
+| Panels                   | Provides a method to display a web page in the side pane of Model Driven Apps forms.            |
+| Views (Get)              | Ability to query grids
+| Views (Actions)          | Ability to select and take actions on view items.                |
+| Security and Roles       | Provides methods to manage and test role-based security within the application.                 |
+| Custom Pages             | The ability to test custom pages |
+| Web API                  | Supports operations for both Online and Offline Web API.                                       |
+| Data Operations          | Allows CRUD (Create, Read, Update, Delete) operations on records within Model Driven Apps.      |
+| Mock Data Operations     | Provide the ability to provide mock values for CRUD (Create, Read, Update, Delete) operations on records within Model Driven Apps.      |
+| Workflow Execution       | Enables triggering workflows and monitoring their progress.                                     |
+| Business Logic           | Ability to trigger and test custom business logic and validations.                            |
+| Notifications            | Capability to display and manage system and user notifications within the application.          |
+| Entities and Attributes  | Powers querying and manipulation of entity attributes and relationships.                        |
+| User Context             | Methods to retrieve and utilize user context information within tests.                          |
+| Global Variables         | Supports the use of global variables for maintaining state and shared data across tests.        |
+| Audit and Logs           | Ability to access and interact with audit logs and system logs for compliance and debugging.    |
+| Solutions                | Allows for importing, exporting, and managing solutions within the application.                 |
+| Localization             | Ability to specify locale for localization of navigation, commands and labels |
+
+## Page Types
+
+Model driven applications provide a range of different page types
+
+| Page Type               | Description                                                                                     | Supported |
+|--------------------------|-------------------------------------------------------------------------------------------------|-----------|
+| Custom | [Overview of custom pages for model-driven apps](https://learn.microsoft.com/power-apps/maker/model-driven-apps/model-app-page-overview)|N
+| Dashboards | [Track your progress with dashboards and charts](https://learn.microsoft.com/power-apps/user/track-your-progress-with-dashboard-and-charts) | N
+| Forms | [Create and design model-driven app forms](https://learn.microsoft.com/power-apps/maker/model-driven-apps/create-design-forms) | Y
+| Views | [Understand model-driven app views](https://learn.microsoft.com/power-apps/maker/model-driven-apps/create-edit-views) | N
diff --git a/docs/Extensions/ModelDrivenApplicationProvider/controls.md b/docs/Extensions/ModelDrivenApplicationProvider/controls.md
@@ -0,0 +1,31 @@
+# Overview
+
+## Common Properties
+
+The following table outlines common properties that could apply and the current state of support for the property in the provider
+
+| Property | Description | Supported |
+|----------|-------------|-----------|
+| Disabled | Get or set whether the control is disabled | Get only |
+| Label    | The label for the control | Get only |
+| Name    | The name for the control | Get only |
+| Value    | The value for the control | Get only |
+| Visible | If the control is visible or not | Get only |
+
+## Grid Controls
+
+Used for displaying and interacting with data in views and subgrids. Includes editable grids.
+
+Items to consider:
+- Get rows, rows and cells
+- Selected rows
+- Total row count
+- Refresh table
+
+## Display Controls
+
+Components like calendars, star ratings, AI Builder business card reader, and Power BI reports.
+
+## Input Controls
+
+Includes checkboxes, number inputs, toggles, and more.
diff --git a/samples/coe-kit-setup-wizard/README.md b/samples/coe-kit-setup-wizard/README.md
@@ -0,0 +1,18 @@
+# Overview
+
+The Power Platform Center of Excellence (CoE) starter kit is made up of a number of Power Platform low code solution elements. Amoung these is a model driven application that can be used to setup and upgrade the CoE Starter Kit.
+
+This sample includes Power Apps Test Engine tests that can be used to automate and test ket elemenets of the expected behaviour of the Setup and Upgrade Wizard
+
+## Getting Started
+
+To get started ensure that you have followed the [Build locally](../../README.md) to have a working version of the Power Apps Test Engine available
+
+## Usage
+
+You can execute this sample using the following commands
+
+```bash
+cd bin/Debug/PowerAppsTestEngine
+dotnet PowerAppsTestEngine.dll -i ../../../samples/coe-kit-setup-wizard/testPlan.fx.yaml -u browser -p mda -d https://contoso.crm.dynamics.com/main.aspx?appid=06f88e88-163e-ef11-840a-0022481fcc8d&pagetype=custom&name=admin_initialsetuppage_d45cf
+```
diff --git a/samples/coe-kit-setup-wizard/testPlan.fx.yaml b/samples/coe-kit-setup-wizard/testPlan.fx.yaml
@@ -0,0 +1,26 @@
+testSuite:
+  testSuiteName: CoE Starter Kit Setup Wizard
+  testSuiteDescription: Verify custom page of CoE Starter Kit Setup Wizard and step through install
+  persona: User1
+  appLogicalName: NotNeeded
+
+  testCases:
+    - testCaseName: Verify
+      testCaseDescription: Verify setup and upgrade Wizard of the CoE Starter Kit
+      testSteps: |
+        = TestEngine.ConsentDialog(Table({Text: "Center of Excellence Setup Wizard"}));
+
+testSettings:
+  headless: false
+  locale: "en-US"
+  recordVideo: true
+  extensionModules:
+    enable: true
+  browserConfigurations:
+    - browser: Chromium
+
+environmentVariables:
+  users:
+    - personaName: User1
+      emailKey: NotNeeded
+      passwordKey: NotNeeded
diff --git a/samples/mda/MDASample_1_0_0_1.zip b/samples/mda/MDASample_1_0_0_1.zip
diff --git a/samples/mda/README.md b/samples/mda/README.md
@@ -0,0 +1,29 @@
+# Overview
+
+This Power Apps Test Engine sample demonstrates how to assert the values of controls in a model driven application form.
+
+## Usage
+
+1. Build the Test Engine solution
+
+2. Get the URL of model driven application form
+
+3. Modify the [testPlan.fx.yaml](./testPlan.fx.yaml) to assert expected values of the form controls.
+
+  > [!NOTE] The controls are referenced using the [logical name](https://learn.microsoft.com/power-apps/developer/data-platform/entity-metadata#table-names).
+
+4. Update the domain url for your model driven application
+
+| Url Part | Description |
+|----------|-------------|
+| appid=a1234567-cccc-44444-9999-a123456789123 | The unique identifier of your model driven application |
+| etn=test | The name of the entity being validated |
+| id=26bafa27-ca7d-ee11-8179-0022482a91f4 | The unique identifier the record being edited |
+| pagetype=entityrecord | The type of page to open.
+
+5. Execute the test for custom page changing the example below to the url of your organization
+
+```pwsh
+cd bin\Debug\PowerAppsEngine
+dotnet PowerAppsTestEngine.dll -i ..\..\..\samples\mda\testPlan.fx.yaml -e 00000000-0000-0000-0000-11112223333 -t 11112222-3333-4444-5555-666677778888 -u browser -p mda -d "https://contoso.crm4.dynamics.com/main.aspx?appid=9e9c25f3-1851-ef11-bfe2-6045bd8f802c&pagetype=custom&name=sample_custom_cf8e6"
+```
diff --git a/samples/mda/testPlan.fx.yaml b/samples/mda/testPlan.fx.yaml
@@ -0,0 +1,26 @@
+testSuite:
+  testSuiteName: MDA Custom Page tests
+  testSuiteDescription: Verify model driven application
+  persona: User1
+  appLogicalName: NotNeeded
+
+  testCases:
+    - testCaseName: Open Page
+      testCaseDescription: Check initial state
+      testSteps: |
+        = Assert(Total.Text, "0");
+        
+testSettings:
+  headless: false
+  locale: "en-US"
+  recordVideo: true
+  extensionModules:
+    enable: true
+  browserConfigurations:
+    - browser: Chromium
+
+environmentVariables:
+  users:
+    - personaName: User1
+      emailKey: NotNeeded
+      passwordKey: NotNeeded
diff --git a/src/Microsoft.PowerApps.TestEngine.Tests/TestInfra/PlaywrightTestInfraFunctionTests.cs b/src/Microsoft.PowerApps.TestEngine.Tests/TestInfra/PlaywrightTestInfraFunctionTests.cs
@@ -767,5 +767,30 @@ public async Task RouteNetworkRequestTest()
             await playwrightTestInfraFunctions.RouteNetworkRequest(MockRoute.Object, mock);
             MockRoute.Verify(x => x.ContinueAsync(It.IsAny<RouteContinueOptions>()), Times.Once);
         }
+
+        [Fact]
+        public async Task LoadScriptContent()
+        {
+            // Arrange
+            var playwrightTestInfraFunctions = new PlaywrightTestInfraFunctions(MockTestState.Object, MockSingleTestInstanceState.Object,
+                MockFileSystem.Object, browserContext: MockBrowserContext.Object);
+
+            playwrightTestInfraFunctions.Page = MockPage.Object;
+
+            PageAddScriptTagOptions tagOptions = null;
+
+            MockPage.Setup(m => m.AddScriptTagAsync(It.IsAny<PageAddScriptTagOptions>()))
+                .Callback((PageAddScriptTagOptions options) => tagOptions = options)
+                .Returns(Task.FromResult(MockElementHandle.Object));
+
+            var javaScript = "var test = 1";
+
+            // Act
+
+            await playwrightTestInfraFunctions.AddScriptContentAsync(javaScript);
+
+            // Assert
+            Assert.Equal(javaScript, tagOptions.Content);
+        }
     }
 }
diff --git a/src/Microsoft.PowerApps.TestEngine/PowerFx/PowerFxEngine.cs b/src/Microsoft.PowerApps.TestEngine/PowerFx/PowerFxEngine.cs
@@ -68,7 +68,7 @@ public void Setup()
             {
                 if (TestState.GetTestEngineModules().Count == 0)
                 {
-                    Logger.LogError("Extension enabled not none loaded");
+                    Logger.LogError("Extension enabled, none loaded");
                 }
                 foreach (var module in TestState.GetTestEngineModules())
                 {

diff --git a/src/Microsoft.PowerApps.TestEngine/Providers/PowerFxModel/ControlRecordValue.cs b/src/Microsoft.PowerApps.TestEngine/Providers/PowerFxModel/ControlRecordValue.cs
@@ -110,6 +110,12 @@ protected override bool TryGetField(FormulaType fieldType, string fieldName, out
 
                 if (jsPropertyValueModel != null)
                 {
+                    if (string.IsNullOrEmpty(jsPropertyValueModel.PropertyValue))
+                    {
+                        result = null;
+                        return false;
+                    }
+
                     if (fieldType is NumberType)
                     {
                         result = NumberValue.New(double.Parse(jsPropertyValueModel.PropertyValue));

diff --git a/src/Microsoft.PowerApps.TestEngine/TestInfra/ITestInfraFunctions.cs b/src/Microsoft.PowerApps.TestEngine/TestInfra/ITestInfraFunctions.cs
@@ -13,6 +13,11 @@ namespace Microsoft.PowerApps.TestEngine.TestInfra
     /// </summary>
     public interface ITestInfraFunctions
     {
+        /// <summary>
+        /// The current page to execute actions
+        /// </summary>
+        public IPage Page { get; set; }
+
         /// <summary>
         /// Return the current browser context
         /// </summary>
@@ -80,6 +85,13 @@ public interface ITestInfraFunctions
         /// <returns>Task</returns>
         public Task AddScriptTagAsync(string scriptTag, string frameName);
 
+        /// <summary>
+        /// Adds a script content to page
+        /// </summary>
+        /// <param name="content">The script to add</param>
+        /// <returns>Task</returns>
+        public Task AddScriptContentAsync(string content);
+
         /// <summary>
         /// Runs javascript on the page
         /// </summary>

diff --git a/src/Microsoft.PowerApps.TestEngine/TestInfra/PlaywrightTestInfraFunctions.cs b/src/Microsoft.PowerApps.TestEngine/TestInfra/PlaywrightTestInfraFunctions.cs
@@ -2,6 +2,7 @@
 // Licensed under the MIT license.
 
 using System.Diagnostics.CodeAnalysis;
+using System.Linq.Expressions;
 using System.Runtime;
 using Microsoft.Extensions.Logging;
 using Microsoft.Playwright;
@@ -26,7 +27,7 @@ public class PlaywrightTestInfraFunctions : ITestInfraFunctions
         private IPlaywright PlaywrightObject { get; set; }
         private IBrowser Browser { get; set; }
         private IBrowserContext BrowserContext { get; set; }
-        private IPage Page { get; set; }
+        public IPage Page { get; set; }
         public PlaywrightTestInfraFunctions(ITestState testState, ISingleTestInstanceState singleTestInstanceState, IFileSystem fileSystem, ITestWebProvider testWebProvider)
         {
             _testState = testState;
@@ -372,5 +373,12 @@ public async Task<T> RunJavascriptAsync<T>(string jsExpression)
 
             return await Page.EvaluateAsync<T>(jsExpression);
         }
+
+        public async Task AddScriptContentAsync(string content)
+        {
+            ValidatePage();
+
+            await Page.AddScriptTagAsync(new PageAddScriptTagOptions { Content = content });
+        }
     }
 }