Proposal for accessing Environment Variables in Power Fx

[MikeStall]

Here is a proposal for #2416 - for accessing Dataverse Environment Variables directly from Power Fx expressions in a strongly-typed, discoverable way.

This is primarily intended to support Dataverse Environment Variables, as defined here: https://learn.microsoft.com/en-us/power-apps/maker/data-platform/environmentvariables . But since it goes through host abstractions, it should be usable in other ways.

The intention is to provide a common language experience across Power Platform products - and so absorb as much policy as possible in the Fx layer rather than the host. For example, Fx should deal with name disambiguation so that it's done consistently across products.

Requirements

The host:

1. enables env variables - these are are not accessible by default.
2. provides the list of names (logical/display) and a type for each.
3. provides a means to fetch the value at runtime.

Existing Art

Today, the Env Vars live in a Dataverse table and can be accessed directly via a table lookup:

LookUp('Environment Variable Values', 'Environment Variable Definition'.'Schema Name' = "test_environment_data").Value

• Example of this in Power Apps: https://powerusers.microsoft.com/t5/Building-Power-Apps/How-to-read-Environment-Variables-in-PowerApp/td-p/578323
• Example of this in Power Automate Desktop: https://www.youtube.com/watch?app=desktop&v=BJs4S9thzqw&feature=youtu.be

And the proposal is to write:
Environment.Variables.TestData where it's strongly typed to number,date,bool,etc.

So we're improving on this experience with simpler access, stronger typing, better discoverability, and display names. Also least-privilege - the maker no longer requires full access to query and enumerate the variable table at runtime.

In Copilot Studio, you can access env variables via Env.logicalname. This provides a similar experience to this proposal except that it a) uses Env instead of Environment, b) uses logical names instead of display names.

Language level experience:

When enabled, makers can access an object Environment.Variables, which is a record where each field is a strongly typed environment variable.

For example, suppose the host has defined a number variable called MaxCount. An expression could access the value as Environment.Variables.MaxCount and use it like a normal number variable in expressions like If(Environment.Variables.MaxCount > 50, "too high")

A key win here is good intellisense and authoring: the maker should get completion after Environment.Variables. and see the list of possible variables.

This is also a regular immutable record, with similar properties to the User object.
Like User, it can be aliases in With, like With({vars : Environment.Variables}, vars.MaxCount)

The values are "current" and a runtime fetch. The host does not need to eagerly fetch and cache. This means that the values should be considered volatile and could change through the lifetime of an expression (like Now()).

Display Names

The Power Fx type system supports display names and this feature can follow existing semantics:

• If the display names conflict, then Power Fx has a standard disambiguation technique.
• It is also ok to access by logical name.
• The host should never persist display names. Convert to invariant to serialize. Use helpers on Engine.cs or CheckResult.cs

Runtime errors

Since accessing a variable is ultimately fetching from an external source, it's always possible for that to fail at runtime. While the strong typing approach here should help makers avoid common errors (typos in the var name), there are inevitable corner cases, such as if a variable was deleted at runtime (after passing design-time checks) or just a plain network failure.

Errors will return an ErrorValue, which will naturally propagate through the expression via data flow semantics unless caught by an IfError.

For example, suppose accessing MaxCount failed at runtime, then:

expr	result
Environment.Variables.MaxCount	error value
Environment.Variables.MaxCount > 50	error value, propagated
IsError(Environment.Variables.MaxCount)	true

Compile-time errors:

1. Attempting to access Environment.Variables if the host has not enabled environment variables.
2. Attempting to mutate this record. This is immutable, similar to the User object.
3. Type mismatches on field access. The env vars are strongly typed.

Dataverse details.

This feature is general purpose, but should be sufficient to fully handle Dataverse env vars.

Dataverse env vars have display and logical names.
They are strongly typed. Most of the types have an obvious mapping to Power Fx (decimal, text, two option --> boolean). Json is mapped as Untyped Object (similar to ParseJson).

Hosting APIs.

Key criteria for hosting:

1. The hosting pattern here is similar to the User object, as described in Enabling User object #2318. The vars are defined statically on the symbol table; and then runtime values are provided separately.
2. The host fully controls enumerating and fetching the vars. Power Fx does not take any direct dependency on dataverse at either design time or at runtime.
3. Fetching values can be deferred until needed and thus should be a network call. The host does not need to pre-fetch.

In this pattern, Environment.Variables is literally just a RecordValue. That already has all the essential patterns we need:

• split between design time (RecordType) and runtime (RecordValue).
• existing virtuals to override.
• support for display/logic names

So the APIs can be:
At design time, add to the symbol table:
var slot = symbolTable.AddEnvVariable(recordType); // adds with "Environment.Variable" name

And at runtime, pass the corresponding values:
symbolValues.Set(slot, recordValue);

Refresh

Runtime fetch calls out to the host to get the latest value.
But suppose the host needs to refresh the list of variables during design time.
It can just refresh the symbols using existing symbolTable APIs. For example, it could:

• create a new symbol table or
• it could call RemoveVariable() and re add with the new definitions.

Dataverse Helpers

We can have helpers to aide in creating the RecordType/RecordValue from a dataverse IOrganizationService

IOrganizationService service;
RecordValue envValues = EnvVariablesToRecordValue(service); // name TBD
RecordType recordType = envValues.Type;

RecordType would be a static type (since the variable list is fetched eagerly), but the RecordValue here would be a derived class that override TryGetField, etc and invoke on dataverse.

[matthewdevaney]

@MikeStall
Would environment variables become cached for use in offline scenarios? As far as I know the “Environment” object cannot be accessed unless an app is connected. I would use environment variables in canvas mobile app scenarios if were possible.

[MikeStall]

That's a good point. One concern about caching them is that we'd need to eagerly fetch them.
We might be able to mitigate that with some analysis to see which ones are actually used and feed that back to the host.

[filcole]

For cloud flows, when an environment variable value changes that's used by the flow, the flow itself is updated.

Would you consider holding the current value of a environment variable within a canvas app, FxExpression or any other component that's could be used offline (for use in offline access)?

[cchannon]

I agree that intellisense could be very nice, especially in environments where there are a large number of EVs such as one that includes the CoE. But we can already get to EVs by just adding environmentvariabledefinition and environmentvariablevalue to our app as datasources. Yes, that doesn't come with intellisense but that seems like a small marginal benefit when you consider that some very large percentage of apps out there do not use EVs: seems odd to require that they eagerly load all definitions--or eagerly fetch values as well--if they aren't using any.

[cchannon]

OK, I agree there is probably more utility in use cases outside Dataverse EVs...

So what we're talking about there is effectively is a very Javascript-esque collection of Key-Value pairs with Value of Any type. But there too we already have an established way of handling such an object in Untyped objects as in the ParseJSON output.

Just playing Devil's Advocate on this one... I think in reality I would be quite annoyed if I had to IsType/AsType coerce every 'variable' in my app to its correct datatype every time.

[MikeStall]

Right - the key here is we want both a) easy discovery of env vars at authoring time, b) strong typing.

[cchannon]

...In the case of Power Automate Desktop, though, the 'Variables' are all Flow-specific, not Environmental (unless again we're talking about Dataverse EVs). Doesn't it seem a tad odd to go to the Environment object? Shouldn't those Variables just be Variables?

[MikeStall]

"we're talking about Dataverse EVs" Yes :)

[MikeStall]

These are all great points - I updated the original post to capture them.

[ndcatalyst]

It would be very interesting if the intellisense would pick up the solution the app is in to help the user find the right value.

[brasov2de]

Oh yes please! But maybe it's not only an PowerFx issue, since we need the same features for PlugIns (and even better, also for form scripting api).
Right now there is no way to cache an EnvVar value for PlugIns.
For JavaScript/TS we can build our own caching, but would be nice to have some typed objects instead of implementing the requests each time,
Maybe we can solve this problems on Dataverse side, and reuse them in PlugIns, ClientScript and PowerFx.