Unquoted strings in unexpected_index_query values of gx.ValidationDefinition.run() result

[vasilicommify]

Describe the bug
"COMPLETE" result_format outputs unexpected_index_query
The value of the unexpected_index_query key cannot be used to extract rows with issue, even the part inside expr( )
One of the reasons is that string values are not quoted

1. • there are strings in value_set, but in unexpected_index_query these strings are not quoted

    {
      "success": true,
      "expectation_config": {
        "type": "expect_column_values_to_be_in_set",
        "kwargs": {
          "batch_id": "my_spark_datasource-my_spark_dataframe",
          "column": "in_set_str",
          "value_set": [
            "Val1",
            "Val2"
          ]
        },
        "meta": {
          "note": "column_values - without distinct"
        }
      },
      "result": {
        "element_count": 100,
        "unexpected_count": 0,
        "unexpected_percent": 0.0,
        "partial_unexpected_list": [],
        "missing_count": 11,
        "missing_percent": 11.0,
        "unexpected_percent_total": 0.0,
        "unexpected_percent_nonmissing": 0.0,
        "partial_unexpected_counts": [],
        "unexpected_list": [],
        "unexpected_index_query": "df.filter(F.expr((in_set_str IS NOT NULL) AND (NOT (in_set_str IN (Val1, Val2)))))"
      },
      "meta": {},
      "exception_info": {
        "raised_exception": false,
        "exception_traceback": null,
        "exception_message": null
      }
    },

2. regex expression is not quoted

"unexpected_index_query": "df.filter(F.expr((email IS NOT NULL) AND (NOT RLIKE(email, ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$))))"

To Reproduce

# expectation_suite = any expectation suite with with string fields validations

validation_definition = gx.ValidationDefinition(
    data=batch_definition, suite=expectation_suite, name=validation_definition_name
  )

validation_results = validation_definition.run(
                                      batch_parameters={"dataframe": data_frame_to_check}, 
                                      result_format = "COMPLETE")            
validation_results 

Expected behavior
string and regex expressions should be quoted in unexpected_index_query - either with single or double quates for spark

Environment (please complete the following information):

• Great Expectations Version: 1.3.0
• Data Source: Spark
• Cloud environment: Databricks

[kujaska]

+100500

[0lgaZv]

Please

[adeola-ak]

thank you for reaching out about this, I could see how this would be helpful. I will share this with my team. Please be sure to check on this issue for updates

[umanggarg754]

Yes please correct the syntax of the unexpected index query for spark