Using the Text property in a TermSuggester or PhraseSuggester results in a parsing exception

[Lukyb14]

Elastic.Clients.Elasticsearch version: 8.15.2

Elasticsearch version: 8.15.0

.NET runtime version: .NET 8

Operating system version: Windows 10

Description of the problem including expected versus actual behavior:
When using the SearchRequest with a FieldSuggester, where the Text property is set, the ElasticsearchClient sends a request with the text field not set in the correct place. The server then responds with a parsing error.

This is an example of an invalid generated suggester:

"suggest": {
        "0": {
            "term": {
                "analyzer": "term_search_analyzer",
                "field": "term",
                "prefix_length": 2,
                "size": 10,
                "sort": "score",
                "text": "elastic"
            }
        }
    }

The "term" object does not support the field "text", so therefore the server returns with a parsing exception.

Steps to reproduce:

1. Create a new SearchRequest object.
2. In the SearchRequest, set the Suggest property with a new Suggester containing a Suggester dictionary.
3. Inside the dictionary, add a TermSuggester, with the "Text" property set to any string.
4. Send the request with the Search API

Expected behavior

The client should generate a valid request, where the "text" property is next to the "term" property.

"suggest": {
        "0": {
            "text": "elastic",
            "term": {
                "analyzer": "term_search_analyzer",
                "field": "term",
                "prefix_length": 2,
                "size": 10,
                "sort": "score"
            }
        }
    }

Provide DebugInformation (if relevant):

# FailureReason: BadResponse while attempting POST on http://127.0.0.1:9200/wiviosug/_search?typed_keys=true
# Audit trail of this API call:
 - [1] BadResponse: Node: http://127.0.0.1:9200/ Took: 00:00:00.2604276
# OriginalException: Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: POST /wiviosug/_search?typed_keys=true. ServerError: Type: parsing_exception Reason: "suggester[term] doesn't support field [text]"
   at Elastic.Transport.DistributedTransport`1.HandleTransportException(RequestData data, Exception clientException, TransportResponse response)
   at Elastic.Transport.DistributedTransport`1.FinalizeResponse[TResponse](RequestData requestData, RequestPipeline pipeline, List`1 seenExceptions, TResponse response)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
   at Elastic.Clients.Elasticsearch.ElasticsearchClient.<>c__DisplayClass28_0`3.<<DoRequestCoreAsync>g__SendRequest|0>d.MoveNext() in /_/src/Elastic.Clients.Elasticsearch/Client/ElasticsearchClient.cs:line 158
--- End of stack trace from previous location ---
# Request:
{"query":{"bool":{"must":{"match":{"term.completion":{"query":"test"}}},"should":{"rank_feature":{"field":"doc_frequency"}}}},"size":10,"_source":{"includes":["term"]},"suggest":{"0":{"term":{"analyzer":"term_search_analyzer","field":"term","prefix_length":2,"size":10,"sort":"score","text":"elastic"}},"1":{"term":{"analyzer":"term_search_analyzer","field":"term","prefix_length":2,"size":10,"sort":"score","text":"search"}},"2":{"term":{"analyzer":"term_search_analyzer","field":"term","prefix_length":2,"size":10,"sort":"score","text":"test"}}}}
# Response:
{"error":{"root_cause":[{"type":"parsing_exception","reason":"suggester[term] doesn't support field [text]","line":1,"col":292}],"type":"parsing_exception","reason":"suggester[term] doesn't support field [text]","line":1,"col":292},"status":400}
# Exception:
Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: POST /wiviosug/_search?typed_keys=true. ServerError: Type: parsing_exception Reason: "suggester[term] doesn't support field [text]"
   at Elastic.Transport.DistributedTransport`1.HandleTransportException(RequestData data, Exception clientException, TransportResponse response)
   at Elastic.Transport.DistributedTransport`1.FinalizeResponse[TResponse](RequestData requestData, RequestPipeline pipeline, List`1 seenExceptions, TResponse response)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
   at Elastic.Clients.Elasticsearch.ElasticsearchClient.<>c__DisplayClass28_0`3.<<DoRequestCoreAsync>g__SendRequest|0>d.MoveNext() in /_/src/Elastic.Clients.Elasticsearch/Client/ElasticsearchClient.cs:line 158
--- End of stack trace from previous location ---

[flobernd]

Hi @Lukyb14,

I can not reproduce this issue. I assume the way you are building the query is incorrect. The following code:

var termSuggester = FieldSuggester.Term(new TermSuggester
{
    Analyzer = "term_search_analyzer",
    Field = "term"!,
    PrefixLength = 2,
    Size = 10,
    Sort = SuggestSort.Score
});
termSuggester.Text = "elastic";

var response = await client.SearchAsync<Person>(new SearchRequest("person")
{
    Query = Query.MatchAll(new MatchAllQuery()),
    Suggest = new Suggester
    {
        Text = "elastic",
        Suggesters = new Dictionary<string, FieldSuggester>
        {
            { "0", termSuggester }
        }
    }
});

as well as the equivalent fluent-style code:

var response = await client.SearchAsync<Person>(r => r
    .Index("person")
    .Query(q => q.MatchAll(_ => {}))
    .Suggest(suggest => suggest
        .Suggesters(suggesters => suggesters
            .Add("0", suggester => suggester
                .Term(term => term
                    .Analyzer("term_search_analyzer")
                    .Field("term"!)
                    .PrefixLength(2)
                    .Size(10)
                    .Sort(SuggestSort.Score)
                )
                .Text("elastic")
            )
        )
    )
);

correctly produces the expected JSON request:

{
  "query": {
    "match_all": {}
  },
  "suggest": {
    "0": {
      "text": "elastic",
      "term": {
        "analyzer": "term_search_analyzer",
        "field": "term",
        "prefix_length": 2,
        "size": 10,
        "sort": "score"
      }
    }
  }
}

You were probably using the Text property that is part of the TermSuggester variant. We will check, if this field should be removed. There is a chance that this is incorrectly modelled in our specification.

[Lukyb14]

You were probably using the Text property that is part of the TermSuggester variant. We will check, if this field should be removed. There is a chance that this is incorrectly modelled in our specification.

That is correct, I was using this property. I was not aware the FieldSuggester type has a Text property as well, and just assumed it would use the TermSuggester Text property. I just wanted to add that I noticed that the PhraseSuggester also has a Text property, which creates a similar invalid request, if it is set.

Thank you for providing the correct examples.

[flobernd]

@Lukyb14 We double checked and it seems like the text property was indeed incorrectly added to these two suggester variants. I'll make sure to remove them from our specification 🙂 Thanks for reporting!