CASSGO-4 Support of sending queries to the specific node

[worryg0d]

Overview

This PR provides a mechanism that allows users to specify on which node the query will be executed. It is not a typical use case, but it makes sense with virtual tables.
For example, when we want to retrieve metrics for a specific node, we have to send queries to the associated system view of this node.

Implementation Overview

A new method SetHost() for the Query allows users to specify the host (node) on which the driver must send the query. It is implemented by adding a GetHost() to the ExecutableQuery interface. When the host is specified for the query the driver ignores any retry, speculative, and host selection policies.

Usage Example

Here is an example of retrieving metrics from system_view.cql_metrics for each node of the cluster:

cluster := NewCluster("127.0.0.1")
session, err := cluster.CreateSession()
if err != nil {
	panic(err)
}
defer session.Close()

hosts, err := session.GetHosts()
if err != nil {
	panic(err)
}

type cqlMetric struct {
	name  string
	value float64
}

// hostId to []cqlMetric
cqlMetrics := map[string][]cqlMetric{}

for _, host := range hosts {
	iter := session.Query("SELECT * FROM system_views.cql_metrics").
		SetHost(host).
		Iter()

	metrics := make([]cqlMetric, 0, iter.NumRows())
	metric := cqlMetric{}
	for iter.Scan(&metric.name, &metric.value) {
		metrics = append(metrics, metric)
	}
	cqlMetrics[host.HostID()] = metrics
}

[martin-sucha]

Should the API allow for other routing preferences? For example only a single data center/rack? Or be extensible to allow targeting a specific shard in the scylladb/gocql fork?

Should the API allow to route to a different host during retry, i.e. allow passing a HostSelectionPolicy or a similar interface/function?

[worryg0d]

Should the API allow for other routing preferences? For example only a single data center/rack? Or be extensible to allow targeting a specific shard in the scylladb/gocql fork?

Should the API allow to route to a different host during retry, i.e. allow passing a HostSelectionPolicy or a similar interface/function?

HostInfo provides API to retrieve information like dc or rack for the host, so users may filter retrieved hosts manually before binding them to the query.

In my opinion, re-routing queries bound to the specific hosts to others is not a good idea even during retries, at least as default behavior. Sending queries to a specific host is not a typical use case and if we try to do it, we assume that the specified host is reachable. Or, at least, we expect that the query was attempted to be sent to the specified host.

The idea of this API being extensible sounds good, but I am not sure about implementation. May you have some advice or ideas on how to properly implement this?

[joao-r-reis]

I think keeping this focused on the virtual table use case is best. Other routing preferences like shard awareness for scylla should probably be handled at the HostSelectionPolicy level. For this, we could add a function on Query and Batch that allows the HostSelectionPolicy to be provided instead of using the session level one but this can be handled on a separate JIRA and PR, it's a different use case than this one which is focused on virtual table queries that should be strongly associated with a specific host.

If a user sets the host using Query.SetHost() then either we disable retries and speculative executions altogether for that query or we allow retries and speculative executions only on that host (basically the host selection policy should never be used in the context of this query).

In the future if we add a way to provide a HostSelectionPolicy in addition to .SetHost() then the driver should probably return an error saying that you can't simultaneously set a host and provide a policy on the same query.

[joao-r-reis]

When the queryExecutor.executeQuery is called then it checks if the provided ExecutableQuery implements the hostGetter interface as well. We need this interface to not change the API ExecutableQuery. Also, the ExecutableQuery is implemented by Query and Batch as well, so it is good to avoid adding redundant methods for Batch as well.

So, if provided ExecutableQuery implements hostGetter, then it type-asserts it and calls the underlying method to get the host. If the host is not nil, it wraps the host into hostIter func which just returns the specified host.

The discussion around the ExecutableQuery interface and the Query API in general probably has a decent overlap with the work being dicussed and implemented in https://issues.apache.org/jira/browse/CASSGO-22 , justfyi.

[joao-r-reis]

How about adding a private interface like:

type cqlQuery interface {
    ExecutableQuery
    getHost() *HostInfo
}

And make gocql internal code reference cqlQuery (name subject to change?) instead of ExecutableQuery so that we don't have to worry about breaking the public ExecutableQuery interface when we add new functions to Query or Batch?

[joao-r-reis]

Alternatively, we could just add new functions to ExecutableQuery because there's no place where a user can provide a custom implementation of ExecutableQuery to the driver is there? If the driver API doesn't expose any function that receives an object of type ExecutableQuery then we should be fine changing this interface since there's no real use case where a user might implement it on their application.

[joao-r-reis]

From a quick glance at the code, ExecutableQuery is provided to implementations of HostSelectionPolicy but it's the driver that calls HostSelectionPolicy.Pick(ExecutableQuery) with Query or Batch objects as parameters so I don't see an issue in adding GetHost to ExecutableQuery

[worryg0d]

I don't mind extending ExecutableQuery interface, but adding private methods is not the best thing to me. The driver exposes HostSelectionPolicy interface and allows user to provide their own implementations, but users cannot properly test them because Pick() accepts ExecutableQuery that cannot be mocked for testing purposes due to private methods inside... Ofc they can just pass Query and Batch objects, but I'm unsure if it is enough for testing

[joao-r-reis]

These private methods would not be used in HostSelectionPolicy because HostSelectionPolicy declares the parameter as ExecutableQuery not cqlQuery or whatever the name would be for the private interface.

[worryg0d]

what private methods are you referencing?

Such methods as execute(), attempt(), etc.

[joao-r-reis]

Oh you're right, I didn't notice them. 👍 So it wouldn't actually make much of a difference if we added a private getHosts method there but we can keep it public

[worryg0d]

The only issue of making GetHost() public is a requirement to implement it for Batch only to satisfy the interface. So there will be a public API Batch.GetHost() which actually does noop. Maybe the first approach with internal query interface for internal use and the public one for exposing is better...

[joao-r-reis]

I think it's fine if Batch.GetHost() does a noop honestly, there's also a possibility that we might make it possible to set the host on batch queries in the future

[worryg0d]

Then let's leave this one as it is for now

[joao-r-reis]

we should probably add another if here to exit right away if specified host is != nil and the host is down

[worryg0d]

It makes sense not to send the query if the host is down. I added a check before this if that immediately returns an ErrNoConnections when the specified host is down.

[joao-r-reis]

👍

[worryg0d]

Force pushed due to conflicts with the trunk