Implement a "between" query operator

[nursix]

Range queries currently implement a combination of ge+le filter expressions (greater or equal + less or equal), which works well for single-range queries. However, it doesn't support multi-range filtering with the traditional URL query syntax.

In the extended query syntax, it would be possible to specify multiple ranges, but it would be much harder to detect and parse filter defaults.

A possible middle ground could be to implement a between-operator that allows multiple ranges:

~.value__between=1|4,8|10,16|24

The corresponding S3ResourceQuery would accept a tuple (min, max) or a list of such tuples, and translate into a number of OR-ed subqueries accordingly.

This is especially useful with filter widgets with pre-defined ranges (e.g. age groups filter for date of birth field), so that they can support multi-select. It will also simplify the implementation of any kind of range filters (and shorten the expression and thus reduce the amount of data that needs to go over the network).

[AlexanderLaughlin]

Is this still an active bug?

[flavour]

This feature request is still active

[AlexanderLaughlin]

I'm going to try to implement this feature

[AlexanderLaughlin]

What is the scale of this project? I'm attempting this as a project for a University class that ends this May. Would all the code for this functionality go within s3query.py and s3resource.py? I'm struggling to understand the underlying architecture. Any advice would be appreciated, thank you.

[nursix]

Generally yes, the primary implementation would involve primarily s3query.py and s3resource.py, plus some unit tests to verify it.

However, I think it should start with a use-case as described above, i.e. a range filter widget that allows the selection of multiple ranges (multi-select drop-down), e.g. extending the existing S3AgeFilter with a multiple-option.

Whether this is an appropriate project for the context you mention, is beyond me. I would say yes, and maybe I can mentor you after 24 Feb - but ultimately it is up to your judgment.

[nursix]

To understand the architecture, you should think of an S3Resource as a database view - i.e. basically a set of related entities plus one or more queries.

A query is represented by an S3ResourceQuery instance, which eventually self-translates into a PyDAL Query object plus all necessary joins.

A resource holds an array of such resource queries (a.k.a. "filters") in the self.rfilter property (a S3ResourceFilter instance).

The query is constructed from the URL query part of a RESTful HTTP request against the resource. This happens during the S3Request construction: S3Request instantiates S3Resource, which instantiates S3ResourceFilter. This process uses the helper classes in s3query.py to process the URL query and construct the S3ResourceQuery array.

So, this is the short version - the long version may take a couple of days ;)

[nursix]

...and to include the use-case part:

A filter widget is a Python class (in s3filter.py) that constructs an HTML widget for the user to enter filter criteria in a multi-record perspective (tables, lists, maps etc). This HTML widget is part of the filter form, which is in turn part of the multi-record view.

The HTML widget is processed on the client side by a JavaScript (s3.filter.js) that reads the input (i.e. the filter criteria the user chooses/enters), and translates it into a URL query, to then Ajax-reload the table/list/map contents with that URL query.

A range filter widget allows the selection of a range (e.g. from-to dates), typically providing two inputs - one for min, and one for max. There is though another variant, where the filter widget would provide a drop-down of pre-defined ranges, e.g. S3AgeFilter - which translates age ranges into a min/max date-of-birth query. And it is this variant of range filtering where it would be highly convenient if multiple (of those pre-defined age) ranges could be chosen.

[AlexanderLaughlin]

Unfortunately I believe implementing this functionality is beyond my ability and the scope of this project. Are there any simpler issues/bugs/tasks that you would recommend working on? Thank you.