Purchases: specify package, data range

[oorlov]

Current Purchases
Suggest following:

Mandatory:

• packageName
• dateFrom
• dateTo

Optional:

• country
• bulk size?

[jesper-sjovall]

For select data-range I suggest to use same format as discussed here.
#3 (comment)

I think the 'country-code' also is a good selector for limit the result which also can be use for the Review request also.

[oorlov]

Updated version:

https://www.distributorappstore.com/openaep/purchases
?package=com.softspb.flashcards.sv - single parameter
[&datefrom=YYYY-MM-DD] - optional
[&dateto=YYYY-MM-DD] - optional
[&country=US] - optional
[&limit=100] - optional
[&offset=7j8ad9go] - optional

Do paginate output there should be offset param used together with limit. And we need to return it in output like this

<?xml version="1.0" encoding="UTF-8"?>
<purchases version="1" offset="7j8ad9go"> <!-- encoded value of request params, db state and offset -->
  <purchase>
...

Offset is critical here, nobody wants to lose financial data

[jesper-sjovall]

Yes offset is critical as you say, but I can see a problem here.
How to known what next offset is to be use or misunderstand I you?
For example: We has get the purchase 1 to 10 from www.distributorappstore.com, how to get purchase 11 to 20,

And in this case would it not be good to also include total number of purchase if "limit" or "offset" not was set
to known how many pages this exists.

  <?xml version="1.0" encoding="UTF-8"?>
  <purchases version="1" offset="7j8ad9go" totalCount="123"> <!-- encoded value of request params, db state and offset -->
     <purchase>
  ...

[oorlov]

First time you specify only 'limit' param. Server returns non-empty offset if there is more purchases remained. If there is nothing to request next time - empty offset returns.

We have 27 purchases,
Requests 10, gets purchases 0..9 and non-empty offset
Requests 10 more, gets 10..19 and non-empty offset
Requests 10 more, gets 20..26 and empty offset

If you decided to request data by 10 items bulks will you change it's size if get totalCount value?
Another thought is: if client specify complex request and Purchases table is huge, doesn't it hurt performance to calculate totalCount for every request?

[jesper-sjovall]

Ok, 'offset' will point on the next page to fetch, it works but I think this is not the correct way to work, the main problem here is this only allow us to fetch data in one order, you can't from the current response fetch previous page of data.

The only reason I can see to do as your propose is if the data not has a natural indexing, but I think all data will has a natural indexing. (as my proposal to always sort the result in chronological order base on date and time for the Purchases).

[oorlov]

Could you help me to understand use case for fetching data in opposite order? I assumed batches is needed to retrieve big amount of data like purchases for a day or a week. Probably it doesn't work well for pulling data multiple times in a day

[jesper-sjovall]

True, this is probably a very uncommon case to read in opposite order.
If we use as your suggests I think this is possible to pre create data before it is need if this is wanted on backend as a optimization.
In this case I think we want to use limit xor offset:

Example:

https://www.distributorappstore.com/openaep/purchases?...&limit=10

Give result 0 to 9 and offset "7j8ad9go"

https://www.distributorappstore.com/openaep/purchases?...&offset=7j8ad9go

Give result 10 to 19 and offset "8iwadhj0"

A ide I have is how to optimize the result and for who, See the new issues #5

[morozstepan]

Hi!
Actually why are we using this "offset" in some magically-encoded form? I see problems in this approach.
For example, I have 15 purchases of some product, all in one second.
Client comes and asks about first 10 purchases. What should I encode in offset to give other portion of data for next request? I would have to save some information like "take all purchases from time and skip first 10". Ok, I'll do this, and what will happen if at this same second I have 2 more purchases arrived into database? I'll have 17 purchases, skip 10 and give 11-17, among which 11 and 12 will be invalid - i already returned them in first request. To make it possible, we need to make full dumps of database and save full state for several minutes, this is not very easy to solve.

[morozstepan]

So, with all that stuff server system for providing feed will be either complicated, or non-consistent. What if we define page size (e.g. 10 items) and prepare these static pages on server? All we have to tell client is url to next page. Yes, it limits the control, but it's much easier to implement and very stable. What do you think?

[jesper-sjovall]

Morozstepan, have you read my issues nr #5 I have in this issues a idea to
precreate result-set base of day to day for minimize the load of the server, but the main problem with this is this will limit the customize of the fetch result.

I also see the problem with how to use "offset" but I think this will be no big problem, In my case I probably will use the database index or time-stamp. A other thing we can do here is to limit the result can't be fetch today that will prevent the system to fetch data that is change under the day.