Create response header with pagination information for Discogs

[elrayle]

Description

Add pagination information for DISCOGs matching the pagination format for LD4P_CACHE authorities...

Challenge

LD4P_CACHE returns pagination as as offset (i.e. record number) and page size.
Discogs returns pagination as a page number and page size.

Processing a page request

LD4P_CACHE

LD4P_CACHE query request looks like...

https://lookup.ld4l.org/authorities/search/linked_data/agrovoc_ld4l_cache?q=milk&maxRecords=4&startRecord=2&response_header=true

key parameters for pagination:

• startRecord - (int) offset starting at 1
• maxRecords - (int) number of results to return
• responseHeader (true|false) - include responseHeader if true

DISCOGs

DISCOGs in the QA request should use the same parameter names.

https://lookup.ld4l.org/authorities/search/discogs/release?q=sinatra&maxRecords=4&startRecord=2&response_header=true

The parameters are converted to DISCOGs pagination parameters as follows.

Ex. startRecord = 11 maxRecords = 5

• page_num = startRecord / maxRecords + 1
• page_size = maxRecords

Use page_num and page_size when making the request to DISCOGs.

Query response header

LD4P_CACHE

LD4P_CACHE creates the following header with pagination information.

"response_header": {
    "start_record": 11,
    "requested_records": 5,
    "retrieved_records": 5,
    "total_records": 256
}

DISCOGs

DISCOGs can calculate the fields...

Ex. page_number = 3 page_size = 5

"response_header": {
    "start_record": 11,          # ((page_number-1) * page_size) + 1
    "requested_records": 5,      # page_size
    "retrieved_records": 5,      # number of results being returned
    "total_records": 256         # does DISCOGs provide this info
}

How to include the response header in results?

if params[:response_header] != true

[ # results as they are currently returned ]

if params[:response_header] == true

{
    "results": [ # results as they are currently returned ]
    "response_header": {
       "start_record": 11,
       "requested_records": 5,
       "retrieved_records": 5,
       "total_records": 256
   }
}

Related Work

Issue #213 Report number of results available
PR #samvera/questioning_authority#283 add option to include a response header in results

[tworrall]

I have a question about the startRecord param and this math:

'''page_num = startRecord / maxRecords + 1'''

If my startRecord param is 5 and my maxRecords is 5, then the query is going to start on page 2 with record 6.

Is that the correct behavior?

(startRecord seems like an odd way to do this. )

[elrayle]

The math is off slightly at the page boundary. This should work instead...

page_num = (startRecord - 1) / maxRecords + 1

>> (1-1)/5 + 1  # 1
>> (2-1)/5 + 1  # 1
>> (3-1)/5 + 1  # 1
>> (4-1)/5 + 1  # 1
>> (5-1)/5 + 1  # 1

>> (6-1)/5 + 1  # 2
>> (7-1)/5 + 1  # 2
>> (8-1)/5 + 1  # 2
>> (9-1)/5 + 1  # 2
>> (10-1)/5 + 1  # 2

>> (11-1)/5 + 1  # 3
>> (12-1)/5 + 1  # 3
>> (13-1)/5 + 1  # 3
>> (14-1)/5 + 1  # 3
>> (15-1)/5 + 1  # 3

>> (16-1)/5 + 1  # 4

It's not an exact translation given that with startRecord, you can start in the middle of a page and ask for a full page (e.g. startRecord=5&maxRecord=10). It is expected that with pagination, the requests will be for the first record on a page (e.g. startRecords = 1, 6, 11, 16, etc. when maxRecords==5).

[elrayle]

You could support both approaches. If the request comes in with startRecord, then use the calculation. If it comes in with pageNum, then just use that.

[tworrall]

The real issue is that we're mixing apples and oranges. A "start record" parameter should not be used with page numbering. It should be either: get me x records starting with record n; or get me page x showing n records per page.