Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added pagination to listJobs endpoint #96

Merged
merged 2 commits into from
Jul 11, 2024
Merged

Conversation

marjo-luc
Copy link
Member

@marjo-luc marjo-luc commented Jul 11, 2024

Added optional pagination parameters to listJobs endpoint. Sample maap-py calls and their corresponding MAAP API requests:

r = maap.listJobs("mlucas") : https://api.dit.maap-project.org/api/dps/job/mlucas/list

r = maap.listJobs("mlucas", 20, 10) : https://api.dit.maap-project.org/api/dps/job/mlucas/list?page_size=20&offset=10

Work needed for this ticket

maap/maap.py Outdated
Comment on lines 262 to 276

query_params = []
if page_size is not None:
query_params.append(f"page_size={page_size}")
if offset is not None:
query_params.append(f"offset={offset}")

query_string = "&".join(query_params)

if query_string:
endpoint = endpoints.DPS_JOB_LIST + "?" + query_string
url = os.path.join(self.config.dps_job, username, endpoint)
else:
url = os.path.join(self.config.dps_job, username, endpoints.DPS_JOB_LIST)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should not be manually adding query params to a URL. The requests library properly deals with them, including doing proper encoding. Simply construct a params dict and pass it as an additional named argument to requests.get:

Suggested change
query_params = []
if page_size is not None:
query_params.append(f"page_size={page_size}")
if offset is not None:
query_params.append(f"offset={offset}")
query_string = "&".join(query_params)
if query_string:
endpoint = endpoints.DPS_JOB_LIST + "?" + query_string
url = os.path.join(self.config.dps_job, username, endpoint)
else:
url = os.path.join(self.config.dps_job, username, endpoints.DPS_JOB_LIST)
url = os.path.join(self.config.dps_job, username, endpoints.DPS_JOB_LIST)
params = {k: v for k, v in (("page_size", page_size), ("offset", offset)) if v}

And then, change the call to requests.get that follows, to this:

        response = requests.get(
            url=url,
            headers=headers,
            params=params,
        )

Copy link
Contributor

@chuckwondo chuckwondo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marjo-luc
Copy link
Member Author

I created #97 to implement the unit test method stubs. It looks like there's more work needed to get this into a functional state as well (removing maap.cfg reference and mocking the API), which falls out of scope of this ticket.

Copy link
Contributor

@chuckwondo chuckwondo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@marjo-luc marjo-luc merged commit 8dc3acf into develop Jul 11, 2024
@marjo-luc marjo-luc deleted the feature/pagination branch July 11, 2024 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants