Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TSPS-357 Add admin endpoint to get a user's quota and update their quota #162

Merged
merged 8 commits into from
Nov 20, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions common/openapi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,51 @@ paths:
500:
$ref: '#/components/responses/ServerError'

/api/admin/v1/quota/{pipelineName}/{userId}:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: for the user-facing endpoint we call it quotas, can we be consistent here?

Suggested change
/api/admin/v1/quota/{pipelineName}/{userId}:
/api/admin/v1/quotas/{pipelineName}/{userId}:

Copy link
Collaborator Author

@jsotobroad jsotobroad Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea i was following the pattern we made for the admin pipeline path, should i update that to pipelines as well?

(i like the idea of it matching with our user facing apis)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah thanks for fixing pipelines too!

parameters:
- $ref: '#/components/parameters/PipelineName'
- $ref: '#/components/parameters/UserId'
get:
summary: Get description for a given pipeline.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update this summary

Suggested change
summary: Get description for a given pipeline.
summary: Get a user's current quota limit for a given pipeline.

tags: [ admin ]
operationId: getQuota
responses:
200:
description: Success
content:
application/json:
schema:
$ref: '#/components/schemas/AdminQuota'
400:
$ref: '#/components/responses/BadRequest'
403:
$ref: '#/components/responses/PermissionDenied'
500:
$ref: '#/components/responses/ServerError'
patch:
summary: Update attributes for a given pipeline.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update this summary

tags: [ admin ]
operationId: updateQuotaLimit
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/UpdateQuotaLimitRequestBody'
responses:
200:
description: Success
content:
application/json:
schema:
$ref: '#/components/schemas/AdminQuota'
400:
$ref: '#/components/responses/BadRequest'
403:
$ref: '#/components/responses/PermissionDenied'
500:
$ref: '#/components/responses/ServerError'

# Pipeline related APIs
/api/pipelines/v1:
get:
Expand Down Expand Up @@ -355,6 +400,14 @@ components:
schema:
type: string

UserId:
name: userId
in: path
description: A string identifier to used to identify a terra user
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description: A string identifier to used to identify a terra user
description: A string identifier to used to identify a Terra user

required: true
schema:
type: string

WorkspaceId:
name: workspaceId
in: path
Expand Down Expand Up @@ -528,6 +581,21 @@ components:
workspaceGoogleProject:
$ref: "#/components/schemas/PipelineWorkspaceGoogleProject"

AdminQuota:
description: |
Object containing the use id, pipeline identifier, quota limit, and quota usage of a Pipeline for a user.
type: object
required: [ userId, pipelineName, quotaLimit, quotaConsumed ]
properties:
userId:
$ref: "#/components/schemas/UserId"
pipelineName:
$ref: "#/components/schemas/PipelineName"
quotaLimit:
$ref: "#/components/schemas/QuotaLimit"
quotaConsumed:
$ref: "#/components/schemas/QuotaConsumed"

AsyncPipelineRunResponse:
description: Result of an asynchronous pipeline run request.
type: object
Expand Down Expand Up @@ -859,6 +927,15 @@ components:
wdlMethodVersion:
$ref: "#/components/schemas/PipelineWdlMethodVersion"

UpdateQuotaLimitRequestBody:
description: |
json object containing the admin provided information to update a users quota limit with
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit wording

Suggested change
json object containing the admin provided information to update a users quota limit with
json object containing the admin provided information used to update a user's quota limit

type: object
required: [ quotaLimit ]
properties:
quotaLimit:
$ref: "#/components/schemas/QuotaLimit"

UserId:
description: |
The identifier string for the user who submitted a job request.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
package bio.terra.pipelines.app.controller;

import bio.terra.common.exception.BadRequestException;
import bio.terra.common.iam.SamUser;
import bio.terra.common.iam.SamUserFactory;
import bio.terra.pipelines.app.configuration.external.SamConfiguration;
import bio.terra.pipelines.common.utils.PipelinesEnum;
import bio.terra.pipelines.db.entities.Pipeline;
import bio.terra.pipelines.db.entities.UserQuota;
import bio.terra.pipelines.dependencies.sam.SamService;
import bio.terra.pipelines.generated.api.AdminApi;
import bio.terra.pipelines.generated.model.*;
import bio.terra.pipelines.service.PipelinesService;
import bio.terra.pipelines.service.QuotasService;
import io.swagger.annotations.Api;
import jakarta.servlet.http.HttpServletRequest;
import org.slf4j.Logger;
Expand All @@ -27,19 +30,22 @@ public class AdminApiController implements AdminApi {
private final HttpServletRequest request;
private final PipelinesService pipelinesService;
private final SamService samService;
private final QuotasService quotasService;

@Autowired
public AdminApiController(
SamConfiguration samConfiguration,
SamUserFactory samUserFactory,
HttpServletRequest request,
PipelinesService pipelinesService,
SamService samService) {
SamService samService,
QuotasService quotasService) {
this.samConfiguration = samConfiguration;
this.samUserFactory = samUserFactory;
this.request = request;
this.pipelinesService = pipelinesService;
this.samService = samService;
this.quotasService = quotasService;
}

private static final Logger logger = LoggerFactory.getLogger(AdminApiController.class);
Expand All @@ -58,6 +64,17 @@ public ResponseEntity<ApiAdminPipeline> getPipeline(String pipelineName) {
return new ResponseEntity<>(pipelineToApiAdminPipeline(updatedPipeline), HttpStatus.OK);
}

@Override
public ResponseEntity<ApiAdminQuota> getQuota(String pipelineName, String userId) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to be super clear, could we rename this getQuotaForUserAndPipeline or similar?

final SamUser authedUser = getAuthenticatedInfo();
samService.checkAdminAuthz(authedUser);
PipelinesEnum validatedPipelineName =
PipelineApiUtils.validatePipelineName(pipelineName, logger);

UserQuota userQuota = quotasService.getQuotaForUserAndPipeline(userId, validatedPipelineName);
return new ResponseEntity<>(userQuotaToApiAdminQuota(userQuota), HttpStatus.OK);
}

@Override
public ResponseEntity<ApiAdminPipeline> updatePipeline(
String pipelineName, ApiUpdatePipelineRequestBody body) {
Expand All @@ -74,6 +91,28 @@ public ResponseEntity<ApiAdminPipeline> updatePipeline(
return new ResponseEntity<>(pipelineToApiAdminPipeline(updatedPipeline), HttpStatus.OK);
}

@Override
public ResponseEntity<ApiAdminQuota> updateQuotaLimit(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly could this be updateQuotaLimitForUserAndPipeline or similar?

String pipelineName, String userId, ApiUpdateQuotaLimitRequestBody body) {
final SamUser authedUser = getAuthenticatedInfo();
samService.checkAdminAuthz(authedUser);
PipelinesEnum validatedPipelineName =
PipelineApiUtils.validatePipelineName(pipelineName, logger);

// Check if a row exists for this user + pipeline. Throw an error if it doesn't
// A row should exist if a user has run into a quota issue before.
if (!quotasService.isUserQuotaPresent(userId, validatedPipelineName)) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious about the thought process behind this approach, versus adding an option to getQuotaForUserAndPipeline that specifies whether to create a new row if the user isn't found and just reusing that method below?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm well calling the other admin endpoint will actually create that row for the user cuz of how thegetQuotaForUserAndPipeline function works so this is all kind of moot. What scenario do we actually want to catch here? Is it that the user has used our service before? is it that the user_id is an actual terra user_id?

we could write a get function that errors out if the row doesnt exist instead of generating a new one for these admin calls

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I'm saying we could add an input to getQuotaForUserAndPipeline that's like addRowIfAbsent=True and toggle that functionality so we could reuse it, if that input is False and there's no row for that user then we throw. if that feels too complicated, that's fine, just an idea that I'm not at all tied to

throw new BadRequestException(
String.format(
"User quota not found for user %s and pipeline %s",
userId, validatedPipelineName.getValue()));
}
int newQuotaLimit = body.getQuotaLimit();
UserQuota userQuota = quotasService.getQuotaForUserAndPipeline(userId, validatedPipelineName);
userQuota = quotasService.updateQuotaLimit(userQuota, newQuotaLimit);
return new ResponseEntity<>(userQuotaToApiAdminQuota(userQuota), HttpStatus.OK);
}

public ApiAdminPipeline pipelineToApiAdminPipeline(Pipeline pipeline) {
return new ApiAdminPipeline()
.pipelineName(pipeline.getName().getValue())
Expand All @@ -85,4 +124,12 @@ public ApiAdminPipeline pipelineToApiAdminPipeline(Pipeline pipeline) {
.workspaceGoogleProject(pipeline.getWorkspaceGoogleProject())
.wdlMethodVersion(pipeline.getWdlMethodVersion());
}

public ApiAdminQuota userQuotaToApiAdminQuota(UserQuota userQuota) {
return new ApiAdminQuota()
.userId(userQuota.getUserId())
.pipelineName(userQuota.getPipelineName().getValue())
.quotaLimit(userQuota.getQuota())
.quotaConsumed(userQuota.getQuotaConsumed());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -75,4 +75,32 @@ public UserQuota updateQuotaConsumed(UserQuota userQuota, int newQuotaConsumed)
userQuota.setQuotaConsumed(newQuotaConsumed);
return userQuotasRepository.save(userQuota);
}

/**
* This method updates the quota limit for a given user quota. This should only be called from the
* Admin Controller
*
* @param userQuota - the user quota to update
* @param newQuotaLimit - the new quota limit
* @return - the updated user quota
*/
public UserQuota updateQuotaLimit(UserQuota userQuota, int newQuotaLimit) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for clarity could we call this adminUpdateUserQuotaLimit ? thank you for mentioning that this is meant to be called from the admin controller.

if (newQuotaLimit < userQuota.getQuotaConsumed()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we would allow setting the newQuotaLimit to be equal to the user's current QuotaConsumed, thereby preventing any further activity, is that right? (sounds good to me)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct

throw new InternalServerErrorException(
String.format(
"New quota limit: %d, is less than the quota consumed: %d",
newQuotaLimit, userQuota.getQuotaConsumed()));
}
userQuota.setQuota(newQuotaLimit);
return userQuotasRepository.save(userQuota);
}

/**
* @param userId - the user id
* @param pipelineName - the pipeline name
* @return - true if the user quota is present, false otherwise
*/
public boolean isUserQuotaPresent(String userId, PipelinesEnum pipelineName) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this doesn't do any other authz checking, can we mark this as also must be called by an admin controller? (just in documentation / function name, as above)

Copy link
Collaborator Author

@jsotobroad jsotobroad Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would say this could be called from wherever cuz its not actually altering the database. The other function is altering the database in a way we dont expect the service to be doing

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a good point, sounds good 👍

return userQuotasRepository.findByUserIdAndPipelineName(userId, pipelineName).isPresent();
}
}
Loading
Loading