Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MODEXPS-273] Improve logging for job launching process #610

Open
wants to merge 19 commits into
base: master
Choose a base branch
from

Conversation

Saba-Zedginidze-EPAM
Copy link
Contributor

@Saba-Zedginidze-EPAM Saba-Zedginidze-EPAM commented Dec 24, 2024

Purpose

https://folio-org.atlassian.net/browse/MODEXPS-273

Approach

  • Improve logging and validation logic for export job
  • Fetch Claim Sent pieces instead of Late for CLAIMS export
  • Conditionally trigger Ftp/Minio steps for CLAIMS
  • Do not download file content in Ftp step, fetch it from the memory
  • Remove Export History step for CLAIMS

@BKadirkhodjaev BKadirkhodjaev requested a review from a team December 26, 2024 07:58
.listener(listener)
.start(steps[0]);
for (int i = 1; i < steps.length; i++) {
builder = builder.next(steps[i]);
Copy link
Contributor

@SerhiiNosko SerhiiNosko Dec 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit harder to understand the sequence now, need to find where steps are declared in this case, but can leave if you prefer to leave

var ediExportConfig = ediObjectMapper.readValue((String)jobParameters.get(EDIFACT_ORDERS_EXPORT), VendorEdiOrdersExportConfig.class);
var edifactOrderAsString = (String) ExecutionContextUtils.getExecutionVariable(stepExecution,"edifactOrderAsString");
var ediExportConfig = ediObjectMapper.readValue((String) jobParameters.get(EDIFACT_ORDERS_EXPORT), VendorEdiOrdersExportConfig.class);
if (ediExportConfig.getIntegrationType() != ORDERING && ediExportConfig.getTransmissionMethod() != FILE_DOWNLOAD) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to minio we should save in any case for both transition methods: File download, and FTP.
We will upload that file into Minio and after this will transfer to Ftp if transitionMethod is Ftp.
But it is required to have file in Minio in any case

return configurationClient.getConfigById(configId);
} catch (NotFoundException e) {
logger.error("getConfigById:: Couldn't fetch configuration entry by id: '{}'", configId, e);
throw new NotFoundException("Configuration entry not found for id: " + configId, e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we just do throw e
Or you need an another text in the excpetion message?

ftpStorageService.uploadToFtp(ediExportConfig, fileContent, FilenameUtils.getName(uploadedFilePath));

var stepExecution = chunkContext.getStepContext().getStepExecution();
var fileName = (String) ExecutionContextUtils.getExecutionVariable(stepExecution, EDIFACT_FILE_NAME);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we rename to ACQ_EXPORT_FILE_NAME? In this case we will have the same convention as for ACQ_EXPORT_FILE

var ediExportConfig = ediObjectMapper.readValue((String)jobParameters.get(EDIFACT_ORDERS_EXPORT), VendorEdiOrdersExportConfig.class);
var uploadedFilePath = (String) ExecutionContextUtils.getExecutionVariable(stepExecution, UPLOADED_FILE_PATH);
var ediExportConfig = ediObjectMapper.readValue((String) jobParameters.get(EDIFACT_ORDERS_EXPORT), VendorEdiOrdersExportConfig.class);
if (ediExportConfig.getIntegrationType() != ORDERING && ediExportConfig.getTransmissionMethod() != FTP) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit hard to read with inverse conditions. Also having integration type here will lead us to modify source code of tasklet during adding new integration type. If we could not do such inverse configuration on some higher config level - we can leave as it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants