-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create a Docker image for topic generator #505
Changes from all commits
1dc6db5
51d70a6
92502dc
ca5ed9b
a7ab9ce
9719459
722519d
04d8560
4d75566
fd79855
4455089
5663ce8
8c8d6a0
9f281ad
542902d
6ea4210
bbf0da3
80ffaa0
d3c477a
8d7f369
81b5e7a
e422ab2
da4268c
7d09b13
7eb18c9
f782db3
ce4ad53
0ff65ac
223cafc
048dc9e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
FROM eclipse-temurin:21.0.2_13-jdk-jammy | ||
|
||
# Update the APT cache | ||
# Prepare for Java download | ||
RUN apt-get update \ | ||
&& apt-get upgrade -y \ | ||
&& apt-get install unzip jq -y --no-install-recommends | ||
|
||
# Install aws cli so dockstore-deploy can use the aws cli to upload files to S3 | ||
RUN curl -s "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip | ||
RUN unzip -q /tmp/awscliv2.zip -d /tmp | ||
RUN /tmp/aws/install | ||
RUN rm -f /tmp/awscliv2.zip | ||
RUN rm -fr /tmp/aws | ||
|
||
# Copy, for example, topicgenerator-1.16.0-SNAPSHOT.jar, but not topicgenerator-1.16.0-SNAPSHOT-sources.jar | ||
COPY topicgenerator/target/topicgenerator*[^s].jar /home/topic-generator.jar | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What's the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. After a build, there are 2 JARs in the target directory: topicgenerator-1.16.0-SNAPSHOT-sources.jar The |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,6 +14,7 @@ | |
import com.beust.jcommander.ParameterException; | ||
import com.google.common.collect.Lists; | ||
import io.dockstore.common.NextflowUtilities; | ||
import io.dockstore.common.NextflowUtilities.NextflowParsingException; | ||
import io.dockstore.openapi.client.ApiClient; | ||
import io.dockstore.openapi.client.ApiException; | ||
import io.dockstore.openapi.client.api.ExtendedGa4GhApi; | ||
|
@@ -320,7 +321,13 @@ | |
} | ||
|
||
private Optional<FileWrapper> getNextflowMainScript(String nextflowConfigFileContent, Ga4Ghv20Api ga4Ghv20Api, String trsId, String versionId, DescriptorTypeEnum descriptorType) { | ||
final String mainScriptPath = NextflowUtilities.grabConfig(nextflowConfigFileContent).getString("manifest.mainScript", "main.nf"); | ||
final String mainScriptPath; | ||
try { | ||
mainScriptPath = NextflowUtilities.grabConfig(nextflowConfigFileContent).getString("manifest.mainScript", "main.nf"); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. First time I ran this, I believe I got a NextflowParsingException, which caused the program to exit early. Put this in a try catch. Kathy thinks that it may have just been logging noise that I saw, so I could be wrong. But the method can throw a NextFlowParsingException, so we should catch it anyway. |
||
} catch (NextflowParsingException e) { | ||
LOG.error("Could not grab config", e); | ||
return Optional.empty(); | ||
} | ||
Check warning on line 330 in topicgenerator/src/main/java/io/dockstore/topicgenerator/client/cli/TopicGeneratorClient.java Codecov / codecov/patchtopicgenerator/src/main/java/io/dockstore/topicgenerator/client/cli/TopicGeneratorClient.java#L326-L330
|
||
try { | ||
return Optional.of(ga4Ghv20Api.toolsIdVersionsVersionIdTypeDescriptorRelativePathGet(trsId, descriptorType.toString(), versionId, mainScriptPath)); | ||
} catch (ApiException exception) { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,8 +3,8 @@ | |
import com.knuddels.jtokkit.api.ModelType; | ||
|
||
public enum AIModelType { | ||
CLAUDE_3_5_SONNET("anthropic.claude-3-5-sonnet-20240620-v1:0", 0.003, 0.015, 200000), | ||
CLAUDE_3_HAIKU("anthropic.claude-3-haiku-20240307-v1:0", 0.00025, 0.00125, 200000), | ||
CLAUDE_3_5_SONNET("us.anthropic.claude-3-5-sonnet-20240620-v1:0", 0.003, 0.015, 200000), | ||
CLAUDE_3_HAIKU("us.anthropic.claude-3-haiku-20240307-v1:0", 0.00025, 0.00125, 200000), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I needed to change these to be cross-region inferences, so QA would work. |
||
GPT_4O_MINI(ModelType.GPT_4O_MINI.getName(), 0.000150, 0.000600, ModelType.GPT_4O_MINI.getMaxContextLength()); | ||
|
||
private final String modelId; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At some point in the future, especially if we end up creating additional Docker images, we might want to consider moving this code to dockstore/workflows. But looks good now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, I noticed we create images and upload checksums for images in dockstore/dockstore and dockstore-deploy as well, so it'll be good to create a re-usable workflow. I'll create a ticket for this or maybe this can be tackled when we implement uploading the image digest for this image