Skip to content

Latest commit

 

History

History
104 lines (98 loc) · 6.42 KB

README.md

File metadata and controls

104 lines (98 loc) · 6.42 KB

Controller

This component is responsible for orchastrating the main logic for Codenator. It has five main actions:

  • Generate: This action Takes a prompt and generates text and script in context of a conversation.
  • Scan: Takes a script and send it to Code Scanner, if recommendations found to fix vulnrabilities, it sends back the recomendations to the LLM model to generate new text and script.
  • Execute: Takes a script, encrypt it and send to Code Executor, if zthe execution fails, it sends back failure message to LLM model to generate new text and script.
  • Save: Uses LLM to generate task description embedding then send it with script to Task Store to save it.
  • Load: Uses LLM to generate input text embedding and send it to Task Store to search for similar stored tasks.

IAM permissions

Controller component requires the following IAM permissions

dynamodb:GetItem
kms:encrypt

Starting the server

To start Controller server, you can use the below command:

$ main.py [optional args]

[optional args] can be any combination of:
--host: host url, default value localhost
--port: host port, default value 8080
--api-layer-host: API Layer host url, default value localhost
--api-layer-port: API Layer host port, default value 8080
--code-scanner-host: Code Scanner host url, default value localhost
--code-scanner-port: Code Scanner host port, default value 8080
--code-executor-host: Code Executor host url, default value localhost
--code-executor-port: Code Executor host port, default value 8080
--task-store-host: Task Store host url, default value localhost
--task-store-port: Task Store host port, default value 8080
--workers: number of uvicorn workers, default value 3
--namespace: namespace used to publish CloudWatc Metrics, default value Codenator/controller/
--kms: Amazon KMS key id or alias used to encrypt script sent to Code Executor.
--prompt-database-file: Loads Prompt Store from file if no DynamoDB table name was provided in --prompt-store-name option.
--prompt-store-name: DynamoDB table name used for Prompt Store.
--models-metadata-db: DynamoDB table name used to load languages and models metadata.
--models-metadata-file: Loads models metadata from file if no DynamoDB table name was provided in --models-metadata-db option.
--languages-file: Loads languages metadata from file if no DynamoDB table name was provided in --models-metadata-db option.
--conv-bucket: Amazon S3 bucket name used for conversation memory and logging.
--conv-prefix: Amazon S3 prefix name used for conversation memory and logging.

API endpoints

Task Store has the following API endpoints:

  • /ping (GET): A standard health check endpoint.
  • /list_languages (GET): Lists all supported programming languages.
  • /generate (POST): for Generate action
    • Required parameters:
      prompt: Text prompt.
      model_family: currently supports bedrock or sagemaker.
      model_name: name of the model to invoke (see DynamoDB section for more information)
      language: programing language to use.
    • Optional parameters:
      model_params: parameters used for model, (see DynamoDB section for more information).
      conv_id: conversation id, if empty a new conversation is started.
      stream: a boolean value used to stream response.
  • /scan (POST): for Scan action
    • Required parameters:
      script: Script to scan.
      model_family: currently supports bedrock or sagemaker.
      model_name: name of the model to invoke (see DynamoDB section for more information)
      language: programing language to use.
      conv_id: conversation id.
    • Optional parameters:
      model_params: parameters used for model, (see DynamoDB section for more information).
      scanner: defaults to semgrep. valid values codeguru or semgrep.
      stream: a boolean value used to stream response.
  • /execute (POST): for Execute action
    • Required parameters:
      script: Script to execute.
      model_family: currently supports bedrock or sagemaker.
      model_name: name of the model to invoke (see DynamoDB section for more information)
      language: programing language to use.
      conv_id: conversation id.
    • Optional parameters:
      model_params: parameters used for model, (see DynamoDB section for more information).
      stream: a boolean value used to stream response.
  • /save (POST): for Save action
    • Required parameters:
      script: Script to save.
      model_family: currently supports bedrock or sagemaker.
      model_name: name of the model to invoke (see DynamoDB section for more information)
      language: programing language to use.
      embedding_model_family: same as model_family but for embedding.
      embedding_model_name: same as model_name but for embedding.
    • Optional parameters:
      model_params: parameters used for model, (see DynamoDB section for more information).
      embedding_model_params: same as model_params but for embedding.
  • /load (POST): for Load action
    • Required parameters:
      prompt: text prompt used to load a task.
      model_family: currently supports bedrock or sagemaker.
      model_name: name of the model to invoke (see DynamoDB section for more information)
      language: programing language to use.
      embedding_model_family: same as model_family but for embedding.
      embedding_model_name: same as model_name but for embedding.
    • Optional parameters:
      model_params: parameters used for model, (see DynamoDB section for more information).
      embedding_model_params: same as model_params but for embedding.
      threshold: similarity threshold, defaulst to 0.1.
      limit: number of search results to return, defaults to 1