[No.10]Support for user customization of plots displayed on the website

[itutu-tienday]

Requirement

Plot images and statistical data posted on the optinist-for-server website, To provide documentation of the customization procedure and adjustments to the system to allow users (non-IT engineers and data researchers) to easily customize the above posted data.

• Supplemental
  • GUI-based customization of displayed data is out of scope.
    • Since the user analysis algorithm is implemented in python code, the customization of the display data may be done by editing the python code as well.
    • However, the customization method should be as user-friendly as possible.

related docs

• requirement
  • https://drive.google.com/file/d/1FTb2BFeDtG4FaN2Dj2IKrTsKjqLMAUs_/view
    • No.4
• notes considered
  • https://drive.google.com/file/d/1sj_yqwqufpFvZLa7EPkg26F3pGXkaDA9/view

Current Specifications

Current Website display specifications

Experimental View

Plot images (right side from pixel image)

• Header column
  • API
    • GET /public/experiments
      • header.graph_titles
  • Definition Location
    • studio.app.optinist.routers.expdb.EXPERIMENT_GRAPHS
• Plot URL
  • API response
    • GET /public/experiments
      • items.graph_urls
  • Definition Location
    • studio.app.optinist.routers.expdb.get_experiment_urls

Cell View

Plot Images (right side from Cell Mask)

• Header columns
  • API response
    • GET /public/cells
      • header.graph_titles
  • Definition Location
    • studio.app.optinist.routers.expdb.CELL_GRAPHS
• Plot URL
  • API Response
    • GET /public/cells
      • items.graph_urls
  • Definition Location
    • studio.app.optinist.routers.expdb.expdbcell_transformer

Statistics (right side from Plot Images)

• API Response
  • GET /public/cells
    • items.statistics
• Definition Location
  • studio.app.optinist.routers.expdb.get_cell_urls

Specifications for generating various data

Flow of data generation

The data creation process consists of the following batch processes.

• Flow of data generation
  1. The data to be processed is input into the specified directory (env:EXPDB_DIR).
  2. The following BATCH program detects data input.

     python run_expdb_batch.py -o 1
     

  3. Each analysis result data is generated.

Plot Images generation location

• studio.app.optinist.core.expdb.batch_runner.__process_dataset_registration

  expdb_batch.generate_plots(stat_data=stat_data)
  expdb_batch.generate_cellmasks()
  expdb_batch.generate_pixelmaps()
  

  • Check to see if there are any other leaks

Cells Statistics generation location

• studio.app.optinist.core.expdb.crud_cells.bulk_insert_cells

  bulk_insert_cells(db, exp.id, stat_data)
  

Tasks

Code modification

• 1. Configuration part
  • Put each constant that can be edited by users into an external file (yaml)
• 2. Code implementation part
  • 1. Simplify the method of adding output destinations for plot images
  • 2. Simplify the method of adding output destinations for statistics
  • *If there are any improvement proposals to make it easier for users (non-IT engineers) to edit, apply them
  • *Cutting out code that may be edited by users into dedicated modules, etc.

Documentation

• To the Experiments/Cells View
  • Procedure for updating the header information of the view
  • Procedure for adding an output destination for plot images
  • Procedure for adding an output destination for statistics

[itutu-tienday]

Scope of request to developer

• Request for 1. of ”Code modification".
• For "Code implementation part" of “Code modification”, please make a proposal once.
  • Request implementation depending on the result of the proposal.
• For “Documentation”, we would like to discuss the scope of response based on the proposal.

[tuanna26]

Implementation plan for your requirements:

• Migrate CELL_GRAPHS and EXPERIMENT_GRAPHS into YAML. Update the backend code to read configuration settings directly from the YAML
• Return the configuration via API, enabling the frontend to dynamically adapt the display based on the YAML settings
• Users can modify the YAML file by adding, updating, or removing entries or simply comment/uncomment sections to customize which columns or data points are displayed.

Some suggestions to increase user experience:

• Implement a Python or Bash script
• Develop an API and a UI for authorized users

Above proposals remove the need for manual file edits and provide a more seamless and intuitive experience. The configuration can be stored in YAML file or database.

[milesAraya]

One yaml for all or multiple yaml for each parameter type?