Skip to content

Commit

Permalink
limitations
Browse files Browse the repository at this point in the history
  • Loading branch information
slobentanzer committed Feb 16, 2024
1 parent 60322cb commit b82b231
Showing 1 changed file with 4 additions and 6 deletions.
10 changes: 4 additions & 6 deletions content/30.discussion.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,13 @@ We allow self-hosting of open-source models on any scale, from dedicated hardwar

### Limitations

The current generation of LLMs is not yet ready for unsupervised use in biomedical research.
The current generation of LLMs is not yet ready for unsupervised use in biomedical research with its vast array of unique subfields.
Effectively supporting this diversity through robust and contextually aware LLM interactions is a daunting task.
While we have taken steps to mitigate the risks of using LLMs such as independent benchmarks, fact-checking, and RAG processes, we cannot guarantee that the models will not produce harmful outputs.
We see current LLMs, particularly in the scope of the BioCypher ecosystem, as helpful tools to assist human researchers, alleviating menial and repetitive tasks and helping with technical aspects such as query languages.
They are not meant to replace human ingenuity and expertise but to augment it with their complementary strengths.

Depending on generic open-source libraries such as LangChain [@langchain] and Pytest [@pytest] allows us to focus on the biomedical domain but also introduces technical dependencies on these libraries.
While we support those upstream libraries via pull requests, we depend on their maintainers for future updates.
In addition, keeping up with these rapid developments is demanding on developer time, which is only sustainable in a community-driven open-source effort.
For the continued relevance of our framework, it is essential that its components, such as the benchmark, are maintained as the field evolves.
Despite the user-friendly design of BioChatter, there may be a learning curve for researchers unfamiliar with LLMs or the specific functionalities of the framework.
Encouraging adoption and providing adequate training and support are critical for maximizing its impact in the biomedical research community.

### Future directions

Expand Down

0 comments on commit b82b231

Please sign in to comment.