Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add potential safety precaution: curated failsafe #178

Open
NatalieZelenka opened this issue Jul 5, 2023 · 0 comments
Open

Add potential safety precaution: curated failsafe #178

NatalieZelenka opened this issue Jul 5, 2023 · 0 comments
Labels
feedback Feedback on Data Hazards

Comments

@NatalieZelenka
Copy link
Contributor

NatalieZelenka commented Jul 5, 2023

This safety precaution may apply to multiple labels, for example "Danger of Misuse" or "Reinforces Existing Bias"

For example, if you're building a chatbot adding a process after the LLM to check if certain keywords are in the output of the LLM and ensuring such responses are not shared if they are, e.g. racist, telling people to commit suicide, or otherwise inappropriate.

Sometimes these kinds of fixes are a bit annoying as they paper over deeper issues in using data science/AI (and it's also possible in this example that they are weak to prompt injections), but I also think that it is one type of precaution to be aware of.

For example, in the phenotype predictor, perhaps it would have been useful to have a curated list of phenotypes that are sensible to predict on. This could really have ensured that the thing I was making was less open to misuse.

@NatalieZelenka NatalieZelenka added the feedback Feedback on Data Hazards label Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feedback Feedback on Data Hazards
Projects
None yet
Development

No branches or pull requests

1 participant