The EthicalAI automated assessment framework project aims to provide insightful metrics in the realm of AI ethics, contributing to the broader AI ecosystem.
📘 Read my draft about ethical AI here.
EthicalAI automated assessment frameowrk is developed in Python and is specifically crafted to analyze and understand the ethical dimensions of AI chatbots that are compatible with OpenAI's API. It operates by posing a series of questions to the AI, gathering responses, and evaluating them against established ethical benchmarks. This tool is structured to function as a weekly GitHub Action.
- Automated Assessments: Conducts weekly automated evaluations of AI's ethical understanding.
- Results Documentation: Generates a detailed results summary in a Markdown-formatted table (model.md).
- Future Enhancements: I am exploring several avenues for improvement, including diverse prompt designs, understanding AI's knowledge limits, multi-domain responses analysis, and integrating game-based assessments.
I am continuously refining our assessment test suites and expanding our coverage to include more AI API endpoints. Your contributions in these areas are highly valued and welcomed.