TrustAI is on a mission to ensure the safety and integrity of AI systems and unlock the full potential of generative AI while maintaining control and trust. We believe in bringing security to the forefront of AI development, safeguarding against potential vulnerabilities, and promoting responsible AI innovation.

Our goal is to empower developers, researchers, and organizations to build secure and trustworthy AI systems.

• Website: http://www.trustai.pro/
• Blog: https://securaize.substack.com/
• Lab: https://lab.trustai.pro/

• ISC.AI 2024 -- LLM Jailbreaking Vulnerability Mining and Defefense
• SecGeek -- The Road Leading to LLM Security Alignment: Research on Vulnerability Mining and Alignment Defense for LLM
• Xcon 2024 -- Next-Generation Detectionand Respbonse Technology Driven by LLM Intelligent Agent
• S-tron China 2024 - S-Talent Talk
• AI x Security Summit - SG Antler

• 2024 Puyuan Large Model Challenge (Summer Season)， Safety and Trustworthiness Track - Third Prize

• CVE-2023-6656
• CVE-2024-0654
• CVE-2024-0936
• CVE-2024-0937
• 22+ CVE

Here are some of main projects we've released:

• Learn Prompt Hacking: The most comprehensive prompt hacking course available.

  • Prompt Engineering technology.
  • GenAI development technology.
  • Prompt Hacking technology.
  • LLM security defence technology.
  • LLM Hacking resources
  • LLM security papers.

• LLM Red: A GUI tool for finding LLM 0-Day 10x Faster with adversarial prompt fuzzing.

  • Automatically generate AI alignment corpus through black box prompt fuzzing.
  • 10x faster with human-in-loop automation instead of manual chat.
  • Exposure GenAI Risks, Aligned with Global AI Safety Frameworks

• LLM Protection: A SDK API that basically a One-click Alignment Proxy for AI App Integration.

  • Detect and address direct and indirect prompt injections in real-time, preventing potential harm to GenAI applications.
  • Ensure your GenAI applications do not violate the policies by detecting harmful and insecure output.
  • Safeguard sensitive PII and avoid data losses, ensuring compliance with privacy regulations.
  • Prevent data poisoning attacks on your GenAI applications through real-time prompt filtering.

• LLM Security CTF: Learn LLM/AI Security through a series of vulnerable LLM CTF challenges. No sign ups, all fees, everything on the website.

  • Stark Game: Very neat game to get intuitions for prompt injection, user need find ways to get Stark to tell the password for the level, except Stark is instructed not to reveal the word.
  • Doc: Intro to Stark Game.