Productionalization of GenAI Apps

Written by: Denise Schlesinger
Principal Cloud Solution Architect
Microsoft

About this workshop

Agenda

• Ops for GenAI apps
• GenAI optimization
• GenAI completion safety
• GenAI completion quality
• Risk mitigation for GenAI apps: red team, risk assessment
• Testing and Automation for GenAI apps

GenAI development challenges

• Accuracy
  • Chunking strategy
  • Consider editing content format for document processing efficiency
• Cost
  • Chunk and storage optimization
  • Cheaper/ Faster model
  • Latency
  • Faster model
  • Use PTUs
• Monitor and iterate

GenAI development stages

• Identify
  • Use cases
  • Requirements
  • Infrastructure
• Pre evaluate
  • Chunking strategy
  • Embedding models
  • Vector DB
  • LLM models
  • Evaluate prompts
• Development
  • Use prompty
  • Generate testing data
• Optimization
  • Use reranking or subqueries
• Deployment
  • Testing - use evaluators for completion Quality and safety
• Create a RAG with AI search