Learn Testing of LLMs and AI Apps with DeepEval, RAGAs and more using Ollama (New Course)
AI Summary
Course Overview:
- Welcome to the course on testing AI applications using Deep Eval, RAGs, and local large language models.
- Aims to introduce AI applications and testing methodologies for those without prior AI knowledge.
Course Content:
- Introduction to AI Applications:
- Understanding different types of AI applications and tools for testing.
- Evaluating AI applications and the importance of local large language models (LLMs).
- Why Learn Now:
- Increased adoption of AI solutions post-2021 by companies.
- Need for new testing approaches beyond traditional UI-based tools.
- Key Learning Objectives:
- Introduction to local LLMs and their applications.
- Deep Eval testing of RAG apps.
- Testing AI agents with Deep Eval.
- Hands-on experience using tools like Hugging Face Evaluate, Galileo, and others.
- Evaluation Importance:
- Evaluating responses from LLMs using standard datasets for accurate performance assessment.
- Understanding metrics for evaluation: traditional (e.g., exact match, BLEU, F1 scores) and non-traditional (e.g., semantic understanding, embedding similarities).
- Hands-On Learning:
- Course utilizes Visual Studio Code and Jupyter Notebooks for practical coding and testing experiences.
Conclusion:
- Emphasis on the importance of testing AI applications and LLMs locally to maintain privacy and ensure reliability in outputs.