# Automated AI Security Testing

> Master automated AI security testing methodologies, leveraging open-source frameworks to continuously evaluate LLMs and agentic systems for vulnerabilities.

Canonical: https://codelivly.com/learning-paths/automated-ai-security-testing
Last updated: 2026-10-14

| | |
|---|---|
| Level | Advanced |
| Category | Cybersecurity |
| Rooms | 5 |
| Lessons | 14 |
| Estimated time | 2 h 30 min |

This advanced learning path explores the automation of AI red teaming and security evaluations. You will learn how to deploy, configure, and scale industry-standard automated testing frameworks like Garak, PyRIT, and promptfoo to systematically probe LLMs and multi-agent systems for prompt injection, data leakage, and agentic misuse. Discover how to integrate these tools into CI/CD pipelines for continuous AI security monitoring and regression testing.

## Rooms in this path

1. [Why Automation Matters in AI Red Teaming](https://codelivly.com/rooms/why-automation-matters-in-ai-red-teaming) - Understand the critical need for automated AI security testing, contrasting manual red teaming limitations with the…
2. [Using Garak for Automated LLM Vulnerability Scanning](https://codelivly.com/rooms/using-garak-for-automated-llm-vulnerability-scanning) - Master the deployment and configuration of Garak, the industry-standard open-source LLM vulnerability scanner, to…
3. [Using PyRIT for Adversarial Testing](https://codelivly.com/rooms/using-pyrit-for-adversarial-testing) - Master Microsoft's PyRIT framework to orchestrate complex, multi-step adversarial attacks and evaluate LLM resilience…
4. [Using promptfoo for Prompt Evaluation & Regression Testing](https://codelivly.com/rooms/using-promptfoo-for-prompt-evaluation-regression-testing) - Master promptfoo to build automated, CI/CD-integrated evaluation suites for testing LLM prompt quality, safety…
5. [Guided Lab: Running a Full Automated Scan](https://codelivly.com/rooms/guided-lab-running-a-full-automated-scan) - A comprehensive, hands-on capstone lab where you will execute a full automated AI security scan using Garak, PyRIT, and…

## Related

- [Career path: AI Red Teaming and Adversarial ML](https://codelivly.com/career-paths/ai-red-teaming-and-adversarial-ml)
- [Bug Bounty Workflow & Disclosure](https://codelivly.com/learning-paths/bug-bounty-workflow-and-disclosure)
- [Security Monitoring and Compliance Automation](https://codelivly.com/learning-paths/security-monitoring-and-compliance-automation)
- [Software Supply Chain Security](https://codelivly.com/learning-paths/software-supply-chain-security)