Lecture 12: Towards Building Safe & Secure Agentic AI
Link to lecture recording on YouTube
Date: 2025-12-08
Speaker: Dawn Song 宋晓东
Speaker’s Social Profile: Website / University Profile / Google Scholar / GitHub / LinkedIn / X (Twitter)
Education:
- Ph.D. in Computer Science, 2002, University of California, Berkeley
- M.S. in Computer Science, 1999, Carnegie Mellon University
Work:
- Professor in Computer Science, Department of Electrical Engineering & Computer Science, University of California, Berkeley
- Co-director, Center on Responsible Decentralized Intelligence, University of California, Berkeley
Notes
Broad spectrum of AI risks
- misuse / malicious use: scams, misinformation etc.
- malfunction: bias, harm, loss of control etc.
- systematic risks: privacy control, copyright, climate / environmental etc.
Important to consider agentic AI in adversary setting:
- history has shown attackers always follow footsteps of new technology (sometimes even leads)
- stake is even higher with AI: 1) attacker will have higher incentives as AI controls more systems; 2) consequence of misuse getting more severe as AI becomes more capable
AI safety: preventing harm that a system might inflict upon the external environment
AI security: protecting the system itself against harm and exploitation from malicious external actors
Goal: advance safe and secure AI innovation to ensure its potential benefits are responsibly realized and widely shared
Overview of Agentic AI Safety & Security
LLM safety: refer to Prof. Song’s keynote speech at ICLR 2025 for more details
LLM agent safety
| System | Example | Components |
| – | – | – |
| hybrid / compound system | agentic AI system | symbolic components + neural components
[Incomplete, work in progress]
References