Alina Oprea is a Professor at Northeastern University in the Khoury College of Computer Sciences. She joined Northeastern University in Fall 2016 after spending 9 years as a research scientist at RSA Laboratories. Her research interests in cyber security are broad, with a focus on AI security and privacy, ML-based threat detection, cloud security, and applied cryptography. She is the recipient of the Technology Review TR35 award for her research in cloud security in 2011, the Google Security and Privacy Award in 2019, the Ruth and Joel Spira Award for Excellence in Teaching in 2020, and the CMU Cylab Distinguished Alumni Award 2024. Alina served as Program Committee co-chair of the flagship cyber security conference, the IEEE Security and Privacy Symposium in 2020 and 2021. She also served as Associate Editor of the ACM Transactions of Privacy and Security (TOPS) journal and the IEEE Security and Privacy Magazine. Her work was recognized with Best Paper Awards at NDSS 2005, AISEC in 2017, and GameSec in 2019.

Recent advances in Reinforcement learning (RL) have demonstrated how to identify optimal strategies in critical applications such as medical robots, self-driving cars, cyber security defense, and safety alignment in large language models. These applications require strong guarantees on the integrity of the RL methods used for training decision-making agents. In this talk, I will present two recent papers addressing backdoor poisoning attacks on reinforcement learning during the training phase. Both attacks use a novel threat model and significantly improve upon prior RL attacks in the literature in terms of attack success at low poisoning rates, while maintaining high episodic return. Additionally, I will discuss the challenges of designing RL algorithms that are resilient against poisoning attacks.

Eleni is a senior research scientist at Google DeepMind, based in London. Her main research interest is around creating methods that allow efficient and effective adaptation of deep neural networks to cope with distribution shifts, rapidly learning new tasks, and supporting efficient unlearning of data points.

Training increasingly-large models on increasingly-large datasets poses risks: the resulting models may compromise privacy, may make errors due to poisoned, mislabelled or outdated training data, or pose safety concerns. Unlearning is an umbrella of methods that is designed to remove the effect of certain data points from models after they have already been trained, aiming to avoid the large cost of training models from scratch when unwanted or problematic subsets of their original data are identified. In this talk, I will focus on unlearning memorized training data from models. I will present findings and open questions derived from the first NeurIPS unlearning competition, as well as two pieces of recent work that leverage the tight relationship between memorization and unlearning to derive better unlearning algorithms. I will close with a discussion of important remaining open challenges for future work.

Franziska is a tenure-track faculty at the CISPA Helmholtz Center for Information Security where she co-leads the SprintML lab. Before, she was a Postdoctoral Fellow at the University of Toronto and Vector Institute advised by Prof. Nicolas Papernot. Her current research centers around private and trustworthy machine learning. Franziska obtained her Ph.D. at the Computer Science Department at Freie University Berlin, where she pioneered the notion of individualized privacy in machine learning. During her Ph.D., Franziska was a research associate at the Fraunhofer Institute for Applied and Integrated Security (AISEC), Germany. She received a Fraunhofer TALENTA grant for outstanding female early career researchers, the German Industrial Research Foundation prize for her research on machine learning privacy, and the Fraunhofer ICT Dissertation Award 2023, and was named a GI-Junior Fellow in 2024.

As language models (LLMs) underpin various sensitive applications, preserving privacy of their training data is crucial for their trustworthy deployment. This talk will focus on the privacy of LLM adaptation data. We will see how easily sensitive data can leak from the adaptations, putting privacy in risk. We will then dive into designing protection methods, focusing on how we can obtain privacy guarantees for adaptation data, in particular for prompts. We will also compare private adaptations for open LLMs and their closed, proprietary counterparts across different axes, finding that private adaptations for open LLMs yield higher privacy, better performance, and lower costs. Finally, we will discuss how to monitor privacy of adapted LLMs through dedicated auditing. By identifying privacy risks of adapting LLMs, understanding how to mitigate them, and conducting thorough audits, we can ensure that LLMs can be employed for societal benefits without putting individual data at risk.

Alvaro Velasquez is a program manager at DARPA, where he currently leads programs on neuro-symbolic and adversarial AI. Before that, Alvaro oversaw the machine intelligence portfolio for the Information Directorate of the Air Force Research Laboratory (AFRL). Alvaro is a recipient of the distinguished paper award from AAAI and best paper and patent awards from AFRL. He has authored over 100 papers and three patents, serves as Associate Editor of the IEEE Transactions on Artificial Intelligence, and is a co-founder of the Neuro-symbolic Systems (NeuS) conference.

Adversarial AI has been instrumental in exposing the vulnerabilities of vision and language systems, particularly in recent years. However, some unconventional domains with great societal implications remain underexplored. In this talk, we briefly discuss two such domains: autonomy and biology. In particular, the sim-to-real gaps in the former present a natural adversary, whereas the latter admits novel threat models with implications on the effectiveness of existing screening tools and the effectiveness of drug discovery.

Hoda Heidari is the K&L Gates Career Development Assistant Professor in Ethics and Computational Technologies at Carnegie Mellon University, with joint appointments in Machine Learning and Societal Computing. She is affiliated with the Human-Computer Interaction Institute and Heinz College of Information Systems and Public Policy. Her research is broadly concerned with the social, ethical, and economic implications of Artificial Intelligence, particularly issues of fairness and accountability through the use of Machine Learning in socially consequential domains. Her work in this area has won a best-paper award at the ACM Conference on Fairness, Accountability, and Transparency (FAccT), an exemplary track award at the ACM Conference on Economics and Computation (EC), and a best-paper award at the IEEE Conference on Secure and Trustworthy Machine Learning (SAT-ML). Dr. Heidari co-founded and co-leads the university-wide Responsible AI Initiative at CMU. She has organized several scholarly events on Responsible and Trustworthy AI, including tutorials and workshops at top-tier Artificial Intelligence academic venues, such as NeurIPS, ICLR, and the Web conference. She is particularly interested in translating research contributions into positive impact on AI policy and practice. She has organized multiple campus-wide events and policy convenings to address AI governance and accountability. Dr. Heidari completed her doctoral studies in Computer and Information Science at the University of Pennsylvania. She holds an M.Sc. degree in Statistics from the Wharton School of Business. Before joining Carnegie Mellon as a faculty member, she was a postdoctoral scholar at the Machine Learning Institute of ETH Zurich, followed by a year at the Artificial Intelligence, Policy, and Practice (AIPP) initiative at Cornell University.

In response to rising concerns surrounding the safety, security, and trustworthiness of Generative AI (GenAI) models, practitioners, and regulators alike have pointed to AI red-teaming as a key component of their strategies for identifying and mitigating these risks. However, significant questions remain about what precisely AI red-teaming entails, what role it can play in risk identification and evaluation, and how it should be conducted to ensure valid, reliable, and actionable results. I will provide an overview of our recent work which analyzes recent cases of red-teaming activities in the AI industry and contrast it with an extensive survey of the relevant research literature to characterize the scope, structure, and criteria for AI red-teaming practices. I will situate our findings in the broader discussions surrounding the evaluation of GenAI and AI governance, and propose an agenda for future work.

Cornelia Caragea is a Professor of Computer Science and the Director of the Information Retrieval Research Laboratory at the University of Illinois Chicago (UIC). Caragea currently serves as Program Director at the National Science Foundation. Her research interests are in natural language processing, artificial intelligence, deep learning, machine learning, and information retrieval. Caragea's work has been recognized with several National Science Foundation (NSF) research awards, including the prestigious NSF CAREER award. She has published many research papers in top venues such as ACL, EMNLP, NAACL, ICML, AAAI, and IJCAI and was a program committee member for many such conferences. She reviewed for many journals including Nature, ACM TIST, JAIR, and TACL, served on many NSF review panels, and organized several workshops on scholarly big data. In 2020-21, she received the College of Engineering (COE) Research Award, which is awarded to faculty in the College of Engineering at UIC for excellent research contributions. Caragea was included on an Elsevier list of the top 2% of scientists in their fields for her single-year impact in 2020.

Language models perform well on emotion datasets but it remains unclear whether these models indeed understand emotions expressed in text or simply exploit superficial lexical cues (e.g., emotion words). In this talk, I will discuss capabilities of small and large language models to predict emotions from adversarially created emotion datasets. Our human-annotated adversarial datasets are created by iteratively rephrasing input texts to gradually remove explicit emotion cues (while preserving the semantic similarity and the emotions) until a language model yields incorrect predictions. Our analysis reveals that all models struggle to correctly predict emotions when emotion lexical cues become scarcer and scarcer, but large language models perform better than small pre-trained language models.