A multi-university team led by Somesh Jha at the University of Wisconsin-Madison has won a prestigious US Department of Defense Multidisciplinary University Research Initiative (MURI) Award. Seven universities comprise the group: University of Wisconsin-Madison, Carnegie Mellon University, University of California San Diego, Penn State, University of Melbourne, Macquarie University, and University of Newcastle. Benjamin Rubinstein of the University of Melbourne will lead the Australian team, or AUSMURI, funded by the Australian Government. The team brings together diverse expertise spanning computer security, machine learning, psychology, decision sciences, and human-computer interaction
The winning project, entitled Cohesive and Robust Human-Bot Cybersecurity Teams, aims to develop a rigorous team science for Human-Bot Cybersecurity teams (HBCT), with the goal of developing a cohesive team that is robust against active human and ML adversaries.
Cybersecurity is the most challenging task that the Department of Defense (DoD) faces today. A typical human analyst in a cybersecurity task has to deal with a plethora of information, such as intrusion logs, network flows, executables, and provenance information for files. Real-time cybersecurity scenarios are even more challenging: an active adversarial environment, with large amounts of information and techniques that neither humans nor machines can handle alone.
In addition to human analysts, machine learning (ML) bots have become part of these cybersecurity teams. ML bots reduce the burden on human analysts by filtering information, thus freeing up cognitive resources for tasks related to the high-level mission.
While we know a lot about how humans use tools to work in teams, little is known about how to manage, observe, and improve hybrid teams that are made up of humans and autonomous machines. In particular, researchers plan to study how to coordinate HBCT in the presence of active adversaries that are also adapting to changing conditions.
The research team will first focus on ways to build trust within the HBCT by investigating techniques to produce explanations for the human analysts of how the ML bots work. These explanations will be presented in an appropriate vocabulary for the analysts and will be specific to the task of the HBCT, providing valuable insight for the human analysts so they will trust the ML model and reduce manual effort.
Adversaries in mission-critical DoD scenarios can be very sophisticated, such as nation-state attackers. Existing work on designing ML models focuses on modalities, such as images and audio, rather than addressing the overall task – that the attacker is trying to thwart the mission of the human-bot team. The second approach for this project investigates robust ML techniques that focus on modalities that are relevant to cybersecurity, such as malware and network logs. While investigating these task-aware techniques, the research team will factor in the high-level mission of the cybersecurity team.
The group will also research how analysts integrate information to arrive at decisions, as well as their mental models of how bots operate. This will allow them to take a step towards automating the decision-making process. Moreover, the mental model can help design robust ML models that are more specific to the task of the HBCT.
Adaptability will be key as well. Human-bot team dynamics change as new adversaries with different capabilities arise, and adversaries adapt in response to new team strategies. Adversaries’ interactive learning must be taken into account to develop methods for the entire team to adapt to adversaries in an interactive manner.
“This project has a very broad scope and thus has team members from very diverse subject areas (formal methods, machine learning, security, and psychology),” says Principal Investigator Somesh Jha. “Having researchers from these diverse areas is essential to the success of this project.”
“Wicked problems require innovative solutions and both basic and problem-based research,” says AUSMURI Principal Investigator Benjamin Rubinstein. “The joint MURI/AUSMURI program is a significant investment in step change research. We expect a range of outcomes from this ambitious program, from human-friendly explanations of AI decisions, to new defences against attacks on machine learning, and mathematical accounts of human decision making within human-bot teams.”
Since its inception in 1985, the tri-Service MURI program has convened teams of investigators with the hope that collective insights drawn from research across multiple disciplines could facilitate the advancement of newly emerging technologies and address the DoD’s unique problem sets. Complementing the Department’s single-investigator basic research grants, the highly competitive MURI program has made immense contributions to both national defense and society at large. Innovative technological advances from the MURI program help drive and accelerate current and future military capabilities and find multiple applications in the commercial sector.
“The science and engineering challenges we face today are highly complex and often intersect more than one scientific discipline,” said Dr. Bindu Nair, director of the Basic Research Office in the Office of the Under Secretary of Defense for Research and Engineering (OUSD(R&E)). “MURIs acknowledge these complexities by supporting teams whose members have diverse sets of expertise as well as creative and different approaches to tackling problems. This cross-fertilization of ideas can accelerate research progress to enable more rapid R&D breakthroughs and hasten the transition of basic research finding to practical application. It’s a program that embodies DoD’s legacy of scientific impact.”
The Cohesive and Robust Human-Bot Cybersecurity Teams project is made up of the following institutions and people:
University of Wisconsin-Madison
Somesh Jha
Xiaojin (Jerry) Zhu
Timothy Rogers
Carnegie Mellon University
Lujo Bauer
Matt Fredrikson
Cleotilde Gonzalez
University of California San Diego
Kamalika Chaudhuri
Penn State
Patrick McDaniel
Ahmed Abdou
Blaine Hoak
Ryan Sheatsley
University of Melbourne
Benjamin Rubinstein
Vassilis Kostakos
Toby Murray
Olga Ohrimenko
Macquarie University
Dali Kaafar
Shlomo Berkovsky
University of Newcastle, Australia
Scott Brown
Ami Eidels
Somesh Jha is a Lubar Professor of Computer Science in the Department of Computer Sciences at the University of Wisconsin-Madison. Jha studies security and formal methods, particularly adversarial machine learning and privacy. He is an ACM Fellow and Distinguished Scientist, an IEEE Fellow, and is the recipient of an NSF Career Award.
Xiaojn (Jerry) Zhu is a Lubar Professor of Computer Science in the Department of Computer Sciences at UW-Madison. Zhu’s research interest is machine learning – more specifically, adversarial learning, active learning, and semi-supervised learning. He is a recipient of an NSF Award, a member of the DARPA ISAT advisory group, and has won numerous best paper awards.
Timothy Rogers is Professor of Psychology at UW-Madison. His research develops neural and computational models of human learning and memory, focusing on how people acquire everyday knowledge about the world. He also directs LUCID, a cross-disciplinary program training PhD students to pursue research at the intersection of human and machine learning and behavior.
The Department of Computer Sciences at the University of Wisconsin-Madison offers a dynamic environment for study, research, and professional growth. Our award-winning faculty continues to grow, with twelve new faculty members hired in the last two years. We are recognized as having the world’s leading research groups in many areas of computer science, and we provide an unbeatable learning environment for students at all levels, in all areas of computer science. Innovative research and teaching make UW-Madison Computer Sciences a top-ranked, top-notch department in one of the world’s premier universities.
The Computer Sciences Department is part of the newly formed School of Computer, Data & Information Sciences (CDIS) at UW-Madison. In addition to Computer Sciences, the School brings together the Information School and the Department of Statistics to leverage our strengths as a group and produce cutting-edge, transformative research, educate leaders and critical thinkers, and accelerate innovation that tackles societal issues.
By Karen Barrett-Wilt