Our research seeks to empower individuals and organizations to control how their data is used. We use techniques from cryptography, programming languages, machine learning, operating systems, and other areas to both understand and improve the security of computing as practiced today, and as envisioned in the future.

SRG
lunch
Security Research Group Lunch (5 May 2017)
Bill Young, Haina Li, Weilin Xu, Mohammad Etemad, Bargav Jayaraman, David Evans, Helen Simecek, Anant Kharkar, Darion Cassel

Everyone is welcome at our research group meetings. To get announcements, join our Slack Group (any @virginia.edu email address can join themsleves, or email me to request an invitation).

Projects

Secure Multi-Party Computation
Obliv-C · MightBeEvil
Practical Secure Computation
Web and Mobile Security
ScriptInspector · SSOScan
Adversarial Machine Learning
EvadeML
Past Projects
Side-Channel Analysis · Perracotta · Splint
N-Variant Systems · Physicrypt · Social Networking APIs

News

Feature Squeezing: Detecting Adversarial Examples

April 10th, 2017 by David Evans

Although deep neural networks (DNNs) have achieved great success in many computer vision tasks, recent studies have shown they are vulnerable to adversarial examples. Such examples, typically generated by adding small but purposeful distortions, can frequently fool DNN models. Previous studies to defend against adversarial examples mostly focused on refining the DNN models. They have either shown limited success or suffer from expensive computation. We propose a new strategy, feature squeezing, that can be used to harden DNN models by detecting adversarial examples. Feature squeezing reduces the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into a single sample.

By comparing a DNN model’s prediction on the original input with that on the squeezed input, feature squeezing detects adversarial examples with high accuracy and few false positives. If the original and squeezed examples produce substantially different outputs from the model, the input is likely to be adversarial. By measuring the disagreement among predictions and selecting a threshold value, our system outputs the correct prediction for legitimate examples and rejects adversarial inputs.

So far, we have explored two instances of feature squeezing: reducing the color bit depth of each pixel and smoothing using a spatial filter. These strategies are straightforward, inexpensive, and complementary to defensive methods that operate on the underlying model, such as adversarial training.

The figure shows the histogram of the L1 scores on the MNIST dataset between the original and squeezed sample, for 1000 non-adversarial examples as well as 1000 adversarial examples generated using both the Fast Gradient Sign Method and the Jacobian-based Saliency Map Approach. Over the full MNIST testing set, the detection accuracy is 99.74% (only 22 out of 5000 fast positives).

For more information, see the paper:

Weilin Xu, David Evans, Yanjun Qi. Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks. arXiv preprint, 4 April 2017. [PDF]

Project Site: EvadeML


Enigma 2017 Talk: Classifiers under Attack

March 6th, 2017 by David Evans

The video for my Enigma 2017 talk, “Classifiers under Attack” is now posted:



The talk focuses on work with Weilin Xu and Yanjun Qi on automatically evading malware classifiers using techniques from genetic programming. (See EvadeML.org for more details and links to code and papers, although some of the work I talked about at Enigma has not yet been published.)

Enigma was an amazing conference – one of the most worthwhile, and definitely the most diverse security/privacy conference I’ve been to in my career, both in terms of where people were coming from (nearly exactly 50% from industry and 50% from academic/government/non-profits), intellectual variety (range of talks from systems and crypto to neuroscience, law, and journalism), and the demographics of the attendees and speakers (not to mention a way-cool stage setup).

The model of having speakers do on-line practice talks with their session was also very valuable (Enigma requires speakers to agree to do three on-line practice talks sessions before the conference, and from what I hear most speakers and sessions did cooperate with this, and it showed in the quality of the sessions) and something I hope other conference will be able to adopt. You actually end up with talks that fit with each other, build of things others present, and avoid unnecessary duplication, as well as, improving all the talks by themselves.


CCS 2017

January 18th, 2017 by David Evans

I’m program co-chair, with Tal Malkin and Dongyan Xu, for ACM CCS 2017.

The conference will be in Dallas, 30 Oct – 3 Nov 2017. Paper submissions are due May 19 (8:29PM PDT). It’ll be a while before the CFP is ready, but the conference website is now up!




Growth of MPC Research

January 13th, 2017 by David Evans

I led a discussion breakout at the NSF SaTC PIs meeting on Secure Computation: Progress, Methods, Challenges, and Open Questions. To set up the session, I looked at the number of papers in Google scholar that match "secure computation" OR "multi-party computation" (which seems like a fairly good measure of research activity in the area):

There were 1800 MPC papers published in 2015! This means in one year, there are most papers published on MPC than there were from the beginning of time through the end of 2004. Gotta get reading…


Aggregating Private Sparse Learning Models Using Multi-Party Computation

December 9th, 2016 by David Evans

Bargav Jayaraman presented on privacy-preserving sparse learning at the Private Multi‑Party Machine Learning workshop attached to NIPS 2016 in Barcelona.



A short paper summarizing the work is: Lu Tian, Bargav Jayaraman, Quanquan Gu, and David Evans. Aggregating Private Sparse Learning Models Using Multi-Party Computation [PDF, 6 pages].

At the workshop, Jack Doerner also presented a talk on An Introduction to Practical Multiparty Computation.


O’Reilly Security 2016: Classifiers Under Attack

November 4th, 2016 by David Evans

I gave a talk on Weilin Xu’s work (in collaboration with Yanjun Qi) on evading machine learning classifiers at the O’Reilly Security Conference in New York: Classifiers Under Attack, 2 November 2016.

Machine-learning models are popular in security tasks such as malware detection, network intrusion detection, and spam detection. These models can achieve extremely high accuracy on test datasets and are widely used in practice.

However, these results are for particular test datasets. Unlike other fields, security tasks involve adversaries responding to the classifier. For example, attackers may try to generate new malware deliberately designed to evade existing classifiers. This breaks the assumption of machine-learning models that the training data and the operational data share the same data distribution. As a result, it is important to consider attackers’ efforts to disrupt or evade the generated models.

David Evans provides an introduction to the techniques adversaries use to circumvent machine-learning classifiers and presents case studies of machine classifiers under attack. David then outlines methods for automatically predicting the robustness of a classifier when used in an adversarial context and techniques that may be used to harden a classifier to decrease its vulnerability to attackers.




Demystifying the Blockchain Hype

October 26th, 2016 by David Evans

I gave a talk introducing the blockchain at a meetup hosted by Willow Tree Apps:
Demystifying the Blockchain Hype, 25 October 2016.

Over the past few years, explosive growth in cryptocurrencies (especially Bitcoin), has led to tremendous excitement about blockchains as a powerful tool for just about everything. Without assuming anyprevious background in cryptography, I’ll explain the cryptographic and networking technologies that make blockchains possible, explain why people are so excited about blockchains, but why you shouldn’t believe everything you hear about them.

The slides are below (I believe a recording will also be available soon).




Secure Stable Matching at Scale

August 30th, 2016 by David Evans

Our paper on secure stable matching is now available [PDF, 12 pages]:

Jack Doerner, David Evans, abhi shelat. Secure Stable Matching at Scale. 23rd ACM Conference on Computer and Communications Security (CCS). Vienna, Austria. 24-28 October 2016.

See the OblivC.org site for the code and data. Jack Doerner will present the paper at CCS in October.


Abstract

When a group of individuals and organizations wish to compute a stable matching — for example, when medical students are matched to medical residency programs — they often outsource the computation to a trusted arbiter to preserve the privacy of participants’ preference rankings. Secure multi-party computation presents an alternative that offers the possibility of private matching processes that do not rely on any common trusted third party. However, stable matching algorithms are computationally intensive and involve complex data-dependent memory access patterns, so they have previously been considered infeasible for execution in a secure multiparty context on non-trivial inputs.

We adapt the classic Gale-Shapley algorithm for use in such a context, and show experimentally that our modifications yield a lower asymptotic complexity and more than an order of magnitude in practical cost improvement over previous techniques. Our main insights are to design new oblivious data structures that exploit the properties of the matching algorithms. We then apply our secure computation techniques to the instability chaining algorithm of Roth and Peranson, currently in use by the National Resident Matching Program. The resulting algorithm is efficient enough to be useful at the scale required for matching medical residents nationwide, taking just over 17 hours to complete an execution simulating the 2016 NRMP match with more than 35,000 participants and 30,000 residency slots.


FTC Visit

August 18th, 2016 by David Evans

Great to visit our former student Joseph Calandrino at the Federal Trade Commission in DC, where he is now a Research Director.

Denis Nekipelov and I gave a joint talk there about using secure multi-party computation techniques to enable data analyses across sensitive, divided data sets in the room where the FTC commissioners meet.



Denis Nekipelov, Joseph Calandrino, David Evans, Devesh Ravel


Private Multi‑Party Machine Learning

August 18th, 2016 by David Evans

I’m co-organizing a workshop to be held in conjunction with NIPS on Private Multi‑Party Machine Learning, along with Borja Balle, Aurélien Bellet, Adrià Gascón. The one-day workshop will be held Dec 9 or Dec 10 in Barcelona.

NIPS workshops are different from typical workshops attached to computer security conferences, with lots of invited talks (and we have some great speakers lined up for PMPML16), but there is also an opportunity for researchers to submit short papers to be presented at the workshop either as short talks or posters.