[SOUND]. In this video, we're going to look at a case study of phishing warnings. This case study is based on the assigned reading that you'll see linked on this weeks Coursera lecture for the paper titled, You've Been Warned, An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. Before you watch this video, you should read the paper. It's a great piece of HCI research exactly about security issues, and once you've read it, you'll have a better understanding of what we talk about in this video. So, lets start by talking about what phishing is. Phishing is a practice where websites pretend to be a different site that a user trusts in order to get them to enter sensitive information like login credentials or financial information. Web browsers are able to detect this. So, if I enter the URL for a phishing website, the browser can bring up a message that says, this page is a reported web forgery. It gives me a security message, this one is in Firefox, and it says this webpage at www.itsatrap.org has been reported as a forgery and has been blocked based on your security preferences. It then goes on to explain to the user that web forgeries are designed to trick you into revealing personal or financial information by imitating sources you may trust. Entering any information on this web page may result in identity theft or other fraud. We then have buttons to get me out of here, go somewhere safe, get more information about why the page was blocked, or down in the corner is a small link to ignore the warning. If we were to click on that, fortunately, this is an example page that isn't actually phishing for us, and we get brought to a more explanatory page at Firefox that says what's going on here. You can also see that Firefox has added a red bar across the top, because even though this website isn't really phishing, it's there to serve as an example, and it's on the list of blocked sites. So, users are warned the entire time they're interacting with a forged site that there is danger involved. >> In this study, the goal was to see how effective these different web browser phishing warnings were. Now, if you remember from our previous lecture, we measure usability in five main ways. Speed, efficiency, learnability, memorability and user preference. Now, when we are looking at security websites and security issues, we often measure how secure a user's behavior is, and that's not a thing on this list. We're not including user secure behavior on the list of usability. So, how do we reconcile those things? Well, if you think about it, taking insecure behavior, for example, entering information into a phishing website, is essentially a mistake the user's making and it's a mistake that could be prevented by the interface. That's what efficiency measures. So, essentially, when we're looking for security mistakes that users make, we're measuring the efficiency of the security features of a website or application. This study created two phishing websites, one that mimicked Amazon, and one that mimicked eBay. We're looking at the actual sites here, but the ones that they created had slightly different URLs, and they registered them so the browsers would know that they were phishing websites and try to block them. They didn't actually collect any personal information from the users, but they had the ability for users to enter their log in credentials. The researchers wanted to see how effective each of the different browser phishing warnings were in preventing users from entering their personal information. The browser warnings that they looked at were the following. The first is Internet Explorer's active phishing warning. This is called active because the actual site you're going to doesn't come up. Instead, you get this page that only has the warning appearing on it. Next is Internet Explorer's passive phishing warning. This is passive because the page actually loads in the background, and instead, you just have this small pop-up window that says it's a suspicious website. The third browser option they looked at was Firefox's phishing warning. This is a little bit different than the one we just saw in the example because they had an older version of Firefox, but you can see that the text is similar to what we saw in our phishing example. The page shows up, but it's grayed out in the background, and the main thing you see on the page is this window that says, you're looking at a suspected web forgery. Again, you should have read the paper and I'd suggest you look closely at the text and appearance of all of these windows, because it will give you a good idea about what's going on with each of the warnings. The study was then relatively straightforward. People were put into each of these four groups. They looked at the Firefox phishing warning, the active one from Internet Explorer, the passive one from Internet Explorer, and there was a control group who used a browser that didn't show any phishing warnings at all. The participants in the experiment received e-mails that pretended to be from either Amazon or eBay that told them they had to log in or else an order will be canceled or that something will happen in their account. The numbers that we're looking at here indicate what percentage of users clicked on the link in the e-mail, and what percentage of users actually entered information into the phishing website and submitted it. If Firefox, 100% of the users clicked on the link, but zero of them actually entered information into the phishing website, which indicates that Firefox did a great job of explaining to users that something wrong was happening, and that they shouldn't enter any information. Internet Explorer's active phishing warning was the next best option. 95% of those users clicked on the link, but only 45% of them actually entered information and submitted it to the phishing website. Now, that's a lot worse than Firefox, but it's much better than the other two options. In both of those cases, 90% of people entered phishing information into the websites. That means Internet Explorer's passive phishing warning was no more effective than having no warning at all. Now, these are pretty interesting results, and this is a great example of a nice empirical study analyzing the effectiveness of these different tools. Essentially, it's analyzing their usability to keep users from making security errors. But another great thing about this paper is how they linked this to mental model. If you remember the first week of class, we talked about the importance of mental models to HCI, and we looked at some traditional psychological mental models. In this paper, the researchers actually pulled mental models from the community of people who study hazards, and what we see on the slide here are some facets of this CHI-P mental model that they use. This looks at how people pay attention to and react to warnings about hazards. It includes things like how to get their attention and hold it, how to make sure they comprehend the warning and remember it, what are their attitudes, beliefs and feelings about it, if they're motivated to respond to the warning, if they actually perform the actions that they should, and how other environmental stimuli change how they behave. The researchers do a great job in this paper of looking at the presentation of these different phishing warnings and connecting it to this mental model. The conclusions that we can draw from this are first, that the interface can have measurable impacts on the usability of security features. In this case, we saw that, for example, Firefox's phishing warning was far more effective than any of the other ones, and this shows us that better interfaces lead to more secure behavior. We have an empirical study that proves that, and we have a nice tie-in that shows for mental models, that active warnings are able to capture and hold more attention than the passive ones, and thus, they yield better results. In short, user behavior is more secure
