It’s also such a dumb metric because most of people’s jobs are to click on links elsewhere on the internet, yet when it’s in an email, it’s bad? Unless you’re running an old browser or there is a 0 day, simply opening a link isn’t going to hack your system, but further actions by the user would need to be taken to be compromised. These simulations don’t account for that.
Totally agreed, I get it’s easier to consider it a fail if you open the link, and that simply opening a random link has some inherent risk, but there should at least be a fake page to enter credentials and evaluate how many people actually go through with that, and break that out as a CRITICAL where the other clicks are HIGH or MEDIUM status, to classify the risk.
Also, this is just an anecdote, but in a similar phishing simulation i helped with, we had to bypass filters for rejecting emails with links for websites registered in the last 60 days. Obviously this isn’t a foolproof way to prevent phishing attempts, but it does cut out a lot of junk, and we’ve indirectly been training employees to not deal with that.
Abstract from the paper itself:
This paper empirically evaluates the efficacy of two ubiquitous forms of enterprise security training: annual cybersecurity awareness training and embedded anti-phishing training exercises. Specifically, our work analyzes the results of an 8-month randomized controlled experiment involving ten simulated phishing campaigns sent to over 19,500 employees at a large healthcare organization. Our results suggest that these efforts offer limited value. First, we find no significant relationship between whether users have recently completed cybersecurity awareness training and their likelihood of failing a phishing simulation. Second, when evaluating recipients of embedded phishing training, we find that the absolute difference in failure rates between trained and untrained users is extremely low across a variety of training content. Third, we observe that most users spend minimal time interacting with embedded phishing training material in-the-wild; and that for specific types of training content, users who receive and complete more instances of the training can have an increased likelihood of failing subsequent phishing simulations. Taken together, our results suggest that anti-phishing training programs, in their current and commonly deployed forms, are unlikely to offer significant practical value in reducing phishing risks.
And the methodology:
Our study analyzes the performance of nearly 20,000 full-time employees at UCSD Health across eight months of simulated phishing campaigns sent between January 2023 and October 2023. UCSD Health is a major medical center that is part of a large research university, whose employees span a variety of medical roles (e.g., doctors and nurses) as well as a diverse array of “traditional” enterprise jobs such as financial, HR, IT, and administrative staff. For their email infrastructure, UCSD Health exclusively uses Microsoft Office 365 with mail forwarding disabled. On roughly one day per month, UCSD Health sent out a simulated phishing campaign, where each campaign contained one to four distinct phishing email messages depending on the month. Each user received only one of the campaign’s phishing messages per month, where the exact message depended on the group the user was randomly assigned to at the beginning of the study (§ 3.1). In total these campaigns involved ten unique phishing email messages spanning a variety of deceptive narratives (“lures”) described in Section 3.2. All of the phishing lures focused on drive-by-download or credential phishing attacks, where a user failed the phishing simulation if they clicked on the embedded phishing link.
I still personally benefit from the social engineering resistance training I’ve had over the years to this day though.
Me too, I use it to get out of situations I don’t want to deal with. “Ohh you’re calling me asking for PII? Sorry, i can’t provide that information unless I initiate the conversation. I’ll call the number I have on file for you to provide that.” <hangs up and never follows up>
You’re going to go to cast iron hell for this
How else can you get reoccurring revenue through a subscription if you can’t remotely brick the device someone paid $2700 for?
Wait, has the guy in the meme been Charlie Kirk the whole time??
Unless the email client is blocking external images, a tracking pixel in the email would be enough to see that the email was rendered, and that the address is valid. The trainings specifically instruct you to review the contents of the email and check the email headers before clicking links, so that alone would confirm to a spammer that the email is valid.