What's a realistic phishing test failure rate?


#1

I’ve looked for and was unable to find metrics related to phishing tests. Is anyone out there willing to share their acceptable failure rate goal and actual value?

We have sent out two phishing tests and each had a 10% failure rate. I’d like to see it get down to 1% or less. Is that realistic?


Free phishing software for organizations of 500 employees or less
#2

If we have a single user fail our phishing tests we consider that a failure. It only takes one to open pandora’s box!


(BJ Beier) #3

I think it’s very difficult to have a simple, single answer for this. IMHO, I think that any failure rate is unacceptable, as @polden mentions, it only takes one person to unleash ransomware on you network. However, If you have only just started your phishing training, you will see the first few campaigns higher than the rest. It should get better over time, but the phishing emails will evolve over time as well. I typically look for employees who have a high phish prone rate from repeat clicks and beef up their training. If you have a 0% failure on a campaign, it was most likely to easy.


#4

We have been using KnowBe4 for approximately 8 months now. Our initial phish prone rate was 33% in a 300+ employee company. Yes, I know - ouch! Our “tech-savvy” employees were appalled that I was making them take the Kevin Mitnick training courses. After the initial moaning and groaning, many of the employees commented how well the training was - kudos to KnowBe4.
Anyway, we are currently running approximately 4% phish prone hit rate - using the most difficult templates and some custom ones as well. Much better, but not great.
Will we ever get to that magical 0% hit rate - I doubt it.


(Sean Coyle) #5

1%, I would consider that lucky. We develop our phishing campaigns intentionally to catch / ensnare our users. As such we see higher failure rates but Ideally we catch them in this teachable moment before the bad guys do.

Our rates for harder campaigns have been upwards of 20% after running these tests for close to a Year now.

I’d be happy if i saw this drop to 5%.


#6

We had around 8% failure initially, still working on gettin git down to 0


#7

Thanks for your feedback everyone. It’s nice to know we are having similar experiences compared to other companies just starting to test their employees.

For those of you who said their goal is “0”…
While the security guy in me agrees 100%, the guy who’s been it I.T. for more than 20 years doesn’t think it’s realistic. The reason why is, we are all human. We see something shiny and we want it. We see a good deal, the new high definition TV, someone is going to click. Puppy pictures, common on… I am not saying we shouldn’t expect the most from our users but we do need to be real.

What is probably as important, if not more important, is what does that person do after they realize they have just been had? Do they immediately call the Help Desk? Do they try to hide the fact they got infected and limp along until their system doesn’t function anymore? Do they get ransomeware and then go on vacation without telling anyone until they return three weeks later? (Yes, this happened).

I think a realistic goal is 1% or less and the expectation is the training makes them realize how important it is to remedy the situation as quickly as possible when they do.


(Susan Bernard) #8

I love your response, Tony. There is no such thing as 100% security. Humans are humans and will make errors, even the best and brightest. That is why I would never set a goal of a 100% passing rate on any social engineering test. The human aspect always leaves room for some margin of error.

The goal doesn’t necessarily have to be a number. Instead, let the numbers guide your training strategy by ensuring everyone is educated all the time and that complacency becomes obsolete in your organization.


(Kelly Murphy) #9

We run about 1 - 3 percent, but it takes time to get there, management support, constant training and phishing campaigns. Look at it like any risk remediation effort. If you would have had 50 fails in the past and only 10 fails recently, you’ve reduced your phishing risk by 80%. If you have other security layers in place such as a good patching program, IPS, anti-malware, etc., you can further mitigate your risk to acceptable levels.

What has helped us the most is that executive management has added a metric to each employees scorecard for number of phishing fails. It counts for 10% of their annual review.


(Sarah Cuny) #10

Now that’s executive buy in! Interesting idea, including an employee’s phish rate with their annual review.


(Don Pak) #11

Our baseline test was a 5.9%. Our second campaign resulted in a 2% and our latest one was 1.9%. I agree with others in this forum…having just one person click on a link is too much since employees are the weakest link in security. I know having a 0% click is possible but not probable.

Even before we implemented KnowBe4, I had semi annual classes and monthly emails about not clicking on unsolicited emails. However, we would still get virus alerts on a monthly basis related to opening or clicking on email links. The online training from KnowBe4 is instrumental in our ongoing cyber security training for our employees.


(Dave Hausmann) #12

I agree we should be aiming for 0% and that 0% is not going to happen. Since we started using KnowBe4 we have seen a decrease in our failure rate which is a plus. I have been able to pinpoint the big problem people and worked on them more. The issue I see is a chunk of our users either do not use email (even though it is setup), or use it so little they miss the tests but they click on everything. I do like linking the users failure rate to their review but in my environment that would be unfair.


(Mike Zirbes) #13

My initial failure late was about 6%-8% two years ago when I started with KnowBe4. Now my rate is below 1% and usually the users never click on the message, maybe the will open it, those users are usually the executives! I have them trained/scared cause they know I am testing. It does cause help desk calls sometimes due to having to tell them what is “OK” to click. But that’s better than click happy users.

The training works. I have social engineered them into being cautious. .


(mike holmes) #14

My initial failure rate was 40% and now after the training video and bi-monthly phish testing my users are under 10% for failure rate. Campaign feature to ‘add’ new enrolled users into the training is awesome!


(Josh Sanderson) #15

While 100% pass rate would be nice I don’t think I would sleep much better at night. There is always a new and ingenious email that just might catch even the most wary user unaware.

The human element is just one security control (albeit a major one) amongst a long list of layered security controls. That 1% that get ransomware should only have access to a limited portion of file shares due to least privilege. Their encrypted files should be easily restored due to a strong backup and restore process and on and on…


(Mike Wright) #16

We implemented a passphrase that was to be included in any internal email that had an attachment or link. If a user received an email from a coworker, with a link or attachment, but missing the passphrase they were not to open it. The passphrase was posted on our intranet and changed periodically. Seemed to help.


#17

That’s an interesting concept. I would make one recommendation though. I personally would not post any kind of passphrase or code on an intranet. If a hacker was able to get into your network with out your knowing they could send out e-mails with that passphrase and you’d be sunk. A better option would be to use a cypher that used a something all users would know and give it to the users in meetings. A simple cypher could be say the day of the year +/- some number or something similar.


(Eduardo R Ortiz) #18

We just baselined this week and are sitting at 12% clickers, but also 3% data entry. For me, the 3% number can be down (and should be ) to near zero levels. I did measure how many calls were received at our help desk and I personally call to thank those that call to notify of the phishing. I am always interested in the behaviors after the fact and trying to eliminate the old “fear culture” that was here before my time since that is a very dangerous factor affecting human behavior and reaction.


(Mike Wright) #19

That’s a good point. Thanks.


(Mel Green) #20

We baselined two months ago at 11%, had everyone do the training and our first test we had 0% clickers. That being said, that first real test was on the easy end of the spectrum. My plan is to slowly ramp up the difficulty and see where our break points are…

I do feel like having them do the 45 minute training was beneficial - I think it showed end users that we (both IT and management) are really serious about the process.