Getting the most value from phishing assessments with the Phishing Assessment Optimizer

Introduction all right hello and welcome everyone i'm scott wright ceo of click armor and uh i help businesses reduce losses from things like phishing and social engineering scams ransomware and even non-compliances so we're gonna talk today about getting the most value from phishing assessments you know the things where your it team sends out these fake messages uh trying to trick people right into clicking on links or attachments and it sometimes can be annoying but there's actually a reason why they do this and if you're an i.t manager or an executive on this session you'll understand what the reasons are but i'm going to dig a little deeper uh in this session based on my experience from years of conducting phishing assessments to try and get a deeper view of whether or not you're getting value on these simulations and so i've developed something called the phishing assessment optimizer we're going to go through it and hopefully you'll learn some cool things and at the end if you stay to the end you'll get a interesting view on what i think we can do to do a much better job of getting value from these initiatives for trying to assess people's uh awareness of phishing and improve uh how they're actually able to handle phishing messages so stick around i really would love to get your input uh on the session see if it provides value to you but um to start with what i want to do is a check in and see who is joining us from Meet the speakers around the world so right now i've got um let's see sue from halifax uh dave from australia awesome and a couple from ottawa matt and uh eli awesome so i'm gonna just uh go through a few of the preliminary things that i need people to understand Background a little bit about some of the background why i think it's important for someone like me to share these things because i see that not a lot of people are asking questions that would enable people to discover whether they're getting value out of their phishing assessments or not so i'm going to take you back to 2009 2008-2009-2000 uh when i had been a security engineer doing security risk assessments and uh lots of different security projects big it projects and i saw that there was actually a need for people to focus more on security awareness but we really didn't treat it the same way on the human side as we did on the technical side on the technical side we could do all kinds of network tests getting data and we could actually discover what the risks were to an organization or a system based on some of the aspects of it like what's the asset value what are the things that we're trying to protect and what can we lose if they actually get compromised and then there's the threats which are the actors bad guys that are trying to get to those things steal them damage them corrupt them commit fraud etc and in the middle is what a lot of people don't understand it's it's the vulnerability aspect of it and the things that people actually need to consider as to what is preventing the threat like the thief from accessing the assets so think of it in your home security system for example you've got your electronics your ipad your your tvs etc and those are a value and if you lost them you'd be suffering a financial loss but also uh the threat who is the the thief the bad guy trying to break into your house uh knows that those things are there and is aiming to get them the only thing stopping him presumably is your door with a lock on it now if the door wasn't locked then that's a high vulnerability so whenever we get a larger amount of vulnerability we increase the risk and we can measure that with a lot of data and models on the technical side but we don't often do that on the human side and so to try and get a metric around the vulnerability of humans is really important uh in order for us to be able to manage risks and and that really risk management people don't really understand that well sometimes and what that really just means is prioritizing what would be the next thing i would do to get a uh if i had money to spend on security what would be the next thing i would do to improve security for our systems and so when you can categorize the vulnerabilities uh better then you can better understand where you're vulnerable and what you need to uh invest in for security so back in 2010 there was almost nothing in terms of metrics around the vulnerability of humans to you know stop an attacker who is trying to launch malware on your computer ransomware didn't even exist back then and so um there was a lot of data and so i started doing something called the honeystick project which i talked about in my last webinar on becoming a security champion um it was really just a project of dropping usb drives uh and eventually smartphones with some uh measurements capability on them to see what people would do with them and that gave me some data to talk about around the vulnerability of humans and around the same time maybe a little bit later these phishing assessments came out from a number of big vendors you know no before proof point well actually time it was wombat co-fence sands so there were a lot of these tools that came out and they were using similar actual technology and mechanisms to what i was using in the honey stick project so it actually made my job a lot easier to use these tools and i used them for quite a while um but i also recognize what the value was in in doing these things how the data could be used um and along the way though i'm i found some issues um it's important though to actually look Measuring human vulnerabilities at what are the things we're trying to do so as i said we're trying to measure human vulnerabilities we need to be able to understand how easily can somebody be tricked by an attacker and the easier it is the more we could expect to lose in an attack so we need to try and measure and improve people's ability to stop these things next the ability to have some kind of a Teachable moments teachable moment it was really attractive with phishing assessments or simulations these fake emails sent by the it team to be able to measure what uh when when sorry when somebody would open a a a link in an email they would actually get sent to a landing page set up by the it teams saying hey this was a test uh no don't worry there's no ransomware but you really should have spotted these things so it is a teachable moment i was going to put you know a president looking at the eclipse here but i thought i might get hit with a a copyright violation i think we all understand right so sometimes it takes a bit of a shock to teach people something and they'll remember it and that was something that early on with phishing assessments we recognized it could be a teachable moment we could get people to remember uh when they clicked on you know an impersonation email from the help desk um and it really wasn't and they learned some things that they should uh do differently next time so that was something we were trying to get to was the teachable moments in addition to being able to measure their vulnerability but there are a couple of other things that people really didn't Business productivity spend a lot of time thinking about except it was inherently somewhat obvious uh but it's important to understand the impact sometimes and so business productivity is one of them both from the point of view of the it team who's deploying these things how much time does it take them they're already busy and stressed out how easy is it for them to run these tests and make use of them and what we found with phishing assessments is actually really easy to run really easy to get some data because it's every time somebody clicks on a link they actually get some feedback and they they get the data and they expect people will actually learn from the types of things that they're clicking on and the landing pages that they get to so when we see the productivity impact for it people we see okay well if it's easy for them to do then yeah let's look at close closer more closely to uh doing phishing assessments using these tools and we also have to take into account how does it impact the productivity of the whole team so everybody that's getting these messages has to spend a bit of time analyzing it and responding to it and you know it seems really easy and nice in the traditional phishing simulations that all these messages just come into people's inboxes they don't have to take time and go do training somewhere for an hour and then come back and get back to their job change context so the the productivity aspect of phishing assessments actually seemed pretty attractive because it was easy for the it team it was easy for people to kind of just deal with uh incoming message and get on with their work and then after that we also have to consider what are the cultural repercussions and we Cultural repercussions really didn't see this initially but i actually saw it you know in some of my early projects that i did with customers on fishing assessments and to give an example um there's a few here that have come up in the last few years so godaddy last year got in the news because they used a holiday bonus as a an incentive for you know trying to get people to click on these links and uh of course during covid that's seen as kind of insensitive there's a lot of backlash people went on social media and it was a pr disaster really similarly the chicago tribune did the same kind of thing they offered ten thousand dollar bonuses and again people were enraged um and a more recent one is the icf next it's a marketing company they promised vaccinations uh in their phishing test messages and that was actually not great either people were feeling that was really insensitive you know bad taste um etc so you know the feedback coming from people was you know geez are these things really you know a problem now and interestingly one of the reporters went and talked to one of the vendors and said well what about these things your customers are are doing these things and it's causing them all kinds of pain and pr disasters and the best the vendor could say was well you have to be really careful about what kind of messages you actually put in these things you know otherwise you know you could get some uh blowback so trying to minimize the uh the real risks here but it really is a problem and we're going to see more of these um going forward so i'm going to go a little deeper into what causes these things and how we're going to try to address them Phishing Assessment Optimizer so i created this uh checklist called the phishing assessment optimizer and i just wanted to collect all the lessons learned whether it was the kinds of things that uh the it manager collecting and running these set tests could do to do a better job or uh to try and communicate better with people and and help set their expectations and make sure that we're not causing um big cultural disruptions so lots of things to take into account here so i'm going to do a really quick poll i just wanted Poll to see if people kind of get an idea how many problems we're actually talking about so i'm just going to launch the poll and see what our respondents say uh give them a few seconds to come back i'm giving options of uh you know five major pitfalls eight eleven or fifteen and so okay i'm going to uh yeah uh end the poll and of course you know the answer is always c right it's the third one so it's actually it's going to be 11. so good guess for for those of you who are on there um so yeah it is 11 pitfalls we're going to talk about in fact Pitfalls there's maybe a couple more that um i didn't put on this list but are worth talking about so one of them um i'm going to go through each one of these in a little bit of detail not too much we're going to skim through them you'll get the idea they're not that hard to really understand but when you look at it there's a lot of things we've got to think about so we'll start and see Unpredictable employee actions how it goes so number one is the unpredictable employee actions so what that means is you know kind of like what we saw with people going to twitter and and social media when these companies were sending stuff in in those cases they were you know really unfair um things that people were accusing the companies of being you know insensitive etc but it could be for any reason it Technical penetration tests could be just some sensational piece of news that people think oh man i gotta tell more people about this because the message could be impersonating somebody outside the company maybe they're reading it on a business computer but actually thinking that it came from somewhere outside so it doesn't matter if they go post on twitter about it because it's nothing to do with work so there are a lot of reasons why people might take some unpredictable action now it's kind of interesting because when you if you're in the security world and you think about technical penetration tests you know you usually hire a consultant or or a specialist in the organization to come and do a test and there's always got to be this agreement that says what could go wrong and i'm going to you know absolve the contractor or the tester of any responsibility if something does go wrong so we can pretty easily identify you know systems could go down databases could get corrupted but there's some finite amount of impact that can typically be expected from a technical penetration test that's not really true on the testing of humans because you really don't know what they might do with you know in a reaction to one of the messages that you set so the the guidance here is you know discuss the expected and potential results of each test message and the risks um that come with them with management to make sure that they're you know not going to cause some weird kind of fallout um an example that i actually had when i was doing the honeystick project it's very similar though is um i was hired by an organization to do a drop of you know 50 usb drives and we sort of set it all up i said you know does everybody know in the organization what they should be doing if they find something like this have they been trained on it and especially the help desk because if the help desk gets somebody bringing this to them what will they do with them well they're supposed to do this and this and this and it will go up the chain and it will get to the manager who's actually responsible for the test so it should all be good well first thing that happened was over the first day or two one help desk guy saw three of these devices and they all had files on them called you know confidential budget uh plans or something for the organization and so instead of escalating it up the chain in the help desk they actually went to one of the executives in the organization who wasn't aware of the test and so we saw some fallout come down to the manager uh that hired me to do the job and a little bit of um effort was uh and maybe some credibility was uh expended uh in trying to explain what was going on here so just that kind of thing you just want to be able to try and anticipate what are the things that could come out of it number two is what i call the impossibly difficult test impossibly difficult test messages messages you know as security people and i t people in general you know we we think we're pretty smart and we like to try and show people how smart we are and sometimes it bleeds into this kind of stuff for developing test messages and you say well you know what i could do a really tough one and and i think i could catch 30 or 50 maybe all of everybody will fall for this um that's not really the point the point is not to show people that we're smarting them smarter than them it's to leave fair clues and give people a chance to exercise their skills presumably that you've taught them uh how to analyze and and respond to so you shouldn't be having too much fun trying to trick users it's actually kind of difficult to create the perfect message that is not just something that will challenge them but something that is fair number three is embarrassing your employees so embarrassing your employees there are cases where sometimes a phishing message might uh be a little bit over the top you know you see lots of spam telling people that you know they've been hacked and stuff like that but not everybody understands you know that um these are potentially just uh extortion attempts and so when you have a test like this and maybe a boss clicks on one of these really kind of salacious kind of email messages and you wonder gee what would uh that person be doing you know clicking on or or being involved in that kind of thing so you really want to steer away from things that get into personal habits and stuff like that it's kind of dangerous to um get people uh so to be in a vulnerable situation you don't want to be leading them too much so you have to be careful to keep within confidentiality those bounds number four is confidentiality so when you think about it every time somebody clicks on a link the raw data that we collect um about you know the fact that that person clicked on a link in that message about that particular topic or that particular kind of threat it's really personal information because uh it's like part of your hr file you know the fact that you have this vulnerability and it could even be of value to attackers so it's important to try and treat the raw data with high confidentiality uh safeguards so that you don't have that kind of risk there's really no reason for very many people at all to have access to the raw data what we do want to have is summary data that can tell people how the organization is doing and there is a need sometimes for remediation to actually uh approach people and tell people you know what um this is something you've got to do or maybe we want to change the program so that it um educates people better on some of these things but it's important to have that confidentiality expectation um there's the issue of people you know feeling targeted and and feeling like they are being you know victimized and so we want to actually maybe consult the hr or legal department early on so that we get their buy-in and say here's why we're doing these tests and what we're actually able to benefit from them so the hr and legal when they buy and they understand okay there could be some pushback there could be some complaints but we want to get the guidance from them on what will prevent it from becoming a real legal issue um and most of all we want to prevent it from doing you know damage to the corporate culture you know the bigger kinds of pr nightmares as well we don't want people to be reacting badly to um Spam filters number six is the unpredictable spam filters and settings so this is a real tricky one because there are so many different kinds of firewalls and gateways and filtering systems out there that um sometimes you're not really sure if people actually saw the message um mostly we expect that you know that you can train a firewall with a white list and say okay well so all the message coming all the messages coming from this uh phishing testing platform or vendor are white listed so they should all get through well that doesn't doesn't always happen i've been to situations where um on the same day one message the same message goes to different people and one of them got blocked and the other one didn't so the white listing issue is is a really big challenge especially in areas where you have potentially very dynamic rules and so you might have firewalls that are just looking for certain kinds of content and it might vary from day to day based on rules it gets from the vendor for example so it can be really hard to test and make sure that any message will not get filtered uh so these are all kinds of things we have to take into account and there's also the case of you know you can think well sometimes people may not see the message if it does go into a filter what are we going to do about it and there are you know ways that you can say you're normalizing and sort of compensating for the people who don't see the message so you might say well we can detect if people open a message and if they open it then we want to see how many of the people who open the message actually click and and that can be helpful but it also could be that people might be turning off their message or their image previewing and image previewing is what actually allows the product to track the uh opening of the email so if you have an email with an image in it and the email client opens the email and it tries to get the image the image could be linked to a website and when it loads that image then you know that that message has been opened regardless of whether people click on the link or attachment in it at all and so that's how we measure open rates and so you can do some of that normalization based on open rates but if people have the image pre-loading turned off then that skews the data and if they have it turned off and then they get the message and then they click on it well they're get you're getting a click from one that you thought you weren't actually gonna get you know that was open i guess you can say well it was obviously open because they clicked on it but it still is pretty unreliable data and if you're normalizing the data only because uh you think maybe they didn't see it there are a lot of other potential variables that you know are in this list as well so normalization is is a big challenge really um the best you can do really is put disclaimers on the baseline and trend data regarding the variables that um are involved so that when it gets reported it's not like this is high precision data that says this is how we're doing and from month to month we're up 0.5 that's not going to be possible so people have to understand what kind of data impacts you can have Level of difficulty another thing that can really impact the data is the level of difficulty in each test message so what you really want ultimately the management wants to be able to look at a trend and say how are we doing this month or this time around versus last time and to be able to do that you have to have some controls you can't have too many variables and the easiest thing that gets varied is the difficulty of the actual message so in this example i'm using you know one of the typical easy easy messages that you see um you just look at and say i know that's uh suspicious the one on the right is one kind of like the first one i talked about the impossibly difficult one i was actually in a business networking group one time and during security awareness month i was offered to try and fish people in the group and so a bunch of them volunteered and i knew they were all pretty active in business so i actually only had to create this one message and it was really just a bit of a you know salacious gossipy kind of email saying you know we but it was done professionally it looked really real and uh even the person sending it you know was a real person it just wasn't their their email address and so uh the the result was many people clicked on this because it looked so real and it was so shocking you know the fact that somebody's going to be disqualified from the business woman of the year awards so just to show you know you can have a wide range of difficulty that will really impact how many people will click on the message so you have to try and keep the difficulty consistent to have usable data now one of the things that um i really Hot button impersonation have found that makes it difficult and and makes the data less and less useful is what i call hot button impersonation so you can have an area of the organization where you're forbidden from impersonating you know the payroll department or the help desk so um just a quick story is you know doing work for government of canada at one time a department i was i was doing work for wanted me to you know ignore the recommendations that i make about uh keeping trends uh sort of consistent and having some common uh fair types of attacks and they they said come up with something more difficult that will cause a high click rate and then we can use that for pr and i guess that's fair as long as you don't consider that to be something that's in part of your trend um but so the first thing i suggested was well let's do one on the phoenix payroll system which was a notoriously um dangerous payroll system dangerous in the sense that everybody was afraid whenever you saw a notice about phoenix if it affected you it meant you might not be getting paid this month um there were just so many bugs in the system so as soon as i proposed a draft a message that you know invoked phoenix they said oh no no you can't send anything from that it'll overwhelm the payroll group and you can see where this is going right so you can't overwhelm the help desk because they have operational things and if they get overwhelmed then that's going to hit our productivity and you know obligations so you start to see areas of the company that get sort of cordoned off from these tests and it makes the test scope smaller and smaller so it becomes much less useful uh as a test and you know the attackers are going to be able to guess what those hot button areas are so you really should try to make them in scope but on the other hand recognize that management may just not allow you to to test with those particular areas so number nine we're moving along um is the Curious employees curious and rebellious employees so you know what i've seen sometimes is people don't care you know they're we'll talk about remediation probably at some point um but when people click uh on a link and they get the message saying um hey you know what this is uh bad and you should have done this um there are the people who start to get really curious the next time they see one they'll say i wonder if this is one too right so then they click on it and they find out if it's real or not and sometimes they will actually send out warnings or tell all their friends and and that'll start to skew the data because people will just be clicking on purpose or what the ones that do click on purpose and learn will probably cause a whole bunch of people not to click so you're gonna get a big variation in who will be clicking and and not for for reasons you don't even know so really it's important to try and discourage this you know communicate with people let them know if they do encounter a phishing message don't communicate with everybody it hurts the organization if we can't get a valid measure so you also want to communicate the aggregate results as soon as possible after a live test not immediately after necessarily because you don't want people to [Music] necessarily skew your data and say oh let's let's do more right but you really want to get people informed within a few days to say hey we had a campaign here's how we did um that can try that can help people understand the value to the organization Handling guidelines um so the tenth one i'm going to talk about is a lack of handling guidelines it's you know sometimes unfair and and organizations do this all the time especially with their first baseline test to um not even give any guidance to say you know what are you supposed to do to analyze and spot a phishing message and what should you do if you find it or if you suspect that a message could be a phishing message so other than having the guidelines on how to spot it the responses are often you know to use the fish hook tool or some kind of reporting tool in your email client to report suspected phishing messages and that will help us in a way understand the people who do know how to analyze these things and we also want to uh make sure people understand that the level of um threat needs to be somewhat uh judged by the receiver to say does this really constitute a phishing message or is it just regular spam because a lot of the it teams i've talked to don't want to get a report a phishing report for every spam message that people get especially if the spam filters aren't great so people need to know these guidelines what they're supposed to be doing to analyze and to report and if these aren't done then it's going to have a negative impact on the results the 11th and final official pitfall i'm Easily spotted tests going to talk about is these easily spotted tests so one of the things that um i mentioned a little earlier is that sometimes people will suspect that it's a test rather than um you know just trying to analyze a message as if it's a real phishing message um and so what happens is management will say okay well we we did a phishing test six months ago we did another one but we need to do them more often get more data so we can do more of a trend well if you have too much variability in the data already doing more tests is not going to really help in fact it's going to hurt because you will start to see a pattern of people recognizing those tests um and you know you you can it also gets to be more work for the the it team to do them more frequently so there's a bit of a trade-off there too but you really don't want people getting too used to the pattern so you have to kind of space them out a little bit and not do them too often so that's the uh whole list of pitfalls that i've gone through quickly and um i want to add one other and that is when you think about how these tests are being run i mentioned you know we're measuring the click rate we're seeing how many people click uh within the test range a group of people and we can sometimes detect uh people that report but typically i've seen that number you know the clickers themselves usually range between two on a really really good uh test where it's really either really obvious or it's some organization that's very good um up to maybe 15 or 20 you know on an ongoing basis that's pretty high but um aside from that if you have a reporting mechanism like the fish hook icon or a reporting button then you can get another 30 percent of the people to report so then you've got data on 40 or 50 of the organization and how they actually were able to decide whether or not the message was was uh uh phishing risk and so the rest of them don't learn anything um so you you won't actually get feedback uh from uh anybody unless uh they have actually made a decision so that's another sort of pitfall i'm not going to put it in here as something that you can really easily identify but it is an inherent problem with testing in this whole live email mechanism we actually don't have a chance to teach people who don't bother uh either clicking or reporting so that's the whole range of them and Distractions when you think about it you know that 11 or 12 actual pitfalls and things that i've had to deal with over many years of doing fishing assessments it as you can imagine it becomes a big time commitment for the it team and there are distractions and people are calling and complaining and management's saying make them harder make them you know easier um and and the i.t team has already overworked and stressed so each campaign tends to be a little more difficult or or adds to the stress and difficulty level for the administrators now you can get automated systems and many of them are better now at being uh automatic at selecting messages to send and scheduling them and so they can be much more hands-off the problem with those is that they are actually easier to spot um the test messages that are generated automatically um just because they they are trying to stay within that band of not being too um not not being too uh provocative and also they can't be too specific or else um you know it's not going to be relevant to everybody so there will be sort of a perfected or or a very polished look to those messages that come in as tests even if they're automatically generated so there's it's kind of a tough spot there for the it guys so when we're looking at this you know i'd like to sort of share that after Live fishing simulation i've done this for a number of years i'm going you know is live fishing simulation really the best way to do this given that you know we've got all these objectives trying to get the teachable moments the vulnerabilities the business productivity and the cultural you know uh considerations as well which the ikey team doesn't really have um much kind of ability to deal with uh for every single campaign um so there's a there's a lot of things here that we can think you know maybe there is a better way and so as i was doing that that's when i started to come across the idea of Gamification gamification and gamified learning and assessment and so i started to study because i had had people ask me about it before they'd say you know could we use gamification in our awareness programs and initially i did actually build a jeopardy game for you know the house of commons for an i.t showcase and it was obvious that it was one of the most popular booths and so um i got you know the idea that there's something there with gamification but the more i studied it i realized that you know there's places like gartner there are i.t training organizations and and other training institutes who have done a lot of studies of gamification especially around learning and what i discovered is there are some really interesting parallels and benefits from the gamification aspect of uh training that parallel and and really help with awareness whether it's phishing or any other kind of awareness training so i'm just going to sort of quickly go through these the engagement is obviously the number one everybody understands that when something is gamified it's visually dynamic and interactive so you get um some people's attention and and and you can get them sort of on board but that doesn't necessarily last for very long so you then have to sort of think what can we do to reinforce our knowledge well games are and gamified environments are highly interactive and they can provide you with challenges that make you think and keep reinforcing your knowledge people don't necessarily mind being challenged they rather have that than having to click through pages and pages of boring training and then there's uh the whole idea of scoring and leaderboards that will really drive the competitive side of people even if they say they're not a gamer quite often i see people really like to see that they can move up on the leaderboard or just improve their score over their last previous score even if they don't care so much about uh the people but the leaderboard really does kind of drive people's behavior to go back in and improve and it's that practice that really is valuable in getting people to improve their skills the other area that is really fun and interesting now too is that you can actually put someone in a captive simulation environment where they can see different kinds of threats that they wouldn't otherwise see except maybe in the real world and this is what you know live fishing assessments are trying to do get you into a real risk decision situation and that's really valuable and important if you don't consider all the other pitfalls that we just talked about but if you can put somebody in a simulation environment they know it's not adversarial it's not um you know life or death uh if they click on something and get it wrong um this is where the immediate feedback and the ability of people to see their assessments makes a lot of difference and and does help improve people's skills and then finally because there has been so much interaction and simulation there is the ability for us to take all that data about what people did and use that to actually assess people's vulnerability rather than using one piece of data that we call a click rate we actually can measure people's ability to sort multiple messages within a minute or two based on the rules that they've learned and so it's a very valuable uh kind of environment to be able to work in so that was what we created you know and my theory was that all this would work but Baseline test now that we actually have some paying customers where we've got people with you know over a thousand uh employees having gone through our phishing course you know we do a baseline test and we teach people then about you know sender information links and body content attachments and after each one we do a little simulation and at the end we'll do a final test to see how they do at sorting the same messages that they had at the beginning and it's virtually impossible for them to remember you know the baseline test after having all the interactive challenges and so sometime later they're going to do the final test and they see a huge improvement in their ability to spot those messages so we can actually measure you know 50 improvement on average it's consistent across the different organizations so it's not just a gimmick or uh fad gamification really is driving people's ability uh to improve their uh Benefits vs pitfalls proficiency at spotting phishing messages so how does this actually stack up with respect to our live fishing simulations and exercises that are being done now you know we want is there an apples to apples kind of um benefits uh versus pitfalls so when you look at you know the engagement side of things you know people's participation in phishing assessments it's kind of hit and miss you don't know other people are going to engage in any given message but with a gamified system everybody goes through and they kind of engage because it is inviting them and it's a more pleasurable experience so the next uh pitfall that we you know talked a bit about was firewalls and gateways and you know the filtering and you know did people open this stuff or not though we have a lot less firewall and gateway issues when we have just a normal kind of a environment that's in the cloud as a platform we're not expecting any uh malicious messages to be coming at you so there's um a much less chance chance that a firewall or gateway is going to block the whole system so it becomes less of a hassle for every time you're trying to test people the testing scope is much broader for a gamified phishing assessment area because we can put people in those simulations for the help desk or the payroll um things that you can't do with the phishing message because management won't let you uh so then the culture issue obviously you know gamification is a much more positive environment and approach that a lot of people prefer and then the data that we're getting from people is more consistent anybody who's completed the course has gone through all of the uh similar messages with similar difficulties and we can test everybody on the more difficult ones or on the easier ones but we're getting consistent data apples to apples we have even more scope available because we can go way beyond phishing we can go to social engineering we can do scams we can do fraud even wire fraud and other kinds of things we can use what we call social simulations they are not using our phishing simulator but we have a little uh choose your own adventure kind of thing it's specially designed not to waste people's time so it actually takes you through social simulations but that's just a different avenue that you know you can benefit from beyond uh phishing and then of course all the data that we're getting is much more valuable for the point of view of measuring people's vulnerability we can really get beyond the participation and completion into what is the proficiency and therefore what is the vulnerability of our team What can you do so what can you do now uh with respect to this information if you you know still think okay i've got to keep using my phishing simulators whether you're just being told to do that or you think it's still got value for you and you're able to compensate for all those pitfalls and you've got the time to deal with all that stuff then the phishing assessment optimizer is pdf is downloadable it's got a lot of these it's got every one of these pitfalls that we talked about um with you know the impacts and and the guidance um so that's something that i hope you'll be able to have a look at and uh if you decide you know what it might be worth looking at gamification because there's something there right it really is showing some good promise some good Free assessment results so we have a couple of things that i'm going to suggest number one is actually we have a free assessment online called canibefish.com and it's really just a place you can go and do a three minute email inbox simulation and test your skills at spotting phishing messages that this one is all about the covid pandemic so it could be messages from you know the help desk about a change in you know procedure or something like that or it could be from the world health organization with some pandemic information so it's a fun little gamified fishing thing where you have to sort uh save for suspicious messages and that just gives you an idea of what that kind of gamified phishing assessment environment looks like and then we can actually do a free trial for you we can set up an organization for you to have a few stakeholders do an evaluation see if you know it's able to provide immediate feedback like i think it will show you how easy it is to deploy how much data you get and how important it can be for your risk management program so at this point Questions i want to open it up for some questions i'm sure there are some questions we've got people live on the chat here and i've got some people on the discord link as well um so i'm going to open up the chat and see what we've got in terms of questions all right so first question is um how can you convince managers that the data they are asking for from phishing exercises might not be very reliable so how do you approach a manager basically um to tell them you know i don't know if we're getting the most value out of this program what can we do sometimes you'll find managers are just trying to get done what they were asked to do so if it's an i.t manager that's responsible for the program and you want to talk to them and say you know what um there's some documented issues with this kind of approach are we addressing them they may not care they might just say i'm doing these things this is the way i'm told to do it i'm going to do it so you might have to go up a level and try and find somebody in senior management or executive because somebody along the way is doing this to improve the uh cybersecurity posture uh and risk posture of the organization uh it might be for compliance reasons so there could be uh just the need to say they've run them and in that case there isn't an awful lot you can do so hopefully that helps you know just trying to engage with people tell tell the managers that there is some documentation you can provide them with the phishing assessment optimizer download and then discuss it with them i'm happy to discuss it as well um any other questions so from sue how often should fishing simulations be run to be effective so that's a good question um you as i said you know you might think you should run them fairly often so you can get data and do a trend analysis but as i showed the trending data is hard to get right especially the more often you're running them the more campaigns you've got to create that are consistent and difficulty so it depends what your objectives are but i would say you want to avoid getting to the point where the messages can actually be spotted by people so in my view i think if you do it more frequently than once a month that's a big risk that people are actually going to be able to spot the phishing messages that are just tests and um i would say my recommendation would be probably about quarterly um is a boat as often as you uh can get good value out of um i think there's uh a lot of different considerations there but anywhere between monthly and and quarterly um let's see uh sem says um does it make sense to do fishing exercises as well as gamified fishing training and assessments yeah actually um it's actually in my view a complimentary thing there are reasons to do phishing assessments for example in the gamified environment we can measure people's ability to make a decision in a simulation of a captive you know risk environment it may not be as uh pressurized an environment let's say you know when when somebody gets a real email they're considering all the other priorities in the day and they might be more prone to opening something if it comes in live rather than within the gamified environment so there is a little bit of you know a risk there so i think that you know having phishing assessments once in a while can actually compensate for that um but as i said you don't want to do the live fishing assessments more than probably quarterly or maybe monthly but you can actually do the gamified fishing foundational courses initially and then repeated gamified challenges that come once a month and people don't you don't have to worry about whether people are expecting them or not they should be expecting it and you should be able to keep their skills refreshed by continually doing gamified assessments and the more you do that the less often you really need to do the sort of audit type of live fishing simulation so you could probably do live simulations every six months three to six months probably if you're using gamified assessments and training as well um what do you do about remediation for repeated clickers so yeah that's a common question that i get um it's uh certainly you know there there are some places that are really strict you know they cannot afford for people to be uh falling for these things and i've heard someone say you know what if if somebody fails a phishing test three times they they're terminated which sounds kind of extreme to me but i'm not going to judge i'm not going to say yeah that's the way to do it i prefer more of the positive incentives to say quite often there will be an escalation of um activities so the first time might be an i.t security briefing one-to-one from a manager or you might invite people to a webinar and say you know what your attendance at this webinar will impact your performance review next quarter that would be you know a pretty good alignment of uh incentives to get people to make sure that they are uh proficient but in a gamified world you know you might just ask them to go back in and repeat the course and do better on the course because it's not something that they will have been able to memorize there's so much interaction and randomness in it um it really does test their abilities and if they can reinforce their you know ability to spot specific kinds of risks and then do better on that gamified test then that could be really the best way to do a remediation all right well it looks like uh we're just about out of time so i just wanted to ask if anybody would be able to type in uh the chat what their best uh memory of this session was you know is there any one thing that you learned about it uh in this session that you feel you should take away maybe you might want to talk to managers about or just something that you'll use to review your own fishing program um [Music] i see on the discord there's sue yeah they're rebellious employees that's fun um yeah you never know the people who will click on things even if even if you have a program that says they're gonna get some kind of um you know warning there's still people that will do it um sem says talk to legal in hr yeah so i mean getting buy-in from the legal and hr groups is always good when you're doing any kind of a change or or a an activity that could impact the culture of the organization or people's responses to that so definitely a good good thing to take away um gamification seems cool and valuable yeah absolutely and i do encourage people to um you know try the can i be fished uh just to see what it's like and then reach out if you'd like to set up a trial all right so i want to thank everybody for joining us and um i hope i was able to bring some value to the session for you things that um you may be able to think about or take back to the management team and get their uh their feedback on try and do something maybe a little different more innovative and get better value out of these things so i'd love your feedback and you can reach out to me on linkedin or twitter uh even by our website there's a little chat bot that has my picture on it we'll pop up and i'll answer your question there or we can follow a little bit of a discussion um i just think it's important to get more people engaged in this discussion understanding what those risks are making sure management is aware of you know the risks of what the traditional fishing simulations have and also the um the benefits of of trying something different like gamification so i appreciate your support and hope to see you next time uh and uh have a great day and a great life thanks everybody