Seize the Flag (CTF) occasions are each enjoyable and academic, offering cybersecurity professionals a strategy to flex their hacking expertise whereas studying new ideas in a constructive and secure setting. Properly-designed CTFs expose people and groups to operational challenges, novel assault paths, and artistic situations that may be later utilized of their work each as offensive and defensive safety professionals.
However not all CTFs are created equal, and there is much more that goes into designing a profitable CTF competitors than simply developing with the challenges. Together with the technical design challenges, there are additionally operational concerns concerned with organising the setting and truly operating the competitors, artistic planning required to arrange an attractive recreation, and factoring in particulars associated to gamifying the challenges, equivalent to tradeoffs in how scoring construction is about up.
“As a designer I would like it [the CTF] to be difficult enjoyable. I need to reward people who find themselves intelligent, who actually work at it, and who’re persistent,” says Jenko Hwong, principal researcher on Netskope’s Menace Analysis Labs staff and staff chief for final 12 months’s DefCon Cloud Village CTF. “It additionally needs to be sensible for us to hold out.”
Enjoyable and sensible was the mindset that Hwong dropped at the DefCon CTF, an enormous multi-day affair that had over 400 people and groups making an attempt their arms on the problem and a staff of 20 working beneath him to run the occasion. A veteran researcher and seasoned CTF participant, Hwong had by no means run a CTF earlier than this occasion. One in all his greatest hopes for his first attempt on the job was to degree up the relevancy and realism of the challenges within the occasion, which might typically be a bugaboo in CTFs as we speak.
“Typically in these CTFs you get a extremely exhausting problem nevertheless it’s like what is the level of this? It’s going to be a decryption or encryption drawback, the place the occasion goes, ‘Here is one thing, good luck,’ after which it’s a must to leap by way of all these hoops that is probably not fully divorced from actuality however do not actually match into a much bigger storyline,” he says. “So, once I obtained the decision, my thought was ‘let’s leap in, let’s work out an excellent story and an excellent set of challenges that will probably be enjoyable but additionally that make sense and possibly relate to the true world of analysis penetration testing, defensive measures, what’s taking place in the true world.'”
As he dove into the mission, although, one factor he discovered particularly difficult is how little info there may be on the market about operating CTFs. Most write-ups are from members who fee an occasion and clarify how they solved challenges, however there’s not often info supplied on greatest practices in operating an occasion. Consequently, he mentioned that he and his staff needed to do a ton of labor was completed creating challenges practically from scratch.
“The neighborhood typically shares a ton, so why aren’t we sharing CTF challenges?” he says. “I feel we are able to do higher.”
In that spirit of safety neighborhood sharing, he shares some essential classes that his staff picked up alongside the best way in order that others accountable for CTF design can be taught and perceive from the method. His aim is to run the occasion once more and construct on what they discovered final 12 months. He additionally hopes others will share their greatest practices, and even technical particulars, in order that the entire safety neighborhood can enhance the standard of CTFs being supplied.
Storytelling is Key
Hwong says that his DefCon Cloud Village staff was very eager on crafting a storyline that was participating and enjoyable. He says he considered the story as a film script with life like cyber situations inbuilt. For the occasion they selected a theme of ‘Gnomes’ that was enjoyable and humorous. nevertheless it wasn’t simply the storyline writing that was essential but additionally how the technical challenges had been deliberate inside the story.
“The goblin and gnomes storyline wrapped round the whole lot however the essential factor was developing cheap situations that you simply may encounter as a safety skilled, together with assault paths and cheap defenses you’d encounter,” he says. “The extra we are able to try this as CTF designers, the higher it’s for studying and the extra enjoyable the CTF.”
Take a Software program Growth Method
CTF creators ought to positively take a software program improvement method to designing the technical parts of their problem, Hwong recommends.
“You have to consider design, implementation and testing,” he says, explaining that he and his staff discovered the exhausting means how troublesome it may be to check challenges in a fancy CTF setting that may be manipulated by members in quite a few methods.
“What occurred—and I am going to take the blame because the lead creator for not guiding the testing—is we missed the adverse testing move, in addition to the viability checks,” he says. “A part of it’s we did not have sufficient time to check, so I used to be persevering with to lock down some environments because the problem was underway so among the challenges would not be too straightforward and there have been no loopholes. I feel at one level for an hour or two I ended up making one thing unsolvable at a sure step.”
So, one of many massive classes he discovered is that CTF designers must deliver software program improvement rigor to the desk that goes all over testing and viability work.
Operational Rigor…and a Little Little bit of Caffeine
Meticulousness in software program improvement is not the one technical functionality that should come to the desk. The crew operating a CTF additionally wants some critical operational rigor as effectively.
“We had some fabulous individuals operating the servers and the AWS accounts and the Google and Azure accounts and ensuring issues saved operating and that we had been monitoring issues,” he says. “All of that stuff needs to be dealt with. And when you ignore it, it simply might imply issues fail, break or you might have efficiency issues.”
One of many operational issues they bumped into was that they skilled some collision between members and challenges, because the staff was working with a constraint in that they could not create a standalone setting for each participant throughout AWS, Google, and Azure.
“As a result of it was in the identical setting, it helped them on different challenges and when you have a problem that requires altering the setting then you might have individuals stepping on one another’s toes, altering a shared object,” he mentioned, explaining that he and his staff needed to reset insurance policies because the CTF rolled ahead so members would not run into each other.
He and his staff are attempting to be taught from the expertise to determine a sensible technique—from time, effort, and expense perspective—to present members a very remoted setting with out making the entire CTF much less viable as a result of issues break or take without end to execute.
Lastly, Hwong says that on the operational entrance CTF present runners additionally must be aware of the fixed communication that they will must facilitate between their staff and members.
“I used to be on Discord after midnight and I am like, ‘I’ve obtained a chat to present within the morning, would you fall asleep?'” joked Hwong, who defined that members could have questions and they will be pinging organizers for suggestions and pointers in any respect hours.
Designing Totally different Issue Ranges Is Laborious
Getting the problem ranges of challenges proper and creating a good scoring system could also be tougher than a beginner CTF organizer could initially suppose, warned Hwong. He defined that a number of of the degrees that his staff designed as simpler had been tougher for members to finish than they’d anticipated, whereas among the tougher ranges had been efficiently completed by extra members than anticipated.
Hand-in-hand with the problem leveling problem is determining a scoring system that is smart. After his expertise at DefCon, Hwong is a proponent of performing some form of Bell Curve scoring system. However he says the issue is not as simple as instituting a curve. There’s additionally the problem of normalizing and balancing out the benefit that massive CTF groups have in racking up problem factors—a problem that one of many members supplied him suggestions about after the occasion.
“So in case your challenges could be divided and completed in parallel a number of gamers, if I’ve obtained 10 individuals I will probably be 10 instances is quick. And so there’s a bonus,” he says. “His level was some form of dynamic scoring ranges it a bit of bit. If there are issues that he is actually, actually good at, he is perhaps the one one who solves it and he’ll get most factors. The bell curve will reward him versus scale does not essentially matter if it is one thing in his wheelhouse of experience by way of 10 versus one. There’s some debatable stuff right here that we’ve got to work by way of.”
One risk is making challenges sequential, however the draw back of that’s it might make the CTF too inflexible and linear, and it might create a bottleneck or dependencies that would blow up a number of challenges. Hwong says he’d additionally like to see extra CTFs reward members on methods like how stealthily they function in an setting or dock factors in the event that they go away too many footprints and fingerprints, and that is an space he’d wish to discover as he designs future occasions.
Regardless, although, dynamic scoring is one thing that would alleviate among the leveling points and he and his staff are pursing that for the approaching 12 months.
Blue Groups Want Extra Enjoyable CTF Challenges
After working by way of his first CTF, Hwong additionally more and more believes these occasions do not do sufficient to problem and actually have interaction blue staff members.
“Blue staff workout routines are likely to go like this: ‘We now have a misconfigured setting with plenty of vulnerabilities. Are you able to go repair them?'” he says. ” And what they do is they only take a look at whether or not these configurations are modified or not or whether or not I can entry this public bucket. And as quickly as you make it personal, we all know you mounted it and also you get factors. It might be means higher to do issues on high of that, equivalent to what when you’re compromised, there’s an attacker in your setting, it’s a must to discover them and kick them out. So you might have an incident happening proper now, and so long as the attacker is, they’ve credentials and so long as they’re going to do issues, you may have the ability to detect it. That is your job as a participant. And till you revoke their entry, you do not remedy it and you do not get most factors.”
These form of situations are tougher to do however they’re extra life like for defenders and can make CTFs extra helpful for them, he says, explaining that’s on his radar for subsequent time.
CTFs Want Extra Contemporary and Related Elements.
Hwong additionally challenges CTF designers—and himself–to incorporate extra contemporary exploit and vulnerability info into their challenges. This was one of many issues he wished he had extra time to dive into in his first go at DefCon Cloud Village and which he is resolved to enhance for subsequent 12 months.
“This is likely one of the areas the place CTFs could be extra of a studying and coaching software,” he explains. “We might love to make use of related concepts and exploits contemporary from researchers occurring earlier within the 12 months and even introduced at DefCon.”
CTF ‘Constructing Blocks’ to Enhance ‘Reusability’
Lastly, one of many greatest classes Hwong says he discovered is that the trade wants to seek out extra methods to create reusable elements for CTF similar to software program builders do for purposes. He has desires of serving to to arrange an open GitHub repository of small workout routines in code that may type the constructing blocks of constructing out a CTF.
“You are still going to must customise it and add your individual twist, however the thought is let’s get the primary 60% out of the best way so CTF organizers can give attention to actually novel issues. That means no person is reinventing the wheel,” he says. “After which the remaining 40% could be including new methods, situations, and storylines.”
