In this week’s bulletin, Charlie looks at the pros and cons of SIMEX (Simulation Exercise) and gives an insight into his experience running a live SIMEX.

About three weeks ago, I planned, ran, and reported on the biggest SIMEX I have ever conducted in my whole career. It was a ‘no notice’ exercise, meaning the organisation was not aware there was going to be an exercise on the day. The exercise involved a client with five different teams responding to a cyber incident. It started at 09:00 and finished at 16:00 with a hot debrief. This week I have been planning two further SIMEXs, not on the same scale, and with only one team taking part. I thought I would share some thoughts on what a SIMEX is, when you should conduct one, and perhaps when you shouldn’t conduct one.

According to the Good Practice Guidelines, a SIMEX or SIMEX Simulation exercise ‘are operations-based exercises designed to be more realistic and challenging. They can be carried out in the normal operational environment, alternative premises, or command centres and involve key participants’. They allow participants to practise decision-making, coordination, and resource management in a controlled, realistic, environment.

The delivery of a SIMEX exercise can mean different things to different business continuity practitioners. When I conduct a SIMEX, my goal is to create an immersive experience that replicates the pressures and tensions of managing an incident response. The exercised team should feel as though they are in a genuine scenario, responding by using the same plans, tools, and techniques as they would in a real incident. They are usually conducted in the same room, command centre, or video conference platform, that a team would use in real life. It is important to ensure that those who would be the primary responders, or at least their deputies, take part in the exercise and that this role is not delegated to people who would not respond if it were a real incident. Injects driving the scenario forward are not presented via slides, but are introduced through the channels that the team would typically use during a real incident, such as telephone calls, emails, or face-to-face briefings. These injects may originate from internal staff or a wide array of external stakeholders.

When I conduct a SIMEX, I use PlanB Consulting’s MITS platform to provide an evolving media and social media environment. The platform simulates social media and news broadcasters, allowing participants to post their own social media updates and press statements. PlanB Consulting’s MITS operator will respond to press statements promptly in real time.

We usually provide a role player coordinator, and role players – who put the injects into the team and the team can receive calls from them – simulating stakeholders. Often, clients’ staff are used as role players as they know the responses to be expected and can be more realistic in the information provided when responding to calls from, and putting calls into, exercise participants. Role players assume roles representing both internal and external stakeholders, aiding in inject delivery and facilitating communications.

During exercise planning, a comprehensive range of injects are developed and introduced to progress the scenario and present various issues associated with it. An exercise director oversees the exercise to ensure participants derive maximum benefit, adjusting the pace of injects as needed to prevent participants from becoming overwhelmed or disengaged.

We also provide umpires to evaluate the team’s performance, and if necessary, may involve the client to provide additional umpires. Media interviews, conducted over the phone, via video conference, or face-to-face, can be arranged based on the client’s requirements and the specific elements of the response they wish to practise. I find that participants who have taken part in my SIMEXs often, at the end, report feeling as though they have responded to a real incident, therefore gaining valuable lessons that can be applied to future incidents.

SIMEXs can be lots of fun to plan and deliver, but what are they good for?

1. They are best suited for mature incident management teams that have completed a number of tabletop exercises and want to advance their response practice to the next level.
2. They practise leadership, teamwork, and team dynamics, which are often not fully exercised during a tabletop exercise. Identifying if a team is dysfunctional or ineffective before an incident occurs is crucial, rather than discovering it during one.
3. They develop leadership, teamwork, and team dynamics, which are often not fully tested during a tabletop exercise.
4. If you conduct incident management training before taking part in a SIMEX, learning skills such as logging or situational awareness, then a SIMEX is a good way to practise these skills under pressure and also to check on the effectiveness of the training.
5. They can give a team confidence when faced with a real incident that they are ready to respond and manage it effectively.
6. They can be a proof for the teams competency, bringing together their skills, knowledge and their ability to adapt and use them in response to a scenario.
7. For some teams, we use our Incident Team Performance Assessment to evaluate their performance across various criteria. The team is scored by the umpires and exercise director, and participants also conduct a self-assessment. The first exercise establishes a baseline, and subsequent exercises are scored using the same criteria. This allows the organisation to track the team’s performance over time and identify areas where additional training could improve their skills.

SIMEXs are not often conducted as they do have a number of downsides:

1. They are time-consuming to plan, expensive if using a consultant, and need a much greater level of planning than a tabletop exercise.
2. They require more people to carry them out than a tabletop exercise.
3. Once they start, they usually continue in real time throughout the full duration of the exercise, so there is the possibility for the exercise to dissolve into chaos or deviate from the planned response. If an exercise does dissolve into chaos, the team can be demoralised, feel they are unable to manage an incident, or learn the wrong lessons and ways to respond. There is a balance for interfering in the response and making the exercise a glorified tabletop exercise to letting the team learn from their mistakes.
4. They are only suitable for those with experience. If you conduct them using beginners in incident management, they will likely ignore the plan, make up the response as they go along, and not conduct some of the finer points of exercising, such as logging.
5. They usually only deal with the immediate response to an incident, and some of the more pressured situations come later in the incident response. This is especially true for crisis teams. They are often most under pressure when their response has gone wrong. SIMEXs usually don’t allow for time jumps, so typically only the initial response is exercised.
6. Most incidents start and then build over time, this could be days and months or hours, and actions by the team responding would influence how the exercise proceeds. Unless the incident is an immediate ‘big bang’ (literally) start, then you need to build in the prequel activities and story which makes the exercise more complex.
7. Not all elements of the response are exercised or explored. Some decisions which should be thoroughly considered, such as whether to ‘pay or not pay’ a ransom after a ransomware attack, may be taken in 2 minutes in a SIMEX, as there are no consequences to the decision. Other forms of exercise, such as tabletops, allow the participants to explore and discuss items and be challenged on their decisions, which a SIMEX doesn’t always allow.

Sometimes I have had clients who want a SIMEX because it seems more fun than a boring old tabletop, and the exercise becomes more about entertainment, like business continuity meets an escape room. Sometimes they have to be dissuaded from this, as an exercise is a learning experience, and you want to get maximum benefit from it. SIMEXs can provide extremely beneficial learning experiences, but they should only be used for well trained and experienced staff.