In today’s bulletin, Charlie discusses all things AI, looking into the risks of using artificial intelligence in the workplace and how we can better prepare for its growing influence.

This week, I travelled to London to speak at a conference and, remarkably, I was able to work online throughout the journey by combining the train’s Wi-Fi with my phone’s hotspot. This was a first for me, as the Wi-Fi is usually unreliable, and I have to do activities that don’t require you to be connected. I find that much of my work is assisted by AI, whether that’s putting together documents, carrying out research, or collating information, and without it I sometimes become much less efficient. As more of us, both personally and in our organisations, use AI, it is becoming an essential tool. Without it, we may not be able to do our work or deliver services to customers. As business continuity practitioners, we need to understand the risks associated with AI, how to document them within our BIAs and risk assessments, how to mitigate them, and how to incorporate the loss of AI into our plans. In this bulletin, I look at the risks we should consider when thinking about using AI.

Do you understand your dependencies?

In IBM’s report, published last month, a couple of quotes caught my eye. The report found that: “Only 9% of executives say they have an excellent understanding of their dependencies on AI vendors, models, and infrastructure”, while “71% say switching their primary AI vendor or model would be difficult if they had to do it today”. [1] Both of these quotes suggest to me that AI is important to many organisations, but the risks associated with it are not fully understood.

Negative societal view of AI

I have read a number of articles on AI, particularly in The Economist, and the general public perception of AI is not always positive. Many see AI as a technology that will take people’s jobs, especially those at the beginning of their careers. There are also concerns over hallucinations, producing incorrect or misleading information, and about the high energy consumption of AI data centres. People understand that we are on the cusp of a technology revolution, but are nervous about how this will impact their lives.

There is also the “that was written by AI” attitude to CVs, applications, articles, reviews and reports, with people feeling that others are in some way cheating the system by using AI to write text much better than they could write themselves. My director, Chris Butler, took great delight yesterday in telling me how much AI use he had spotted in the job applications he was reviewing. AI use now is, I think, often seen as a negative rather than an embrace of technology to make your work more efficient.

So what risks associated with AI should we, as practitioners, identify?

1. The use of AI to assist in cyber attacks

This could include using advanced AI tools to detect software flaws and develop exploits to attack them. AI can be used to create more realistic and authentic phishing and spear-phishing attacks, or to generate authentic or recognised voices for vishing attacks. One of the most high-profile examples was the loss of $20 million by Arup after fraudsters reportedly used AI-generated deepfake technology during a Microsoft Teams call to impersonate senior executives.

2. AI use in service delivery

Organisations increasingly rely on AI-powered chatbots for customer service and sales. AI is also used in research, in SOCs to detect attackers, and in the assessment and collation of information. In the recent Iran/USA conflict, I read a number of articles about how AI was used to assess, categorise, and rank targets for US and Israeli plans to attack targets in Iran. AI was able to greatly increase the number of targets that could be attacked, as the limitation was the ability of humans to assess the targets, rather than the number of aircraft available to attack them. For many organisations, AI is a key part of their business processes, in the same way as traditional IT applications, so we have to consider its loss as part of BC planning.

3. Over-reliance on AI information

There have been a number of cases where over-reliance on AI has produced reports where many of the quotes and references have been hallucinated by AI. One of the most well-known cases was that of Deloitte Australia, which admitted using generative AI in a $440,000 report for the Department of Employment and Workplace Relations. The report was later found to contain fabricated citations, a false Federal Court quote and other errors. The department republished a corrected version and Deloitte agreed to refund part of its fee. Once the issue was recognised and addressed, the report was republished, mainly unchanged, so the underlying report content was valid. However, the misuse of AI in this case greatly impacted Deloitte’s reputation.

Another example occurred in the United States in 2023. Lawyers in the case Mata v. Avianca used ChatGPT to prepare a legal filing that included fake case citations and fabricated quotations. The court sanctioned the lawyers, highlighting the risk of relying on AI-generated content without proper professional verification. There have been a number of other cases where lawyers, or those representing themselves, have used AI-generated hallucinated cases. Where people have relied on AI to provide information and this has been wrong, there can be a major reputational impact. This seems to me to be based partly on the fact that the use of AI can, in some way, be seen as cheating.

4. Misuse of shadow AI

Employees misusing AI can be a reputational issue, especially when AI is not provided to them or is tightly controlled, leading people to use shadow AI. I remember a client talking about a case in which someone working for their outsourced IT provider used shadow AI to process the company’s personal information. When this was discovered, the incident had to be reported to the ICO for losing control of PII.

5. Security flaws

With AI, it is very easy to produce software without needing to code. This is enabling individuals within companies to produce internal IT tools or tools and applications that they then sell. AI allows you to build the tool, host it, and even charge a fee for using it. Often, these systems have not been properly security-checked, so they may be highly vulnerable to attack. As more people are producing AI tools and applications, this is going to become more of an organisational security issue.

6. Reputational issues

The use of AI has produced a number of reputational issues for organisations. Air Canada lost a case in which its AI chatbot said a customer would be refunded for a bereavement flight. The customer tried to claim a refund for the flight and was told by Air Canada that the chatbot was purely advisory. The customer sued, and Air Canada was found liable. In addition, DPD disabled part of its AI-powered chatbot after a customer managed to get it to swear, call itself useless, and write a poem criticising the company.

AI has also created risks in recruitment. The US Equal Employment Opportunity Commission alleged that iTutorGroup’s recruitment software automatically rejected female applicants aged 55 or older and male applicants aged 60 or older. The company settled for $365,000.

The National Eating Disorders Association (NEDA), a US non-profit supporting people affected by eating disorders, faced criticism in 2023 after ending its human-staffed helpline shortly after workers had unionised and moved towards the use of the Tessa chatbot. Criticism increased when Tessa was reported to have given harmful weight-loss-related advice to users. This led NEDA to take the chatbot offline.

All these cases resulted in reputational damage for the organisations involved. In cases such as NEDA, where human staff were replaced by AI and this then went wrong, the reputational impact is increased.

7. Loss of the AI service provider

The AI used in organisations is generally provided by a third party on a SaaS model, so the loss of the AI provider is a risk. With the huge amount of money going into various AI companies and the recent AI IPOs, there is a chance, as with the dotcom bubble, that some AI companies will fail. In our planning, we should consider how our organisation would continue to provide services if its AI provider failed, as mentioned in IBM’s report. [1]

8. Backup of AI

Where AI has instructions or scripts, these should be backed up. These need to be documented and saved securely, as with application backups. I am not aware of best practice on this, especially with agents that are constantly learning, so it is worth looking at what best practices exist.

9. Emerging and unknown risks

Useful sources for tracking real-world AI failures include the AI Incident Database (AIID), which records examples of AI-related harms and failures, and the OECD AI Incidents and Hazards Monitor, which tracks AI incidents and emerging hazards from public sources. These might be worth keeping an eye on for any new threats.

10. The Armageddon risk

This is the risk that an intelligent AI rebels against its human creators, develops its own consciousness, and sees us as a threat. This is a future threat, but at the moment I think there are much more likely risks associated with AI than worrying about having a business continuity plan for this one!

AI is very much an emerging technology and is changing almost by the week, but as BC professionals, we have to be aware of how our organisations are using AI and its relative importance for the delivery of products and services. It may be a tool individuals use to make their work more efficient, or a key component in service delivery. We need to think through how we recognise the use of AI in our business continuity management system, especially how we record its use in the BIA and risk assessment. We also need to understand the risks associated with AI so that we can include them within our response plans and possibly exercise the loss of AI.

References

[1] Dekkers, H., Olaizola, J., Goyal, M., Srinivasan, S., Kurien, P. and Zabel, R. (2026) The calculus of AI sovereignty: Balancing control, flexibility, and risk. IBM Institute for Business Value. Available at: https://www-api.ibm.com/adobe/assets/urn:aaid:aem:57477381-e34f-4f94-871c-d08682c27397/original/as/the-calculus-of-ai-sovereignty-report.pdf (Accessed: 3 July 2026).