This post is based on our conversation with Peter Holcomb on Agent Hour. Peter is the founder and CEO of Optimo AI, a fractional CISO practice that has run more than 100 engagements helping AI-native companies adopt AI safely while getting through frameworks like SOC 2, ISO 27001/42001, GDPR, and HIPAA. Before starting Optimo, he spent nearly two decades in security leadership, including CISO roles at Datavolo (acquired by Snowflake) and eMed.
In today's fast-evolving technological landscape, companies are increasingly harnessing the power of Artificial Intelligence (AI) to enhance productivity and streamline operations. However, with this adoption comes the challenge of managing what is known as "shadow AI"—the use of AI tools and systems that are not officially sanctioned or monitored by an organization. In this post, Peter Holcomb walks us through the implications of shadow AI, the importance of governance, and how organizations can ensure accountability when things go awry.
What is Shadow AI?
According to Holcomb, shadow AI refers to the use of AI systems and tools by employees without the knowledge or approval of the IT or security departments. This can include personal accounts of popular AI platforms, like ChatGPT, which may not be compliant with corporate policies or security standards. As organizations encourage innovation and productivity, Holcomb notes, the risk of unmonitored AI use increases, leading to potential security vulnerabilities and compliance issues.
The Risks of Shadow AI
Drawing on the patterns he sees across client engagements, Holcomb points to three consequences of unmanaged shadow AI:
- Lack of Visibility: When employees use unapproved AI tools, organizations lose visibility into what data is being processed and how it is being used.
- Data Security: Shadow AI can expose sensitive information to unauthorized access, leading to data breaches.
- Compliance Challenges: Without proper oversight, companies may inadvertently violate regulations, such as GDPR or HIPAA, putting them at risk for fines and legal repercussions.
Establishing Accountability in AI Use
With the emergence of shadow AI, questions of accountability become paramount. When an AI agent performs an action that leads to unintended consequences, who is responsible? Holcomb points to a recent Wisconsin court ruling as an early answer: "The organization is gonna be the one that's gonna be held liable," he says — not the vendor, and not the AI itself. This emphasizes the need for robust governance frameworks to manage AI deployments effectively.
Key Aspects of AI Accountability
Drawing on what he's seen across client engagements, Holcomb recommends three things every organization should have in place:
- Clear Policies: Companies must establish clear policies on the acceptable use of AI tools, ensuring employees understand what is sanctioned and what is not.
- Risk Assessments: Regular risk assessments should be conducted to evaluate the security posture of AI tools in use. This includes understanding the potential impact of using unapproved tools.
- Training and Awareness: Employees should be educated on the risks associated with shadow AI and the importance of adhering to company policies regarding AI use.
Best Practices for Managing Shadow AI
To mitigate the risks associated with shadow AI and ensure accountability, Holcomb recommends organizations adopt several best practices:
1. Emphasize Governance Over Documentation
As Holcomb puts it, governance should not be seen as a burdensome documentation process but as an integral part of the AI development and deployment life cycle. Companies should embrace GRC (Governance, Risk Management, and Compliance) engineering, which integrates compliance into the engineering process rather than treating it as an afterthought.
2. Implement Dynamic Permissions
Holcomb draws a sharp line between how identity should work for a human employee versus an AI agent: as AI systems evolve, so too must the frameworks governing their use. He recommends organizations implement dynamic permission systems that allow AI agents to operate with context-aware access controls, ensuring they only perform actions that are explicitly allowed — minimizing the risk of unauthorized actions.
3. Continuous Monitoring and Auditing
"No human's gonna be able to protect that — you're gonna have to have agents against agents," Holcomb says of monitoring AI activity at scale. To maintain oversight of AI activities, companies should employ continuous monitoring systems that track AI performance and compliance with established policies. This includes maintaining robust audit trails for all actions taken by AI agents, enabling organizations to quickly identify and respond to any issues that arise.
Conclusion
The rise of shadow AI presents unique challenges for organizations striving to integrate AI into their operations safely and effectively. By establishing clear governance frameworks, emphasizing accountability, and implementing best practices, organizations can harness the benefits of AI while minimizing the associated risks. As Peter Holcomb puts it, proactive management and oversight will be critical to ensuring that AI technologies serve their intended purpose without compromising security or compliance as the landscape continues to evolve.
Catch our full conversation with Peter Holcomb on Agent Hour — available on Spotify, Apple Podcasts, YouTube, or wherever you get your podcasts.
