AI didn't remove the need for secure design — it amplified our mistakes. The OWASP Top 10 for LLM Applications lists nearly 100 mitigations across just 10 risks. Nobody can memorise that. So I took the classic software security principles, pressure-tested them against the AI threat landscape, did some rethinking, which ended up with a list of just 7 principles that any development team can act on. Here's how I got there.
By Michael Ustrup, Software Security Architect at Systematic
The principles didn't change. The system did.
In 2002, Gary McGraw and John Viega published "Building Secure Software" and gave us ten principles for software security. Secure the weakest link. Practice defence in depth. Fail securely. Principle of least privilege. Compartmentalise. Keep it simple. Promote privacy. Hiding secrets is hard. Be reluctant to trust. Use community resources. For over two decades, these principles served us well. They still hold. But AI shifted the centre of gravity.
Writing traditional software was like laying tracks for a train. As long as the rails were strong, the train stayed on its predictable path. You knew the inputs. You knew the outputs. You could reason about the system because the system behaved deterministically.
Once you integrate an AI agent, those tracks vanish. The inputs become unbounded — natural language in all its ambiguity. The outputs become probabilistic rather than deterministic. And the component you just added to your architecture doesn't just follow instructions — it interprets them, sometimes creatively, sometimes dangerously. Your role changed. You went from programmer to AI pilot, navigating systems of unbounded complexity.
Three shifts that forced the rethinking
Your attack surface silently exploded. A traditional application has well-defined entry points. An AI-powered system introduces attack vectors we are not used to thinking about: prompts, model outputs, training data pipelines, fine-tuning interfaces, Retrieval Augmented Generation (RAG) databases, and autonomous agent behaviours. Each of these is a door. Each door can be picked.
You are not coding alone anymore. The AI agent beside you is a mathematical genius — but it is also incredibly gullible. A model trained on the vast internet doesn't just know how to parse text. It possesses the knowledge to produce highly sophisticated malicious scripts that could exploit your system's weaknesses. You wouldn't wire your copilot's console to the missile launch button. So why give an AI agent unconstrained access to your production systems?
The adversaries got faster. With autonomous AI on the offensive side too, the time from vulnerability discovery to weaponised exploit has compressed dramatically. What once took nation-state actors months can now happen in hours. Your patch cycles need to reflect that reality.
These three shifts meant I couldn't just carry the original ten principles forward unchanged. Some survived because they became even more critical. Some needed reinterpretation. And new patterns emerged that the original ten never anticipated.
From ten to seven
I kept four of the original McGraw and Viega principles — least privilege, compartmentalise, be reluctant to trust, and promote privacy — because AI made them more urgent, not less. Least privilege matters more when an AI agent can act autonomously on behalf of a user. Compartmentalisation matters more when prompt injection can socially engineer a machine into ignoring its guardrails. Being reluctant to trust matters more when input validation now extends to natural language semantics, not just code syntax. And promoting privacy matters more when LLMs don't just process data — they absorb it.
Three new principles earned their place. Perform threat modelling came first, because you cannot secure what you haven't mapped — and it is naive to think you can eyeball unbounded complexity. Minimise the attack surface became essential because an LLM out of the box arrives with every capability enabled by default. And human in the loop emerged as the necessary speed limit in an industry racing towards full autonomy. AI is a statistical prediction engine, not an infallible oracle. The AI plots the course. The human activates the launch.
Together, these became the 7 AI Software Security Principles — or 7AISSPs. They are deliberately scoped to what your development organisation can act on directly. They won't solve everything. But they give you a thinking framework you can carry with you, in a domain where the OWASP list alone would fill a small book.
Start with one
You don't have to adopt all seven tomorrow. Pick the one that resonates most with your current project. Live by it. Make it part of how your team thinks, not just how your architecture looks.
It is better to live by one principle tomorrow than aspire to all and achieve none.
Michael Ustrup is a Software Security Architect at Systematic with more than 20 years in systems engineering and 15 in software security. He holds two master's degrees — in Computer Science and Cyber Security — and divides his time between threat modelling, red teaming, coaching development teams, and real project work. He authored Systematic's Security Development Lifecycle and the software security course which he enjoys teaching.