A finance employee at the engineering firm Arup joined a video call with several colleagues, including the company’s CFO. Every one of them was an AI-generated deepfake. The call ended with $25 million transferred to the attackers. No malware, no phishing link, no exploited vulnerability: just AI, used convincingly enough that trust did the rest. That single incident captures why AI in cybersecurity has become the defining topic in the field. The same technology built that deepfake is, elsewhere in the industry, catching the anomaly that would have stopped it. This AI security guide covers both sides: how AI attacks, how AI defends, and what that means for where security teams should focus next.
AI in cybersecurity, in one paragraph
AI in cybersecurity refers to the use of machine learning, natural language processing, and large language models on both sides of the same fight: to launch attacks and to detect and stop them. Attackers use AI to generate convincing phishing content, clone voices and faces, and probe systems faster than a human team could. Defenders use the same underlying techniques (pattern recognition across huge volumes of data, natural-language understanding, behavioral modeling) to spot anomalies, cut through alert noise, and respond before damage compounds. The result is genuinely dual-use technology inside a single industry, which is why AI in cybersecurity isn’t a category with a clear “good” and “bad” side. It’s a capability, and whoever applies it faster and more precisely currently has the advantage.
AI as an attack vector
- AI-generated phishing at industrial scale. Phishing used to be catchable by its tells: awkward phrasing, generic greetings, obvious spoofed domains. AI removed most of those tells. According to Hoxhunt’s 2026 Phishing Trends Report, AI-generated phishing jumped from 4% of all reported phishing attempts in November 2025 to 56% in December 2025, a 14x increase in a single month, and has held around 40% of all reported phishing attempts through mid-2026. The effectiveness gap is just as stark: a human-subject study from Harvard Kennedy School researchers found that fully AI-automated spear phishing emails achieved a 54% click-through rate, matching skilled human experts and far outperforming the 12% click-through rate of the control group’s generic phishing emails. AI doesn’t just write more phishing emails. It writes ones that work.
- Deepfakes that target trust, not infrastructure. The Arup case above isn’t an outlier anymore; it’s a template. AI-generated voice and video are convincing enough to pass as a real executive on a live call, turning social engineering from “trick someone into clicking” into “trick someone into believing they’re looking at their CFO.” That’s a fundamentally harder problem for security awareness training to solve, because the deception happens at the perceptual level, not the technical one.
- Faster reconnaissance and adaptive malware. Beyond social engineering, AI shortens the distance between “attacker has an idea” and “attacker has a working exploit.” Large language models can accelerate vulnerability research against a target’s public-facing code and infrastructure, and adaptive malware can alter its own behavior mid-execution if it detects it’s being analyzed in a sandbox, a capability that used to require a skilled human operator watching in real time.
- Credential attacks at machine speed. Feeding leaked credential sets into a model to learn the patterns people actually use when they think they’re being “clever” with a password turns guess-and-check into something closer to prediction. Combined with automated credential stuffing, this compresses what used to be a slow, noisy attack into something fast and quiet.
AI as a defense tool
- Anomaly detection at a scale humans can’t match. The core defensive value of AI is processing volumes of network traffic, login activity, and system logs that no analyst team could review manually and flagging what deviates from an established baseline. This is what catches attacks with no known signature yet, including zero-days and insider misuse that wouldn’t trip a rule-based alert.
- Cutting through the alert queue, not adding to it. This is also where AI in cybersecurity has the most immediate operational payoff, and where the industry still has real work to do. Vectra AI’s 2026 research found that organizations receive an average of 2,992 security alerts a day, with 63% going uninvestigated, and Microsoft and Omdia’s State of the SOC: Unify Now or Pay Later report found that 46% of all alerts prove to be false positives and 42% go uninvestigated entirely. AI-driven triage doesn’t just add another detector to that pile. Done well, it correlates related alerts into a single incident, scores what’s actually worth a human’s attention, and lets analysts spend their time on the fraction of alerts that represent real risk instead of manually closing the rest.
- Malware and behavioral classification beyond known signatures. The same behavioral-analysis approach that flags anomalous network activity works on files and packages too: instead of matching against a list of known-bad signatures, AI can score install-time behavior, unusual system calls, and obfuscation patterns to catch malware that’s never been seen before, at the moment it appears rather than after it’s been fingerprinted.
- Automated, contained response. Detection without action just moves the clock. AI-driven response can isolate an affected device, block suspicious traffic, and start containment automatically, closing the gap between “we saw it” and “we stopped it” from hours to seconds in the cases that matter most.
Why do the same techniques cut both ways?
The uncomfortable truth in this AI security guide is that attackers and defenders are frequently using the same underlying capability toward opposite ends. The natural-language modeling that writes a convincing phishing email is the same technology that reads a suspicious email and correctly classifies it as one. The pattern recognition that lets malware adapt to evade a sandbox is architecturally similar to the pattern recognition that flags the malware’s behavior as anomalous in the first place. Neither side has a monopoly on the technology, which means the advantage goes to whoever integrates it more completely into their actual workflow, not whoever has access to a more advanced model.
That symmetry is also why AI in cybersecurity introduces its own risks on the defensive side. AI systems can generate false positives that bury real signal in noise if poorly tuned, inherit bias from incomplete training data and miss entire classes of threats as a result, and can themselves be targeted through adversarial techniques, where an attacker deliberately feeds a detection model misleading data to manipulate what it flags. None of that is a reason to avoid AI defensively. It’s a reason to keep a human in the loop for judgment calls, and to treat the AI system itself as something that needs monitoring, not something you deploy once and trust indefinitely.
What this AI security guide means for your team
Five moves separate teams that are getting real value from AI in cybersecurity from teams that have just added another tool:
- Don’t wait for “AI-powered” attacks to become the majority before you invest defensively. The phishing data above shows how fast a minority tactic can become the dominant one. Build detection capacity before the surge, not during it.
- Point AI at your triage queue first, not just your perimeter. The fastest, most measurable win for most teams is cutting the volume of false positives an analyst has to manually clear, not adding a new detection layer.
- Treat deepfake and voice-clone risk as a process problem, not a technology problem. No detection tool fully closes the Arup-style gap. Out-of-band verification for any request involving money movement or credential changes.
- Extend the same AI-driven behavioral thinking to code and pipelines, not just network traffic. Attackers increasingly target the software supply chain directly (malicious packages, hallucinated dependencies, unreviewed agent instructions), a risk category with its own dynamics, covered in depth in our AI Supply Chain Security guide.
- Keep a human accountable for what the AI decides. Automated response is valuable exactly because it’s fast, which is also why it needs a clear escalation path when it gets something wrong.
Where AI in cybersecurity is heading
Two shifts will define the next few years. First, the attacker side is consolidating around fewer, more convincing tactics rather than more numerous crude ones: AI-generated phishing and deepfakes aren’t a wider net; they’re a sharper hook. Second, the defender side is moving from “detect and alert a human” toward “detect, score, and act before a ticket exists,” which is exactly the shift that closes the gap between the 2,992 daily alerts most SOCs face and the fraction of those that ever get a real look. The organizations pulling ahead are the ones applying that shift everywhere AI already touches their environment, including the code their developers ship, not just the traffic crossing their network perimeter.
Xygeni’s take: AI security has to cover code, not just the inbox
Most AI-in-cybersecurity coverage stops at the SOC: phishing, malware, network anomalies. That’s necessary, and it’s also incomplete, because a growing share of AI-driven risk now lives inside the software development lifecycle itself, where SOC tooling doesn’t look.
Xygeni’s CoreAI applies the same behavioral, anomaly-based thinking described in this guide to application security specifically: correlating findings across your existing scanners (not just Xygeni’s own), scoring what’s actually exploitable instead of flooding teams with raw alert volume, explaining why a given risk matters, and generating a remediation pull request instead of another backlog ticket. DevAI extends that same defensive posture into the newest part of the attack surface: AI coding agents and assistants, validating skill files and agent instructions, and blocking a malicious install before an agent ever acts on it. The goal is the same one this guide argues for at the SOC level, applied to the pipeline: turn AI’s advantage over volume and speed into a defender’s advantage instead of an attacker’s.
Start free. Sign up with GitHub, GitLab, or Google and get visibility into up to 25 repositories and 50 AI scans a month at no cost, no credit card required.
FAQ
What is AI in cybersecurity?
AI in cybersecurity is the use of machine learning, natural language processing, and large language models on both the attacking and defending sides of security. Attackers use it to generate convincing phishing content, deepfakes, and adaptive malware; defenders use the same underlying techniques for anomaly detection, alert triage, and automated response.
Is AI more dangerous as an attack tool or more useful as a defense tool?
Neither side has a durable advantage because both are built on the same underlying capability. The organizations at greater risk are the ones that haven’t yet applied AI defensively at the same pace attackers have applied it offensively, not the ones facing a technology gap they can’t close.
Can AI fully replace human analysts in a SOC?
No, and the risks of trying to (false positives from poorly tuned models, bias from incomplete training data, and adversarial manipulation of the detection system itself) are exactly why human oversight stays necessary. AI’s real value is clearing the queue so analysts can focus on the judgment calls that still need a person.
How is AI-driven attack risk different for software development teams specifically?
Beyond phishing and deepfakes, development teams face AI-specific risks inside the SDLC itself: AI coding agents installing hallucinated or malicious dependencies, unreviewed agent instructions in skill files, and AI-generated code that reproduces hardcoded secrets. That side of AI in cybersecurity is covered in more depth in our AI Supply Chain Security guide.
