Open source has become the foundation of modern software development. Nearly every application today relies on a complex web of third-party libraries, frameworks, models, and build tools. This reality alone already introduces significant software supply chain security challenges. At the same time, artificial intelligence has entered the software development lifecycle as a powerful accelerator, generating code, suggesting dependencies, automating fixes, and even influencing architectural decisions. Together, open source and AI have transformed how software is built, and, inevitably, how it is attacked. The intersection of AI security, AI and software security, and software supply chain security is no longer theoretical. It is now one of the dominant sources of software supply chain risk faced by engineering organizations.
That reality framed our recent SafeDev Talk: Open Source, AI & the New Attack Surface: Weaponized Code, Smarter Defenses, featuring security leaders from Red Hat, TikTok, and Xygeni. The discussion focused on what security and engineering teams are already experiencing in production environments, particularly around open source supply chain attacks, malicious open source packages, and the growing tension between speed and control in AI-driven software development. What emerged was a clear picture: the attack surface is expanding faster than traditional security models can keep up, and AI is acting both as a force multiplier and a stress test for long-standing assumptions in AI security and software supply chain security.
If this description feels uncomfortably close to how your organization currently builds software, that is not a coincidence. Many teams only realize how much trust has shifted to automation after something breaks.
AI Security and Software Supply Chain Security Are Now the Same Problem
A recurring theme throughout the discussion was that AI security can no longer be treated as a separate discipline from software supply chain security. AI systems do not operate in isolation; they are built, trained, deployed, and integrated through the same pipelines, dependencies, and registries that already struggle with open source supply chain attacks.
In AI-driven software development, models suggest code, generate fixes, and select dependencies automatically. These decisions directly affect open source dependency management, often without explicit human intent. As a result, dependency risk is no longer driven solely by developer choice; it is increasingly shaped by AI behavior.
This convergence means that failures in AI and software security often manifest as traditional supply chain incidents: compromised dependencies, tainted build artifacts, or vulnerable CI/CD processes. The tooling may be new, but the software supply chain risk is very real and increasingly difficult to reason about.
If your threat models still separate “AI risk” from “supply chain risk,” it may be worth revisiting where that boundary actually exists in your build and deployment workflows.
Open Source Supply Chain Attacks at Machine Speed
Open source supply chain attacks are not new, but AI changes their economics. Attackers do not need novel techniques; they need scale. AI enables rapid ecosystem analysis, automated discovery of weak dependencies, and fast iteration on attack payloads.
From an offensive standpoint, this industrialization of reconnaissance dramatically increases the success rate of attacks involving malicious open source packages. Components that would previously have gone unnoticed can now be discovered, analyzed, and exploited quickly, often before defenders realize they are in use.
This is why software supply chain security cannot rely on delayed signals alone. Registries, advisories, and post-facto disclosures operate on human timescales, while attackers increasingly operate at machine speed. The resulting exposure window is a direct contributor to growing software supply chain risk.
If your primary detection signal is “the registry removed the package,” you are already operating downstream of the attacker’s timeline.
Want to a Deep Dive into Open Source Software Supply Chain Attacks?
Dependency Risk in AI-Driven Software Development
One of the clearest risks discussed during the SafeDev Talk was dependency risk, particularly in environments that rely heavily on AI-driven software development. AI coding assistants are optimized for convenience and speed, not for minimizing attack surface.
In practice, this leads to aggressive dependency introduction. New libraries are added instead of reusing existing functionality, transitive dependencies expand silently, and open source dependency management becomes reactive rather than intentional. Over time, teams lose the ability to reason about what they are actually running.
This is not simply a hygiene issue. Each new dependency introduces additional software supply chain risk, new trust assumptions, and new opportunities for open source supply chain attacks. When dependency decisions are automated and reviewed superficially, dependency risk becomes systemic rather than accidental.
If your dependency graph is growing faster than your team’s ability to explain it, this is not a tooling problem; it is a trust problem.
AI Coding Assistants, Security, and the Collapse of Review
Another failure mode discussed was the erosion of peer review in the presence of AI-generated code. AI coding assistants, security is not just about prompt injection or model misuse; it is about how much unreviewed logic enters production systems.
AI-generated changes are often large, coherent, and difficult to review under time pressure. As a result, peer review becomes shallow or symbolic. This quiet collapse removes one of the most effective controls in software supply chain security.
The problem is not developer negligence. It is workflow misalignment. When speed is rewarded and friction is penalized, AI and software security controls that depend on human attention inevitably weaken. Attackers do not need to bypass review if review no longer functions as a barrier.
Many teams assume review still works because the process exists. Fewer ask whether it still functions as a meaningful control.
Malicious Open Source Packages and the Myth of Popularity
A common belief in open source dependency management is that popular projects are safer. In reality, popularity often increases exposure. Widely used libraries are high-value targets for open source supply chain attacks, precisely because compromise yields broad downstream impact.
Many popular projects are maintained by small teams or single individuals. Even when issues are detected, malicious open source packages often remain available for hours or days before removal. During that time, organizations continue to ingest them through automated builds.
This delay reinforces the need for proactive software supply chain security controls. Relying on popularity, reputation, or registry action alone is insufficient when facing modern software supply chain risk.
“Widely used” is not the same as “actively defended,” and treating it as such is one of the most persistent supply chain misconceptions.
Provenance in Software Supply Chains and AI Security
Throughout the discussion, the need for provenance in software supply chains emerged repeatedly. In AI-assisted environments, attribution becomes blurred. Code may be generated by a model, modified by a human, merged by automation, and deployed without clear accountability.
Without verifiable provenance, organizations are forced to trust artifacts implicitly. AI security demands a shift away from trust toward verification: signed artifacts, build attestations, and traceable origins. While provenance does not prevent malicious behavior outright, it significantly reduces ambiguity and limits attacker maneuverability.
This applies equally to models, data, and code. In AI-driven software development, provenance is a foundational requirement for both AI and software security.
SBOM and AI Security in Modern Pipelines
The role of SBOM and AI security was another implicit theme. SBOMs provide visibility into dependency graphs, but visibility alone is not enough. In AI-heavy environments, SBOMs must evolve to capture not just libraries, but models, build steps, and automated decisions.
When combined with behavior analysis and provenance, SBOM and AI security become powerful tools for reducing software supply chain risk. They allow organizations to detect unexpected changes, reason about impact, and respond more effectively to open source supply chain attacks.
CI/CD Pipeline Security Under Automation Pressure
Finally, CI/CD pipeline security emerged as a critical control plane. Pipelines increasingly execute actions suggested or triggered by AI systems. If those pipelines lack strong identity controls, artifact verification, and policy enforcement, they become ideal entry points for attackers.
Inadequate CI/CD pipeline security allows malicious open source packages to affect not only production systems but also developer environments and build infrastructure. As automation increases, pipelines must be treated as high-value assets within software supply chain security programs.
Watch the SafeDev Talk
To hear more about all these insights directly from the practitioners shaping the field, watch the full SafeDev Talk: Open Source, AI & the New Attack Surface: Weaponized Code, Smarter Defenses, featuring Roman Zhukov (Red Hat), Leon Johnson (TikTok), and Luis Rodríguez Berzosa (Xygeni).
Practical Implications for AI Security and Software Supply Chain Security
The practical implications of these shifts extend beyond tooling. Organizations must recognize that AI security, AI and software security, and software supply chain security are now deeply intertwined. Decisions that were once considered low-risk, dependency updates, code generation, and automation now carry meaningful software supply chain risk, especially when those decisions are made implicitly by tools rather than explicitly by people.
During the SafeDev Talk, this point was summarized succinctly. As one speaker put it, when AI systems participate in software development, security teams are no longer just securing code; they are securing decisions. Automation does not remove responsibility; it redistributes it.
In practice, this means restoring intentionality where convenience has taken over. Open source dependency management must account for AI-driven behavior rather than assuming human deliberation. Dependency risk can no longer be treated as an occasional review exercise. CI/CD pipeline security must enforce verification, not assume benign input. And provenance in software supply chains must move from aspiration to baseline.
Another insight from the discussion was that speed itself is no longer neutral. Most supply chain failures do not come from a single catastrophic decision, but from many small automated choices that no one explicitly approved. This is precisely why traditional trust models fail under AI-driven software development.
None of this implies abandoning open source or AI. On the contrary, it acknowledges their central role in modern engineering. But without evolving security assumptions, organizations risk letting automation define trust by default.
To Conclude…
A useful way to think about this shift is that software supply chain security is no longer about protecting artifacts alone. It is about protecting decision paths. In an AI-assisted world, the most important security questions are not only “Is this component vulnerable?” but “Why was this introduced, by whom or what, and under which constraints?” Organizations that adapt to this framing will not eliminate risk, but they will be far less surprised by it.
