Listen to this Post
Introduction: AI Is Changing Cybersecurity, But Accuracy Remains the Biggest Battlefield
Artificial intelligence has transformed nearly every aspect of software development. From writing code to automating documentation and accelerating security reviews, Large Language Models (LLMs) promise a future where developers spend less time on repetitive tasks and more time building secure applications.
Yet one of
While vendors increasingly advertise AI-powered security scanners capable of identifying software weaknesses within seconds, reality paints a more complicated picture. Security professionals continue to face overwhelming numbers of alerts, inconsistent results, and false positives that consume valuable engineering time.
Recent research presented ahead of Black Hat USA reveals a concerning truth: despite their impressive capabilities, today’s LLMs still lack the contextual understanding necessary to reliably distinguish dangerous vulnerabilities from harmless code patterns. Instead of reducing workloads, many organizations are discovering that AI sometimes creates even more work.
AI Is Finding Too Many Problems That
Security teams depend on application scanning tools to identify weaknesses before attackers do. The expectation is simple: scanners should highlight genuine risks while filtering out irrelevant findings.
Unfortunately, current AI-assisted scanning solutions often fail at this balancing act.
Testing across more than a dozen application security tools revealed that over 60% of reported vulnerabilities ultimately fall into one of several problematic categories:
False positives
Low-severity findings
Vulnerabilities located in unreachable code
Issues that pose little practical security risk
Rather than simplifying vulnerability management, these inaccurate reports overwhelm security teams with alerts that require manual verification.
As organizations continue producing increasingly complex software, this flood of questionable findings creates a serious operational burden.
Security Teams Face an Impossible Choice
Modern development environments generate thousands of dependency updates and vulnerability notifications every month.
Automated dependency management systems continuously create pull requests intended to patch newly disclosed vulnerabilities. While automation accelerates updates, it also creates alert fatigue.
Organizations are left with two undesirable options:
Accept every automated recommendation, potentially introducing unnecessary changes into production.
Manually inspect every reported vulnerability, an impossible task for most security teams.
Neither approach scales effectively.
With cybersecurity talent already in short supply, manually reviewing thousands of alerts becomes economically unsustainable.
The Explosion of Software Vulnerabilities
The challenge continues to grow.
According to industry estimates, the number of publicly assigned Common Vulnerabilities and Exposures (CVEs) is expected to increase dramatically, with projections showing approximately a 50% increase compared to previous years.
Meanwhile:
Microsoft continues releasing record-breaking Patch Tuesday updates.
Open-source software dependencies continue expanding.
Cloud-native applications introduce additional complexity.
AI-assisted software development accelerates code production.
As software ecosystems grow, vulnerability management becomes exponentially harder.
Ironically, AI-generated code also increases the amount of software requiring security analysis.
Why Large Language Models Fall Short
Although LLMs excel at natural language understanding and code generation, vulnerability analysis requires far more than pattern recognition.
Security decisions depend heavily on context.
An AI model may correctly identify that a cryptographic function appears in code, but it often cannot determine:
Why it exists
Whether it is actually executed
How sensitive the protected data is
Whether surrounding security controls eliminate the risk
Without this contextual awareness, AI frequently recommends fixing issues that are not vulnerabilities at all.
This reduces developer confidence in AI-generated security advice.
Context Matters More Than Detection
Finding suspicious code is only the beginning.
Correct vulnerability classification requires understanding three different layers of context.
Organizational Context
Every company has different security priorities.
A vulnerability considered critical inside a financial institution may represent only a minor concern for an internal testing application.
Business impact cannot be inferred from code alone.
Technical Context
Security tools must understand how software is deployed.
Questions include:
Is the vulnerable component internet-facing?
Does authentication protect it?
Can attackers realistically reach the vulnerable function?
Ignoring deployment architecture results in inflated severity ratings.
Application Context
Applications contain millions of lines of interconnected code.
Analyzing one file without understanding surrounding components frequently leads to incorrect conclusions.
Real vulnerability assessment requires whole-program reasoning rather than isolated code inspection.
The MD5 Example Shows Why Context Is Everything
One commonly cited example involves the MD5 hashing algorithm.
Traditional scanners often classify MD5 usage as a medium-severity issue.
However, reality is far more nuanced.
If MD5 protects passwords or confidential information, the vulnerability is severe because attackers can exploit weaknesses in the algorithm.
If the same MD5 function simply generates file identifiers or non-security checksums, the issue may be harmless.
The code appears identical.
Its purpose changes everything.
Current LLMs frequently fail to make this distinction.
Reachability Is the Missing Piece
One of the most effective methods for reducing unnecessary security work is analyzing reachability.
Reachability determines whether vulnerable code can actually execute during normal application operation.
Research shows remarkable statistics.
Approximately 62% of open-source libraries included in applications are never executed at runtime.
Similarly, although around 71% of Java applications consist of open-source code, only about 12% is actually used while the application runs.
This means many reported vulnerabilities exist inside code paths users never reach.
Without reachability analysis, scanners continue producing alerts for software components that cannot realistically be exploited.
AI Still Produces Inconsistent Results
Another challenge is reproducibility.
Traditional security scanners generally produce consistent findings when executed repeatedly against identical code.
LLMs behave differently.
Running the same model against the same application multiple times may produce different vulnerability classifications.
One execution may label an issue as a false positive.
Another may identify the exact same code as a critical vulnerability.
This inconsistency undermines trust among developers and security teams.
Reliable security requires deterministic results.
Current LLMs often struggle to provide that consistency.
Developers Need AI, But They Also Need Guardrails
Rather than replacing traditional security tools, AI performs best when integrated into carefully designed workflows.
Experts increasingly recommend combining LLMs with:
Static Application Security Testing (SAST)
Dynamic Application Security Testing (DAST)
Software Composition Analysis (SCA)
Reachability analysis
Runtime telemetry
Human security review
AI should enhance existing workflows rather than operate independently.
Human expertise remains essential for validating high-impact security decisions.
Deep Analysis
How Security Teams Can Improve Vulnerability Triage
Modern AppSec teams can significantly reduce false positives by combining multiple security testing techniques instead of relying solely on LLM recommendations.
Run Semgrep for Static Analysis
semgrep scan --config auto .
Analyze Open Source Dependencies
syft packages dir:.
Search for Known Vulnerabilities
grype dir:. Perform Reachability Analysis (Java Example)
mvn dependency:tree jdeps -recursive target/application.jar Audit Node.js Dependencies
npm audit npm audit fix
Scan Python Packages
pip-audit
Container Image Security Scan
trivy image myapp:latest Generate a Software Bill of Materials (SBOM)
syft dir:. -o cyclonedx-json
Identify Unused Dependencies
depcheck
Continuous Security Pipeline
SAST ↓
Dependency Scan
↓
Reachability Analysis
↓
LLM Classification
↓
Human Validation
↓
Patch Deployment
This layered approach dramatically improves accuracy by combining deterministic scanners with contextual human review, allowing LLMs to act as assistants rather than final decision-makers.
What Undercode Say
The excitement surrounding AI in cybersecurity has created unrealistic expectations. Many organizations assumed LLMs would quickly replace traditional vulnerability management, but this research demonstrates that security remains deeply dependent on context rather than raw intelligence.
One of the biggest misconceptions is that identifying vulnerable code automatically means identifying exploitable software. In reality, exploitability depends on runtime behavior, deployment architecture, authentication layers, network exposure, and business logic. These factors are difficult for generalized language models to infer.
False positives are more than an inconvenience. Every unnecessary alert consumes developer time, delays software releases, and contributes to security fatigue. Eventually, teams begin ignoring scanner output altogether, increasing the likelihood that genuine threats are overlooked.
Reachability analysis may become one of the most important technologies in modern AppSec. Instead of asking whether vulnerable code exists, organizations increasingly ask whether attackers can realistically execute it. This simple shift dramatically reduces remediation workloads.
Another concern is AI determinism. Enterprise security processes depend on repeatable, auditable results. If an LLM reaches different conclusions each time it analyzes identical code, it becomes difficult to justify automated remediation or compliance decisions.
The future of AI-assisted security will likely involve specialized models trained exclusively for software security rather than general-purpose conversational systems. These models will be paired with static analyzers, runtime monitoring, software inventories, and policy engines that provide the missing operational context.
Organizations should also invest in Software Bill of Materials (SBOM) generation, dependency mapping, and runtime observability. Feeding structured metadata into AI systems will substantially improve prioritization accuracy.
Human expertise remains irreplaceable. Experienced security engineers understand architectural intent, business risk, and exploitation feasibility in ways that current LLMs cannot consistently replicate.
The most successful AppSec programs will not eliminate humans from the loop. Instead, they will automate repetitive analysis while reserving strategic decisions for experienced professionals.
Ultimately, AI should function as a force multiplier rather than an autonomous security analyst. The goal is not replacing judgment, but amplifying it with faster data processing and broader code visibility.
Prediction
(+1) AI Will Become More Context-Aware, Not More Autonomous
Within the next several years, vulnerability classification platforms will increasingly combine LLM reasoning with runtime telemetry, reachability analysis, SBOM data, and organizational policies. This hybrid approach will significantly reduce false positives while improving developer trust. Rather than replacing security professionals, AI will evolve into an intelligent assistant capable of accelerating triage, prioritizing genuine threats, and helping organizations respond faster to an ever-growing volume of software vulnerabilities. 🚀
✅ Accurate: Current application security tools continue to generate high false-positive rates, and vulnerability prioritization remains a major operational challenge for security teams.
✅ Accurate: Context, including runtime reachability, deployment architecture, and business purpose, is essential for correctly determining the real severity of software vulnerabilities.
✅ Accurate: While modern LLMs are increasingly capable of assisting with code analysis, they still require deterministic tooling and human oversight before organizations can safely rely on them for vulnerability classification and remediation decisions.
▶️ Related Video (78% Match):
🕵️📝Let’s dive deep and fact‑check.
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
References:
Reported By: www.darkreading.com
Extra Source Hub (Possible Sources for article):
https://www.reddit.com/r/AskReddit
Wikipedia
OpenAi & Undercode AI
Image Source:
Unsplash
Undercode AI DI v2
🔐JOIN OUR CYBER WORLD [ CVE News • HackMonitor • UndercodeNews ]
📢 Follow UndercodeNews & Stay Tuned:
𝕏 formerly Twitter 🐦 | @ Threads | 🔗 Linkedin | 🦋BlueSky | 🐘Mastodon | 📺Youtube
