During World War II, Abraham Wald was asked where to add armor to bomber planes. The military's instinct was to study the planes that came back and reinforce wherever the bullet holes were clustered. Wald said to do the opposite. The planes in front of him had already survived hits to those spots. That's exactly why he could study them. The planes that took hits to the engine or the cockpit never made it back to be studied at all.
Detection engineering has the same blind spot, and it's easy to miss because the surviving planes look like a complete picture if you never ask what's not in front of you. Every rule we write and every pattern we call "known bad" is drawn from attacker packages that made it back to us, which means they got caught. We're studying bullet holes on the planes that returned and calling it the full picture of what's out there. Criminologists have a name for the version of this they deal with: the dark figure of crime. This is the space between the incidents that get detected and the incidents that happen. Every crime statistic is built on the criminals who got caught. The ones who study it know that and factor it in.
The implication for rule writing is uncomfortable but important. String level rules can only catch attackers who reused artifacts we have already seen. They can’t, by construction, catch an attacker who is sophisticated enough to have never appeared in our tracking data, the same way no amount of studying returned planes will ever show you where the planes that didn't return got hit. Behavioral level rules are our only real shot at the unknown. The attacker still has to exfiltrate the data, still has to execute the code, still has to phone home. What they do is constrained. What they call it, how they obfuscate it, and where they send it is not.
But behavior identification only helps if we know what normal is supposed to look like in the first place. Wald could point to the missing planes because he already knew what a whole, undamaged plane looked like. That's what let him map every bullet hole on every returning plane against a complete expected shape, wings, fuselage, tail, engine, cockpit, all of it. Once he had that full map, the thing that stood out wasn't where the damage was. It was where it wasn't.
Detection engineering doesn't get that for free. If we don't have a clear, complete picture of what normal package behavior looks like, an unexpected network call or an unfamiliar process spawn doesn't register as a hole in anything. It just looks like more noise in a system that isn’t mapped.
The real cost of an incomplete behavior data set goes beyond a missing rule or two. It's that we lose the ability to even recognize the shape of what's missing. An attacker who never appears in our string-based detections still has to act, and that action is only visible as anomalous if we've defined the baseline it's deviating from. Without that baseline, the invisible packages stay invisible for a second reason on top of the first. Not only did they avoid leaving a signature, but they're also operating in the space we never bothered to describe, so nothing about them ever looks wrong unless we look at the capabilities and behaviors of packages.
Cheap string and IOC rules are a rational use of effort. The strategies used in the overwhelming majority of malicious packages are low sophistication and reused, with the same payload copied across a whole wave of packages. A throwaway string rule catches that entire wave at almost no cost, dies the moment the attacker rotates, and gets rewritten just as cheaply. Cheap string rules “dying young” is the cheap tier doing exactly the job it was built for. Demanding an expensive behavioral baseline for threats that a five-minute string rule already stops would be an inefficient use of time.
The answer is a portfolio of rules across the whole pyramid. Cheap low-tier rules handle the high volume. Expensive high-tier rules are the only reaches we have into the sophisticated few hiding in the dark figure. Different tiers, different jobs, and which one you reach for depends entirely on which attacker you are trying to catch. That is exactly what the following adapted pyramid lays out.
An adapted pyramid
David Bianco's pyramid of pain [1] gives us a way to think about this in layers instead of treating detection as one flat category, adapted here for the package-supply-chain context. We can rank indicator types by how much it costs an attacker to change them, from trivial to nearly impossible. This ranges from happenstance occurrences like hash values to overtly deliberate behaviors like techniques and strategies.
Every level except the top is something the attacker chose. A domain, an encoding scheme, even a tool, all of it is a decision, and decisions get remade the moment they stop working for the attacker. That's what makes the highest level of the pyramid the place to build durable rules, and it's also exactly the hardest place to write one, since technique and strategy-level detections need a real baseline behind them or they don't work at all.
Which raises the obvious question of where that baseline comes from, because the pyramid does not hand it to us. Ranking indicators by how costly they are to change tells us where the durable rules live, not how to see a deviation in the first place. The most practical way to build that picture is to stop looking for one universal baseline (which does not exist) and instead map normal behavior by package type. A build tool, a command line utility, a web framework, and a tiny pure function utility are different kinds of software with different capabilities.
A build tool that spawns child processes and touches the filesystem on install is behaving as expected, while a string formatting helper doing the same thing is not. The package's capabilities and actions determine the acceptable behavior, and the same package categories tend to behave alike across npm, PyPI, and NuGet, so the profile is stable enough to reuse. If we get that map complete enough, an unfamiliar action stops reading as noise and starts reading as a hole in the expected shape, which is the whole point. It is a large amount of work, and the map will always be partial, but it is the work that makes the top of the pyramid reachable at all.
Mapping the shape
Every package Aikido Security ingests gets scanned statically and runs in a sandbox that records what it actually did. That part is ordinary. What changed is that an indicator now carries structure beyond the fact that it fired. Each one describes what the matched code is able to do and what kind of evidence it represents, so a signal that means little alone can be weighed against the others around it.
That lets a detection rule describe a shape rather than a string. Code that runs at install time, reaching for credentials it has no reason to touch, sending something outward, in a package whose stated job explains none of it. There is no domain in a rule like that, no hash, no package name. Nothing in it for an attacker to rotate.
On August 4th 2026, we flagged a newly published version of Keyv, which presented itself as the same popular and trusted key value utility that has been downloaded 600 million times from NPM. This time, however, on install it executed a heavily obfuscated JavaScript file, which then downloaded Bun (an alternative JavaScript runner instead of Node.js), and finally executed another obfuscated JavaScript file. Those behavioral tools and techniques were enough for our system to pick it up. If we were just using named indicators, we would have missed this package. But because we’re looking for the unexpected, we’re able to catch these packages that otherwise would fall into that dark space.
Our system correctly concluded the following even though it couldn't see the whole picture due to the heavy obfuscation: “The combination of concealed install-time execution, remote binary retrieval, payload staging, and launch is sufficient for a high-confidence malware verdict even though the final payload behavior and any data-theft objective were not fully decoded”.
It is currently and has been, at the very least, difficult to decode and understand obfuscated malware, especially from a static analysis point of view. That’s why the attackers put it there after all. What we can prove instead is that the behavior of malicious packages is enough for us to find the dark figures.
Some small dark figure still exists. Wald never saw the downed planes either. What let him work out where they had been hit was knowing the shape of a whole plane well enough to notice where the pattern broke. Our version of that shape is a baseline of what ordinary package behavior looks like, kept current rather than assumed, and every package we triage adds to it. The attackers we have not caught yet are still constrained by what code has to do to steal something. Each of those constraints is a hole we already know the location of on a plane that hasn't come back.
See what we're catching on Aikido Intel. We detect malware in minutes. Connect your repo to Aikido to continuously scan your app dependencies for malicious packages.
References
[1] Bianco, D. J. (2013). The Pyramid of Pain. Enterprise Detection and Response. http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html (revised January 17, 2014, adding the hash values tier).
