Automated Scanning vs the OWASP Top Ten

The OWASP Top Ten is a list of the most critical website security flaws – a list also often used as a minimum standard for website vulnerability assessment (VA) and compliance. There is an ongoing industry dialog about the possibility of identifying the OWASP Top Ten in a purely automated fashion (scanning). People frequently ask what can and can’t be found using either white box or black box scanners. This is important because a single missed vulnerability, or more accurately exploited vulnerability, can cause an organization significant financial harm. Proper expectations must be set when it comes to the various vulnerability assessment solutions.

For our part, WhiteHat Security is in the website security business and provides a vulnerability management service. Our Sentinel Service incorporates expert analysis with proprietary scanning technology. Using a black box process, we assess hundreds of websites a month, more than anyone in the industry. What we’ve come to understand is that a significant portion of vulnerabilities are virtually impossible for scanners to find. By the same token, even the most seasoned Web security experts cannot find many issues in a reliable and consistent manner. To achieve full vulnerability coverage and therefore complete vulnerability management, we must rely on a combination and integration of both methods.

We’d like to share some of our experiences that led to this conclusion. Using situations we’ve seen in the real world, and the OWASP Top Ten as a baseline, we’ll demonstrate why scanning technology alone cannot find the OWASP Top Ten. To begin, we’ll focus on a single feature of a fictitious Web Bank responsible for funds transfers from one account to another account. Here is the full URL:

http://server/transfer.cgi?from_acct=1235813&to_acct=31415&amount=
1000.00&session=1001

The “from_acct” is the current user’s account number. “to_acct” is where the money should be sent. “Amount” is obviously the transfer amount, and the “session” is the authenticated session ID after having properly logged-in. This is a fairly typical and straightforward business process.


Unvalidated Input

Scanners must hazard a guess about what “transfer.cgi” does. Otherwise, it would be impossible to determine what it should NOT do.

A website security expert can easily figure this out, but scanners aren’t equipped with that intelligence: There is no knowledge of or appreciation for context. For the sake of discussion, let’s say a scanner has the ability, because there’s a dollar figure present and the “transfer” keyword in the URL might help it decide that this feature moves money. Realistically, these parameter names could be anything and are often far more cryptic. To attempt a classic funds transfer attack, let’s change the above URL substituting the “1000.00” amount to “-1000.00.”



Negative Amount Example:
http://server/transfer.cgi?from_acct=1235813&to_acct=31415&amount=-1000.00&session=1001

By transferring a negative amount, this custom Web application would potentially deduct money from the target account instead of adding to it! The challenge for a scanner is being able to decide whether or not the attack succeeded. How would it tell?

If the fraudulent transfer succeeded, the website might respond with, “Success, would you like to make another transaction?,” “Transfer will take place by 9 AM tomorrow,” “Request received, thank you,” or any number of possible affirmations. If the attack failed, “Transfer failed,” “Error: Transfer amount must be a positive number,” or, “Bank robbery detected, men with guns have been dispatched to your location!” Every custom Web Bank application will likely respond in a different manner. That’s precisely the problem! Pre-programming all the possible keyword phrases or behavioral aspects is simply unfeasible and for all mathematical provability, impossible. However, human gray matter (or, a crack website security operations team) can make this determination.