One of the first steps of a black-box penetration testing of a website is mapping its files and directories.  And in order to do that, security scanners crawl into the website first, and then try to guess the possible directories and files. These scanners use the response header or body of the page to investigate a valid file or directory. For instance, the header status “404” can be the sign of “File Not Found” and “200” can be the sign of a valid file. Also, the status “403 Forbidden” can be the sign of a valid directory without any index page. However, many websites such as Yahoo, Google, Facebook, Microsoft, and so on do not like to show the “403 Forbidden” errors for a valid directory, and instead, they show a “Page Not found” or another default page to the users. Although this functionality makes the website more user-friendly, it is not good for the scanners at all; as there is no difference between a valid and an invalid directory then.

Therefore, we need something else as a signature to improve the scanners result. And as a solution we can use a “/” as an identifier. In case of requesting a valid directory without adding a slash at the end of it, the web-server will add an slash automatically, and in case of having an invalid directory there will not be any slash at the end of the directory name.

Some examples:

Invalid Directory: http://www.microsoft.com/foobars

Valid Directory: http://www.microsoft.com/test

——–

Invalid Directory: http://code.google.com/foobars

Valid Directory: http://code.google.com/js

——–

Invalid Directory: http://www.facebook.com/foobars

Valid Directory: http://www.facebook.com/admin

——–

Invalid Directory: http://uk.yahoo.com/foobars

Valid Directory: http://uk.yahoo.com/private

——–

Cheers,

Soroush Dalili