Error Codes as a means for Information Gathering

Our next step is called Analysis of Error Codes.  In this stage, we are using common responses to certain errors as a way to get information.   One common error that pretty much all internet users has ran into at least once is the HTTP 404 error.

Example, if someone goes to the following url:  http://r.martint.net/asdads.html - they will be greeted with:

Not FoundThe requested URL /asdads.html was not found on this server.Apache/2.2.3 (CentOS) Server at r.martint.net Port 80

So, this error tells me the page is hosted on a Apache web server on port 80 running CentOS (Linux Variant).

Wen servers are not the only type of servers affected by this - people often use common errors from databases for penetration testing.  Some database errors can give us the IP Address of the database server, and then we try SQL injection to fool the server into authenticating our request.

Analysis of Error codes ends our discussion on Information gathering and now it is time to move on to configuration management testing.  More on this will be posted in the next post.

Penetration Testing #3

It's always good to know which applications are installed on a server before trying to exploit it.  Many applications already have existing vulnerabilities  and other applications are not updated by the company for various reasons.  Due to virtual web servers, many websites are hosted under one IP address. This means that to access a certain application, you would have to go through the DNS name rather than the numeric IP.  Finding all the web applications on a web server through whatever means is the definition of web application discovery.

Once we are given a target (a DNS name or IP address), there are three factors that influences finding  applications that are hosted on that target. First factor is a Different Base URL. The base URL is the target given to us (the DNS name). So if our target is http://google.com, the base URL is google.com, but what if the application is hosted on http://google.com/app1/?  Secondly, the use of non-standard ports. By default, people use port 80 for web servers , however what happens if people start using random ports? Lastly, we have the idea of virtual hosts.  That is, multiple domains pointing to one IP Address.  So, how do we find solutions for the above situations?

It's difficult to find applications if they are installed somewhere other than the base URL. The best way to find is through search engines (if application is on the internet) or through intelligent guessing (for example, a mail application could be installed in a directory called 'mail').  If an application is ran on a non-standard port, we can easily find it through running a port scanner like nmap. Lastly, the best way to combat the virtual host setup would be using a reverse IP tool such as the one provided by domaintools.com. The reverse IP tool works by just entering an IP address and it tries to list all the domains hosted on that IP address. 

Penetration Testing #2

One great way to find information is through the public domain.  There are plenty of "private" websites that can be found using search engines. This happens if a company does not setup their robots.txt file correctly and allows search engines to crawl their private website (*note - robots.txt can be completely ignored by search engines, but reputable search engines like Google follows the guidelines). Let's say you found a "private" website - now how do we get more information? Well, using the advanced search operators, we can restrict our searches to a specific website (example) or (cache example ). Once we have access to this information, we must now find points of entry to the web application.  Once such way to accomplish is through analyzing GET and POST (for more sensitive information) requests through "an intercepting proxy" (61).  An example of a post and get requests can be seen below:

• GET: http://domain.com/shopping/buyproduct.php?customerid=1&item=1001$quantity=5
• We can pretty much change the value of the parameters any given time.

• POST: http://domain.com/shopping/accounts.php?action=login
• The parameters are sent through the body of the message rather than in the URL.

Tools like Burp Suite or OWASP's WebScarab are great intercepting proxy tools for analyzing these requests. After finding potential entry, it's time to move on to fingerprinting.  In this stage, our goal is figure out what web server the web application is running. This is important because different commands will react differently on each web server and that is exactly how we will figure out the web server type.  For example, each webserver has a different order in how they display header information.  If one sends a simple HTTP request, the date and time parameters of the http header will be in different order in each web server.  Another way is to simply use a tool such as httprint.

Our next step is figure out which web application is hosted on a specific web server which I will cover in the next blog post.  

Penetration Testing #1

A penetration test is one method of finding security vulnerabilities of a web application. It works by simulating potential web attacks. So the test is done from the perspective of a potential intruder. This means that the source code of the web application is not used as a resource to find weaknesses. Before continuing, it is important to know the relationship between a vulnerability and a threat. A vulnerability is a flaw or weakness in the application. A threat is an attack that exploits the vulnerability.

Penetration testing has two modes:
Passive mode: this is more information gathering. We are trying to find more about the system while also looking for access points of the application. We can use various tools for information gathering.
For example, let's say that a user is trying to login to a system with this url (http://example.com/?login=username=1&pass=2).
In the above system, there are two access points or gates: Username and password. 

The other mode is called Active Mode: This is were we actually do all the different type of penetration testing.
OWASP describes 9 different sub-categories for active mode testing

• Configuration Management testing 
• Business Logic Testing 
• Authentication Testing 
• Authorization Testing 
• Session Management Testing 
• Data Validation Testing 
• Denial of Service Testing 
• Web Service Testing 
• Ajax Testing 

For the rest of this blog post, I'll be focusing on the passive side. As mentioned earlier, the passive mode is all about finding information through various methods. Information gathering can be done by public tools such as search engines or private tools that can be used to connect to the application and send specific information requests. Even something as error messages can reveal a lot of information about the application (version #, database type, etc…). So just how do we find information? The next blog post will have more information about some of these methods.

The OWASP Testing Framework

Before any testing is done, an organization must create a testing framework that consists of "techniques and tasks that are appropriate at various phases of the SDLC" (44).  The activities in the framework should happen:

•  before development begins
• In this stage, it is important to have a solid SDLC already developed.  With a strong SDLC, one must also have the right policies, standards, requirements and documentation ready for the development team.  It is extremely important for the development to team know what exactly they are doing.

• during definition and design
• The requirements and documents that are made in the first stage must be logically tested to make sure they make sense.
• Once the design of a system is completed, it is important to create UML models for it. Models just help everyone in the project to understand how the system works.
• Using the system model, try to create realistic threat models. In other words, if we find a weakness in the system model, we should create threat models that describe how an intruder can penetrate the system - then, try to make sure these weaknesses are patched.

• during development
• The first step in this stage is to perform a code walkthrough where the developers walkthrough the system's code to the security team. This allows the security team to understand what is going on in the code.
• After understanding what the code does, the security team should now review the code to find security flaws.

•  during deployment
• Deployment testing consists of penetration testing and making sure the system is  correctly configured correctly. Although the application could be safe, the system could have been installed improperly.

•  maintenance and operations
• Period health checks should be done on the system as new security flaws could be found with the age of the system.
• If the system needs any changes, make sure the change is well documented and thoroughly tested before implementation.

Foreword, Frontispiece & the OWASP Testing Framework (pages 7-44)Foreword, Frontispiece & the OWASP Testing Framework (pages 7-44)

Foreword
What’s limiting our technological advancements is the security aspect of it. The goal of the Open Web Application Security Project (OWASP) is to take out this limiting factor so technological improvements can continue to progress.  To do this, OWASP created the Testing Guide (v3) for software developers, software testers, and security specialists (IT Audit companies).  It is important for people to actually understand security issues rather than relying on automated tools.  Automatic tools are a great asset to have if they are used as a supplement rather than fully relying on them.  This is due to the fact that generic tools are not to find issues in custom code (ex, applications), but rather, they are suited for an already defined environment (ex, network security).
Frontispiece
OWASP recommends that when viewing application security, we must view it as a “people, process, and technology problem because…security includes improvements in all those areas” (14).
OWASP describes testing “as a process of comparing the state of a system/application against a set of criteria” (16). OWASP wants developers, testers, and specialists to all understand the “criteria” before any testing begins. Testing should be done as a means to eliminate the differences between a company’s security practices and that of the industry’s best practices.
So when should the testing process begin? Surprisingly, most people only test applications once it has been deployed (code already created).  This not only increases headaches, but also costs. OWASP recommends that security testing be done at each phase of the project’s SDLC.
Once security is implemented, we must decide what exactly should be tested. OWASP recommends that we take a holistic approach and test not just the software (technology), but also the people and processes. A lot of companies just test the technology, which leads to problems down the road when the companies’ people and process could include security risks in the technology. This approach “eradicates bugs early and identifies root causes of defects “(17).
Before any testing occurs, one must first understand the principles of security testing
1. There is no silver bullet – There is no one service or a tool that can fully keep your application secure.
2. Think Strategically, not Tactically – Don’t use the patch-and-penetrate model (fixing a bug, but not reporting the root cause of it).
3. SDLC Is King – it is extremely important to incorporate security into each phase of the SDLC.
4. Test Early and Test Often – Find common security issues and test them in each phase of the SDLC.
5. Understand the Scope of Security –  Information needs to be classified depending on how important/valuable they are. For example, personal information should be treated with due care.
6. Develop the right mindset – You have to think outside the box to find some security vulnerabilities. Think like an intruder rather than the average user. Automatic testing tools lack the ability to think outside the box.
7. Understand the Subject -  Accurate documentation of an application must exist (data-flow diagrams, use cases). It is also important to have some type monitoring system setup to monitor intrusions.
8. Use the right tools – Although tools are not perfect, they do help “automate many routine security tasks” (20).
9. The Devil in the Details – Don’t just breeze through the security testing. Go into every possible “section of  [the] application logic” (20).
10. Use Source Code when Available – You shouldn’t just always do a black box testing. Since the source code is there, use it find security flaws.
11. Develop Metrics - it is important to quantify the results to make sure the security testing is actually doing its job.
12. Document the test results - All testing results should be documented for "developers, project managers,  business owners, IT department, audit and compliance" (20). This will allow the right person to make the right decision moving forward.
Testing Techniques
1. Manual Inspections & Review
1. It's the idea of talking to the system owner or looking at the system documentation  and finding out "how something works and why it was implemented a specific way" (21). It's a way to make sure the SDLC works as desired and has  the right policies in place. Identifies threats during development of the system.
2. Threat Modeling
1. Is the idea of finding out potential threats that the web applications could face and allocating resources to  mitigate these threats. Identifies threats during design of the system.
3. Code Review
1. It is one of the best way to test for security concerns. It is the idea of looking at the source code and finding inherent security flaws. A lot of threats cannot be found with simply a black box approach.  Identifies threats during development of the system.
4. Penetration Testing
1. Is the idea of testing an application or network from a user perspective. Source Code is not given and thus this testing is akin to how most hackers will try to penetrate the system. Although it works well for networks and operating systems, it is often a poor choice for application testing because  applications are always custom coded (bespoke) and different exploits exists in different applications.
Which testing to use?
1. OWASP  recommends in using a balance approach (using a combination of the above techniques) when trying to test for security flaws because it will "cover testing in all phases of the SDLC" (25).  Most of the testing should take place in the early stages of the SDLC (define, design…) with emphasis on manual inspections & review and  code review. Again, black box testing is not recommended for web applications.
What are the objectives of testing?
1. The objectives of the testing are placed in the security requirements document . The content of this document derives from our business requirements. For example, if our business is selling products online, then we must have security guidelines to product customer data and also comply with government regulations.  We validate our security requirements document by testing it at various stages of the SDLC.

Tentative Semester Reading Schedule

1.       Pages based on pdf from: OWASP Testing Guide v3

Foreword, Frontispience & The OWASP Testing Framework

a.       Pages (7-43)

                                                              i.      Deadline: January 27, 2012

2.       Web Application Penetration Testing (4)

a.       Pages 44-99

                                                               i.      Deadline:  February 3, 2012

b.      Pages 100-142

                                                              i.      Deadline: February 10, 2012

c.       Pages 143-166

                                                              i.      Deadline: February 17, 2012

d.      Pages 167-181

                                                              i.      Deadline: February 24, 2012

e.      Pages 182-213

                                                              i.      Deadline: March 2, 2012

f.        Pages 214-239

                                                              i.      Deadline: March 9, 2012

g.       Pages 240-261

                                                              i.      Deadline: March 16, 2012

h.      Pages 262-281

                                                              i.      Deadline: March 23, 2012

i.         Pages 282-300

                                                              i.      Deadline: March 30, 2012

j.        Pages 301-326

                                                              i.      Deadline: April 6, 2012

k.       Pages 327-345

                                                              i.      Deadline:  April 13, 2012

3.       Writing Reports + Appendix (5) – (346-369)

a.       How to Value Real Risk

b.      How to Write the report of the testing

c.       Appendix

d.      Deadline: April 20, 2012