PCI Compliance: Penetration Testing

PCI Compliance: Penetration Testing

May 10, 2011 2:00 pm

(Save to cal)


Adam Goslin completes his 3 part series on PCI Compliant Hosting by describing penetration testing and improvement network and application security.

Tuesday 5.10.11 @ 2pm

View Slides




Adam: Good afternoon everyone. Welcome to the third part in the PCI DSS series that has been hosted, thank you very much, by Online Tech. In this third presentation we are going to cover penetration testing for PCI, describe what penetration testing is and some of the questions surrounding it. So let’s get started.

A little bit about High Bit Security, we help companies obtain and maintain their PCI compliance from Level 1 through Level 4. Identifying where your company stands against PCI DSS standards through a gap analysis, remediation advice, guiding your team through the process, coordinating your chosen Qualified Security Assessor (QSA), participate in on site audits to ease your mind, and helping with any remediation items from the on-site audit. We have an ongoing PCI compliance management solution that will mitigate any surprises on your next year audit as well. In addition, High Bit provides custom penetration testing. External and Internal scans against network and application layers and our manual penetration testing is performed by security engineers with industry recognized certifications.

I wanted to start with a little bit on vulnerability scanning. One of the common questions we get is what is the difference between vulnerability scanning and penetration testing. So we will address that issue and then get into some of the penetration testing. Vulnerability scanning is performed by pre-configured computer programs that will evaluate your networks and applications for vulnerabilities and produce a report. The report will contain false positives that will require interpretation. What we mean by false positives is that the scan will report things on your report that may not be true and need to be validated. So that is the reason for the fact that it requires interpretation.

If you have any questions throughout the webinar, please log those into the question panel and at the end of the webinar we will look at questions and see if we can get a few answered before we wrap this up.

External vulnerability scanning which is performed from outside your network is required for PCI DSS and must be performed by an Approved Scan Vendor or an ASV. High Bit Security can perform your scanning requirements through one of our partners. So if that is something you need feel free to reach out to us and we can assist you.

Internal vulnerability scanning can be performed by a qualified internal or third party source. So if you already have a firm doing penetration testing for you, they should be able to perform the internal vulnerability scanning as well.

Now for penetration testing that is an engagement that is performed by highly skilled security engineers. All of High Bit Security engineers hold at least one industry recognized certification, and have background in multiple development languages. The test is run against the network and/or application layers either externally or internally.

Vulnerability scanning is included with all penetration tests from High Bit Security, but the primary focus of the penetration test is intensive manual testing by our security engineers. The High Bit Security team advises our clients of what we found, where we found it and specifics surrounding how to fix it.

One of the questions we get is: why do we have to do penetration testing if we are already doing vulnerability scanning? Vulnerability scanners are good at finding known vulnerabilities and are not very good at identifying logical faults and they often fail to find serious security flaws in custom coded applications specifically. Since vulnerability scans leverage pre-configured pattern recognition, there are many aspects of a system that cannot be tested completely (or at all) through a vulnerability scan. Penetration testing provides coverage for serious security faults that scanners are incapable of of testing. Ultimately, the difference between a vulnerability scan and a full penetration test is that security engineers have the capability to think, analyze, track, follow up and judge where as scanners do not. Reliance on scanners alone will almost certainly lead to an insecure posture.

For penetration testing experience, one of the things you are looking for when you are selecting a penetration scanning organization is the volume of experience they have in the field. The engagement for penetration tests is most beneficial when you have a firm with some experience under their belt. In High Bit’s case, testing of the network layer (firewalls, web servers, email servers, FTP servers, etc.); the application layer (all major development languages, all major web servers, all major operating systems, all major browsers); wireless systems; internal workstations, printers and fax machines. Also something called “war-dialing” phone numbers where hackers scan through phone numbers looking for open modems. Virtual environments that include a cloud computing-based environment. Internet enable devices, things like printers and other devices that are out there. We have tested law enforcement systems, state and municipal government systems, and private sector systems from online gaming to financial institutions.

With thousands of hours of experience, we have performed single engagements covering more than 4000 IP addresses and other engagements with thousands of web pages covering multiple systems.

So for the penetration testing side of things, why do it? In a lot of cases, penetration testing is required for compliance requirements such as the Payment Card Industry Data Security Standard. It can greatly improve your security posture and it should be performed regularly, at least annually. There are several reasons for that, not the least of which is continuous. Due to constant additions and removals of hardware, code releases for your web applications or for your client server applications. There are things like new patches that either do or do not get deployed as well as manual environment modifications to the network layer. There are a fair number of reasons to do penetration testing at least annually. In a lot of cases most organizations are migrating to a semi-annual penetration testing style engagement.

So for the areas of impact for penetration testing it is performed against several layers of your environment. It is performed against the network layer (web servers, file servers, firewalls, routers and email servers). This layer is evaluated for vulnerabilities and configuration issues, with all results being validated by a security engineer. For the application layer it is performed against applications (primarily web applications) looking for application layer vulnerabilities, logical faults and web server configuration issues.

In the case that you have an external penetration test that is where the testing is being performed from outside your environment, network and application. Similar to a hacker in another country. Now, internal penetration testing is performed from inside of your environment. It is similar to a hacker who has breached your outer defenses and they are now on your internal network. For PCI compliance, it mandates that both external and internal penetration testing is performed, as well as external and internal vulnerability scanning.

So how does the consultation work? It is a 30 minute consultation for scope gathering and we are completely serious about 30 minutes. In about 30 minutes we can grab enough information to be able to put together the proposal. We do that so we fully understand the requirements of the engagement so we are quoting exactly what is required. In High Bit’s case, we do not use engagement averages, you know, just fill out this sheet. We want to talk to the customer to make sure we clearly understand what we are quoting. We prepare custom quotes for every engagement. The proposal is then generated, contract approved, and then we go ahead and schedule the engagement. Testing is performed between testing windows so, we will identify when the testing will start and when it will be completed. From there finding reports are generated and delivered and a post testing consultation if that is required. We will talk about that a little later in the process. The customer will correct open issues and request remediation testing or validate to make sure everything is fixed and we will go back over the open issues to make sure everything is corrected.

For finding reports there are several different kinds of reports that will typically come out of a penetration testing engagement. Finding reports are what will be produce every time there is an instance of an issue. These finding reports will discuss the type of issues that was discovered as well as a detailed description of that issue type. Specific examples of where the issue was found so you know exactly what it is we are talking about and specific instructions on how to fix it. As appropriate these will include screenshots, code samples and sample scripts that can be used by internal staff for issue validation. These reports are of such detail that in most cases, customers will start remediation immediately without having to ask additional questions. These quite literally will include specifics. If it is a configuration issue we will tell you where to go to find the configuration element that needs to be changed. If it is a code that needs to happen we will tell you the specific code that needs to be inserted and where to insert it. Also in the detailed description will be screen shots as to how we find a specific issue and what it looks like so it is contextually relevant to your staff as they are looking through these finding reports.

The final report that is generated will contain all of the individual finding reports we just talked about, but it also includes a summary of all of the testing results, whether the testing yielded finding reports or not. That section is very important, because you need to review that section in detail. You want to look at that, because in our case we do not know with certainty whether a certain port is needed or not, whether this access is required or not. The report will contain detailed overview of the tests performed and the validated findings that come out of the report. That is something that your staff needs to take a look at in detail so that they have a keen idea on whether the environment is or is not configured properly. That is a great time to go through and validate your business requirements against the list that comes with the final report.

For a remediation report, this report will include detailed descriptions around the testing and provide a designation against each of the finding reports. In other words it will have a list, let’s say the penetration testing engagement was done and there were 12 different findings. Each of the 12 will be listed and will indicate a pass or fail on those and indicate whether or not the issue is corrected. In the event that the issue requires further work, we will provide details about the remediation testing results with screenshots, scripts and the descriptions of the findings. Once all issues have been corrected, the remediation testing report will reflect accordingly and can be used as proof to an auditor of successful testing completion.

As we go through these engagements, the auditor usually comes back with a question from the client around one of the findings and because of your environment or the way your environment is structured the Qualified Security Assessor (QSA) will indicate that that particular issue does not need to be resolved in your case. It does happen on a relatively regular basis as we are going through and doing penetration testing working with an auditor. The auditor is ultimately the end all, be all of the decision making process. They have a contextual relevance of your system, your application, how you use it and whether a specific vulnerability (while in the code it may exist) may not be applicable. In a lot of cases, customers will go through the remediation report findings with the Qualified Security Assessors and and come back with, say out of these ten issues, three do not need to be corrected because of the contextual relevance to their system. In that case what we will do in the remediation testing report is we will note those as the QSA has deemed those as not needed to be fixed. When all is said and done, the combination of the final report along with the remediation testing report will give the whole perspective of the testing engagement as well as the results.

We often get asked if customer facing reports are available. The answer is yes. Once all of the items are remediated or a QSA has exonerated some of the issues, we will provide a sanitized customer facing letter indicating the results of the testing engagement at high level.

We also get asked if sample reports are available. The answer is yes. Please send us an email either through the website or directly. If there are any questions that were not answered, please feel to contact us at anytime, we would be happy to help. You can also go to www.HighBitSecurity.com and take a look at our FAQ page that should answer the vast majority of questions we’ve come across. We have been working on improving the content of the standardized questions.

A couple of additional items about penetration testing. While penetration testing is certainly required for PCI compliance, in some cases we have clients who are not under any type of compliance requirement, they are just interested in the security and have never had one of these tests done. The general level for the types of findings we will come across as we go through the penetration testing engagements are two areas that tend to have the highest volume of findings and that is for custom coded web applications that are externally accessible that are typically coded up by the customers internal staff. That is where we find a lot of issues. And interestingly enough the performance of the internal penetration testing is another area where we typically find a fair number of issues. There is a lot of focus in the industry that is made around securing the outside and securing your perimeter and trying to make sure hackers cannot get through it. The bottom line is that you have to work under the assumption that at some point in the game whether it is through an internal fault or new vulnerability that is found and not yet corrected, hackers are going to have the capability to get through your external defenses. Once they are on your internal network that is where you need to make sure that you have got security really tight on the inside so that if they do get to that point you know exactly what they can and cannot do.

An example, we were recently doing testing against a doctor’s office where we had proven that we could get through the external defenses of their network and on the internal network found a wide variety of issues on the inside, including some outdated software and existing vulnerabilities. Pretty much to the extent that we could own most of the machines that were on that network. Internal penetration testing is definitely something that if you have never had it performed before would be highly recommended.

For the webinar series overall this is the third of the PCI compliance webinar series. The first of which was PCI Compliance: Overview and First Steps to Success. The second was Detailed Requirements Walkthrough and the third was this one, Penetration Testing and Enhancing Security for Networks and Applications. The previous two are already up and posted to the Online Tech website. The third of which will be up there in the next day or so. Those will certainly all be accessible.

We now have some time to go through some Q&A on penetration testing, vulnerability scanning, etc. Just letting you know that we offer free consultations for PCI DSS compliance and Penetration Testing. My contact information is up on the screen. You can reach me by phone or email at: 248.388.4328 or agoslin@HighBitSecurity.com.

Question: Is there a sample report that we can walk through?

Adam: Sure. Here is an example of a sample report that we have. There will be an executive summary section for the engagement that will include a high level summary of the types of issues that were discovered and found through the testing scope. It will also cover recommendations that we have, as well as the scope of the testing. Then we start getting into the testing details. So there is network discovery section which will cover the tools used for the network discovery. The table for network discovery issues. We will also do passive discoveries (Whois, NsLookup, TraceRoute and Zone Transfer requests). We will attempt to do testing against ICMP for Echo, Timestamp and Netmask. For Port Scans we will go through and scan the TCP Ports and UDP Ports. We will also go through the Stateful Firewall Test, TCP Ack Scan and Source Port. Vulnerability scanning will be performed with a summary of the scanners used and scanning results. This section for the Summary of Scanning Results will be the reviewed and validated scanning results that we have seen. Penetration testing will cover some paths of reconnaissance if the customer is looking for that to be included. What types of information we can find out in the marketplace about you, ways that we would be able to get into the system, etc. We will look at information disclosure issues. So look the Robots.txt files on a web application, Hidden HTML fields, etc. We will look at session handling, encryption, authentication (Logon Logic, Password Recovery, Account Creation, etc.) We will look at Logical Faults, Path Traversal, and Predictability in access Controls.We look for any reflection issues on your website looking at XSS. Input Validation Issues with SQL Injections, Command Injections, etc. We will also cover Error Handling and configuration issues that we see.

Then we have testing items that are discovered so we will go through and provide a list of the various testing items that are included. We then have a couple of appendices on there for severity level, etc. So that is an example of an external penetration report. Certainly if anyone wants to have samples feel free to contact us and we can get that out to you and be happy to have that conversation.

Question: Do you recommend starting with automated testing?

Adam: It depends on the circumstance. If you are first starting out, there are two schools of thought on this. One is let me go out there and do a full scan of my environment and I will cure everything I can then go to penetration testing. Frankly speaking, it really comes down to what the objective is, because bottom line is if you are going to pick up a regular and repeated penetration testing engagement, we recommend starting with the penetration testing because it is going to include a vulnerability scan and also validate all of the results so your internal staff are not wasting their time pouring through what can quite literally be a few thousand pages of vulnerability that will show up on a report. Take advantage of the experts to go through and really give you just what it is that you need to go ahead and cure.

Once you have gone out and done your initial scan, for companies that are brand new to penetration testing we always recommend that they go in and do a penetration test (I recommend doing another in six months) and then after that go to an annual at least, because what will happen on the first time you go in and get a penetration test performed you will be faced with a lot of issues. The budget for the first engagement is usually fairly well consumed with just the volume of issues that are found and discovered. When you get back around to your second step and run at doing a penetration test now most of the quick hits are taken care of and now you are really starting to get into the depth of the applications. When we take on a penetration testing engagement we request of all of the customers that they provide us with all of the information. So for example, IP addresses, URL, Logon accounts so we can get onto the accounts and perform testing. In some cases we are asked why they need to provide all of this information and they say you guys should just be able to hack into our system, no? The answer is absolutely we could do that, but do you really want to be spending your dollars having us try to breach your system? Or would you rather gives the information and breadth of coverage that you are looking for so we can do more testing across more of your applications. I would say that 99.5 times out of 100 the answer comes back that they would much rather have us get the breadth on their testing engagement rather than just trying to prove it out on a case by case bases.

Question: Would you recommend seeking the help from a penetration testing professional?

Adam: The answer is absolutely. The bottom line is that there are a lot of tools out there (this is a good one to cover) that are called penetration tests when really all they are are glorified vulnerability scans. The real key you really want to get out of your penetration testing engagement is you really want someone who has done this work before, has a lot of experience and can really give you a bang for your buck. Penetration testing engagements are expensive for specialized environments that professionals for penetration testing have. Its not an inexpensive adventure. You really want to try and get as much out of it as you can and not short changing yourself by just running out and getting a scan or trying to have one of your internal IT staff attempt to do it because they have run one of these before. Often times that is one of the comments we get back in the converstation, can’t my internal staff go ahead and take care of this? If you are lucky enough to basically have a penetration tester on staff absolutely, but the vast majority do no have the expertise to run a penetration test.

So I covered all of the questions we had on the board. Thank you very much everyone for attending the webinar. Certainly feel free to reach out to myself if you have anymore questions on PCI or penetration testing. Thank you very much for your time.

Back to Top

Webinars    |    Online

Get started now. Exceptional service awaits.