176. Report Data Gathering (OBJ 4.1)
In this lesson, we’re going to discuss how you gather data for the report at the end of your engagement. Now, data can come from numerous different sources including your open source intelligence, reconnaissance, enumeration, vulnerability scanners, and your attack and exploit tools. As you conduct your engagements, you should always use proper note taking techniques to help you track of additional details that you discover when using your tools. Your notes should include a chronology of all the things you’re working on what commands you ran, and what results you received. This chronology can then be used as your ongoing documentation during the test as well. As you receive results from the various tools, you can either transcribe those results into your notes or simply take screenshots and file captures of those results for inclusion in your notes.
Personally though, I like to use a combination of screenshots, video share captures, and creating my own written notes when I create documentation during my engagements. Usually, your notes are going to be for your own internal use and consumption, but having them available when writing your final report can also be really helpful to remind you of what occurred during that engagement, especially if it was during a long duration engagement. Now, as you take your notes they can be used to help you create ongoing documentation during your engagements as well. As you gather all the results from these different tools though, you’re going to need to normalize that data. Normalization is simply the process of combining all this data from various sources and bringing it into a common format in a common repository that gives us a consistent event reporting format that we can use. Now, all of the data collected from our reconnaissance scans vulnerability scans, credential attacks, and others need to be normalized so that that data can all be combined in a way that helps us make sense of it all.
Everything needs to be aggregated together, combined, normalized, and then correlated in order for us to be able to make sense of it because each tool is going to use its own different notation and format. And we want to make sure they’re all in a similar format so we can read through it. Once this data has been normalized, it can be consolidated using a common database, a Wiki, or an internet portal for inclusion into our final report, if you’re working as part of a larger penetration testing team. Now, let’s take, for example, a tool called Dradis. Dradis is a framework that can be used to gather and share data and findings amongst the penetration testing team. You can actually download the Dradis framework for free using their community edition, at dradisframework.com/ce.
Or you can pay for their upgraded paid version that has a lot of additional features and capabilities. Dradis is an open source reporting and collaboration tool and it’s used by information security professionals across the world. It’s an extensible, cross platform security project that can be used to collaborate across a team, conduct project management during your engagement and create reports that combine the output of different scanning tools, manual findings, and your own notes to create exceptional reports that can be presented to your client at the end of your engagement. Now, there are other tools out there like Nessus and OpenVAS that will create basic reports as well in PDF, HTML, and CSV formats. But those only contain the results from their own scans and not a consolidated report like Dradis provides. Now, when it comes to your notes, your reports, and your findings, you always want to make sure you’re storing them securely. Remember, all of this data has all the vulnerabilities for that target organization. And so we don’t want it to fall into the wrong hands. Anytime you’re storing it, it should be stored in an encrypted format. This will make sure your notes and its accompanying database remain secure because they’re in an encrypted and stored format that only people with the right access key or permissions can access.
177. Written Reports (OBJ 4.1)
In this lesson, we’re going to talk about report writing. Now at the end of your engagement, you are going to owe a deliverable to your customer, and this is normally in the form of a written report and it may also involve you doing an actual presentation of that report to some of the key leaders and decision makers in the organization. For this lesson though, we’re going to focus really on the report itself and the different pieces of that report. This includes the executive summary, the scope details, the methodology, the findings, the metrics and measures, remediation, conclusion, and appendix sections. The first section of your report should usually be the executive summary. An executive summary is a high level overview that is written for the management and executives who are going to read this report. Remember those C-suite executives we talked about before?
The C-suite executives don’t have the time to read a 300 page report. So instead, they’re looking for a one to two page summary that we call the executive summary. In this executive summary, we’re going to give them a quick overview of the engagement and what we found. Now here we’re not going to give them every single vulnerability we found, but instead we want to give them some big overarching themes. We’re going to tell them whether or not the network was in good, bad, or okay condition. Did it have a lot of vulnerabilities or few vulnerabilities? How do they compare to other networks in the same industry you’ve looked at? And things like that. The idea here inside of this is that we want to be able to give them in just a page or two all of the big things they need to know about the engagement that you just spent time doing. At the end of this section, you need to have some kind of a conclusion statement that says what happened in this network. For example, in conclusion, I found that your network was more secure than those in your industry. Or in conclusion, I found that this system was extremely insecure and needs to be remediated immediately. This is the kind of thing you’re looking at inside of the executive summary section.
When we get to the second section, we’re going to focus on the scope. Now, as we look at the scope detail section, we’re going to list out exactly what was the scope for this engagement. Because in most cases, you’re not going to be looking at the entire organization in a very in-depth way. So instead, you’re going to reiterate what was the agreed upon scope during this engagement? Were you looking at only client facing systems? Were you looking at only internal systems? Were you only looking at a particular web application or a particular database? All that should be listed inside of the scope details. So that way, everybody who reads this report and goes through the rest of it will be able to know exactly what you covered. The third area we’re going to look at is the methodology.
Now in the methodology, you need to provide a high level description of what standards or frameworks you followed when you did the penetration test. Did you use the PTES standard, or did you follow the NIS guidelines? By having this section there, it allows your readers of your report to know exactly the methods you used during your penetration test. Also under this section, you’re going to provide a brief attack narrative. Essentially, you’re going to write a short story of what you did during this penetration test. For example, first, we did a reconnaissance to look at the footprint of the organization on the public internet. Second, we started doing footprinting of that organization to find their endpoints. Third, we started enumerating those endpoints to find the vulnerabilities. Fourth, we went through and started doing vulnerability scans targeting those specific servers that we believed would have the highest likelihood of being vulnerable. Fifth, we went through and did this type of attack against that type of vulnerability to gain initial access to the server.
Sixth, we moved from that initial point to establish a pivot point and laterally move across the network until we reach the domain controller. Seventh, we establish persistence on the domain controller so that every day at 3:00 AM, it would call back to us so we’d always have access to that server throughout the rest of our engagement. You get the idea here. In your attack narrative, you want to give them a quick event by event from a large overview of what you did, what type of events were successful as you did your exploits, and which ones didn’t work. The fourth section you’re going to have in your report is going to be your findings. Now, there are many different ways of doing this finding section so it depends on what organization you’re working for, and what your client is to figure out how you’re going to describe this. In some organizations I’ve worked at, they’ve done a full list of every single finding right here in section four. While others use this more as a summarized section of the big issues they’ve found, and what those different findings may be. For example, you might list the fact that you saw a lot of insecure password usage being done, or you might have saw that there was no two factor authentication across multiple systems, and so you’ll list out the major findings in this section but then leave the details into an appendix later on in the report. This again is going to be up to your client and you to decide what you want that report to look like at the end of your engagement. Now this section is going to be made up with the bulk of your report because you’re basically going to list out everything you found, all the significant vulnerabilities, and how you broke into the network. This is going to be important to list out also the remediations for those particular vulnerabilities because this gives the organization advice on how to fix your systems.
So for example, if you found an old window system that was vulnerable to the Want to Cry ransomware, this means that was your vulnerability. But the remediation is to install the patch, and the patch is Microsoft 17-010, and you’d be able to solve this vulnerability by simply installing that patch. Or for example, if you had something like passwords that were found to be too weak, you could recommend that they increase the password complexity, or better yet, they implement multifactor authentication. The basic idea in this section is to list out all of your findings, and then list out what your recommendation is to fix those things. In addition to that, you should also put what the threat level is for that particular finding, what the risk rating is, and whether or not you are able to exploit that particular finding during your engagement. Now, why would you want to include things like the threat leve;l; and the risk rating? Well, the reason is we don’t have enough time or money in organizations to fix every single thing. So we need a way to prioritize that.
If you said that these findings were critical, these other ones were high, these other ones were medium, and these others were low, I’m going to focus my time on the critical things first instead of the low priorities. And that way, the organization can become much more secure by eliminating the criticals and highs before we ever get down the lows. Remember, you’re never going to be able to solve everything as a defender so you have to prioritize your work, and you as a penetration tester need to help that organization do that by listing out your findings in an order that makes sense based on the threat level and the criticality. Now when we talk about the risk rating, this comes from some sort of basic reference framework, and this should have been defined back in your planning and scoping phase.
We’ve talked before about risk appetite. Remember, risk appetite is the amount of risk an organization is willing to accept. Every organization is a little bit different, and the risk appetite is something that’s defined by your client, not by you as the penetration tester. Now, when we learn what the risk appetite is for that organization, that’s going to set our threshold for us to be able to help develop what we call a risk rating. Now, the risk rating is going to decide exactly how bad something is. Normally, we do this with a basic framework that measures the likelihood against the impact. If we have something that is low, moderate, or high in likelihood and low, moderate, or high in impact, we can then determine how risky that thing is. This is essentially a qualitative assessment of a given risk. For example, if I have something that is a low likelihood but a high impact, that would actually be a low overall risk. An example of this might be that there’s a hurricane that’s going to hit Kansas. Now that’s pretty unlikely to happen because Kansas is pretty far from the ocean.
So if it did happen, they’re not very prepared for that, and so it’d have a high impact. But the likelihood of it happening is pretty low, and therefore it’s going to have a low overall risk and when we figure out what findings we want to prioritize, things with a lower risk are going to be a lower priority for us to fix because they’re less likely to happen or they have a lower impact. Conversely, if I’m sitting in Puerto Rico, there’s a very high likelihood that a hurricane can hit us. And if it does, we may have a very high impact. So that would be a high risk, and it’d be something we want to address by putting in something like backup generators, or having a continuity of operations plan where we shift operations back to some place that isn’t being exposed to a hurricane at that given time.
These are the kind of things you have to think about as you’re listing out your findings. The final thing you want to list out in your findings is what is the business impact? Now, for example, you may have a vulnerability on a system, but there’s a web application firewall in front of it that is going to protect that system from ever having that thing exploited. If that happens, there’s no business impact. So it’s a very minimal risk, and we can leave that thing there because it’s being mitigated away by that web application firewall. On the other hand, if we have some kind of an activity that has a very high business impact such as the ability to take down a server that’s customer facing, maybe it’s going to take down our ability to process credit cards or something like that, that is something that we would want to focus on and be able to remediate and overcome those consequences as part of our remediation efforts when we’re listing out our different findings.
As you start thinking through these things, you need to make sure you are listing out what is the risk rating, the risk prioritization, and the business impact analysis that you’ve performed when you look through these findings. Just running a vulnerability scan and throwing a bunch of findings at a customer is not going to help them. They need your help to assess this stuff and put it into a prioritized order so they can start attacking those things based on their budget and resource limitations. The next section we’re going to have is section five. Section five is going to be defining our measures and metrics. Now one of the things you can do inside your table for your findings back in section four, is to list things out based on a prioritized order based on given metrics or measures you’re going to define with your client. In section five, if you’ve created these custom measures and metrics, you’re going to want to define them inside here. Now when you deal with a metric, this is a quantifiable measurement that gives you the status or results of some kind of a process. Essentially, it’s a number. For example, when you look at the CVSS score, that is a number from one to 10, and this would tell you exactly how vulnerable something is or how severe something is for a particular vulnerability. When you’re dealing with measures on the other hand, these are specific data points that contribute to a given metric. The measures are the thing we’re going to measure, the metric is the amount of it when we actually measure it. So if you look at the number of critical vulnerabilities found throughout the architecture, that is a measure. When I actually count all those up and I find out there are 563 critical vulnerabilities, that means it is a metric. So measure is the thing we are measuring, metrics is the amount of something that we have measured.
Then we get into section six. This is our suggested remediations. Now here’s a place that you can go and summarize what are the biggest five priorities or biggest 10 priorities that this organization should focus on to be able to remediate their vulnerabilities. Now in our table, we talked about in section four, we are going to have some remediations on a per vulnerability basis. But here in section six, we now have a dedicated place to really describe in long form such as a couple of pages, where we describe exactly what this customer should be doing and able to solve their issues. For example, you might find that there are multiple ways to solve something. We mentioned weak passwords earlier. There’s lots of ways to solve weak passwords. One way is to use complex passwords. Another way is to use multifactor authentication. You can put in your recommendations in the remediation section that we recommend multifactor authentication, and it’s going to cost you $1.2 million based on your size of network.
And you could say if you can’t afford that, then we would recommend you use complex passwords and this will cost you $100,000 based on the time it’s going to take to train your staff and configure your systems. That will allow the organization to make an educated decision based on which of those two options they want to do based on your recommendations for those remediations. The seventh area we have is the conclusion. Now in the conclusion area, you’re going to basically summarize this report. We had a good summary at the beginning of the executive level, but that was a summary of the entire report for the executive audience. Here in the conclusion, we are kind of going to wrap up this thing into a short summary statement of a couple of paragraphs. You might list some of the failures of the organization, some of the successes of the organization, and basically what are the main things that they need to take away. If they did nothing else, what is the most important thing they need to know? That is what the conclusion is used for. And then the final area we have is known as the appendix. The appendix is kind of your catchall section to put all of your details in. You might have an entire Nessus report, or OpenVAS report, or SQL Map results. They’re going to be put into your appendix, and you can have multiple appendixes if you want. When you put in your appendix, this is where you’re going to list any supporting evidence or attestation of your findings that you want to include to the report that have all of the details.
This can include screenshots, this can include printouts, and this can include any other kind of evidence that you’ve gotten during your testing. Essentially, this is going to be able to take all the details that you’ve listed in your summarized table back in findings, and then link them to your appendix. So back in your findings, you may say I found this vulnerability C appendix A Nessus scan results. And then they can cross reference that to find the entire Nessus report that may be three or 400 pages worth of data, but at least in the table they have it in a single line or two so they can know what was wrong with that particular network. So as you can see, there is a lot to writing up these reports. Now I just gave you the basic sections that you’re going to use. Things like your executive summary, your scope details, your methodology, your findings, your metrics and measures, your remediation, your conclusion, and your appendix. There are many different ways to format these reports, many different styles you may use, and some places just use PowerPoints instead of an actual PDF or long form written report. This again is going to be something you decide during your planning and scoping phase with your client to determine what is the deliverable they’re expecting from you at the end of your penetration test.
178. Common Themes (OBJ 4.1)
In this lesson, we’re going to talk about your final out brief with an organization or another section you might want to add into your written report. This is the common themes and root causes section. Now, when you’re providing an out brief to the executives or even the technical leadership you want to make sure you’re highlighting the common themes and root causes that you discovered during the your penetration test. This can include things Lax Physical Security, your employees not following corporate policy or best practices. There might be a lack of adequate cybersecurity training or certifications. There might be a lack of software patching and updating because you have a horrible Patch Management System. There might be an issue with a lot of outdated network protocols being used or even obsolete cryptographic protocols being used. You might have people who are doing improper software development processes, or they’re simply not hardening their Operating Systems and servers. Whatever these root causes or common themes are, you want to highlight these to the executive leadership so they can start working to address these concerns. Now notice, this is not saying that a certain server is missing a certain patch, but this is a common theme across lots of different servers or lots of different workstations.
As you do your assessment and you start looking through all of the findings you had what are those three or five or 10 common themes that you notice going across all of the organization. Those are the things you want to highlight in your executive summary, as well as directly in your out brief to the leadership. So they know where they should be focusing their efforts. Now, when you’re looking at this, you’re going to be looking at things like the different vulnerabilities you’ve identified or the best practices that they’re not following. And you also want to provide your own observations in this too. So for example, if I was in an organization and I have that full list of vulnerabilities and I look through all the vulnerabilities that we found, I might find that there was a common issue with insecure or weak passwords being used. Well, I might go and tell the organization you have a problem with insecure and weak passwords. To solve this, I recommend you move to multifactor authentication or if you’re not willing to do that I recommend you use long, strong, complex passwords.
They might say well, we can’t do that because people won’t remember lots of different, long, strong passwords. In which case I might say something like based on my observations and interactions with your employees, I think they would be willing to use something like a password manager if you provided that to them. So using industry best practices, we want to go ahead and set up long, strong complex passwords for all of your systems and require your employees to use a password manager. So each system has a different, long, strong password. This would be one way to identify that vulnerability, outline best practices and share the observations you had in your assessment. This is the idea of going ahead and sharing those common themes or identifying those root causes and how you can help them overcome those issues by using proper remediation based on industry best practices.
179. Securing and Storing Reports (OBJ 4.1)
In this lesson, we’re going to talk about securing and storing reports. Now, when you have a report, it’s going to have lots of highly detailed information about all the areas of an organization that are vulnerable to attack. And so both you and your client need to make sure that those reports are secure from falling into the wrong hands. For example, if you simply have the report and post it on a website, any attacker can go there and see it. And if they see it, they now know exactly how to break into that network because you’ve given them the roadmap. So whenever possible, you want to store those reports in an offline server or in an encrypted format. This will allow you to make sure that those organizations have access to the file as needed but then it is properly protected from prying eyes. Anytime you’re going to store one of these files inside of a server, inside the internet of the organization or inside your own organization as the penetration tester, you want to make sure it is fully protected and encrypted. You’re going to make sure that these reports are own only allowed to be seen by those who have a need to know. This means that inside your own organization, for example, if there’s 100 penetration testers, but only five of them are working on this project, only those five should have permissions and access to all the accessor’s report, because the other 95 people don’t have a need to know what the vulnerabilities are in particular target organization.
The idea here is twofold. You always want to make sure you have proper access control and maintain the smallest distribution group to having access to that finalized report and the data it contains as possible, as well as ensuring that those reports are always securely encrypted to make sure they maintain confidentiality. Now, in a addition to maintaining the confidentiality of the reports and its contents, you also want to make sure the report maintains its integrity. To do this, you’re going to make sure you hash that report or you digitally sign that report. If you digitally sign that report, it’s going to automatically create a hash of that report and then encrypt that hash using your private key inside of a PKI infrastructure. This will make sure that if anybody makes any changes to that report, it’s going to make sure that digital signature becomes invalid and this way, people will always know that when you sign that report, that becomes the integrity baseline that we’re going to compare all future reports against. Now, when you’re dealing with those reports, you always want to make sure it’s only available to those who have a need to know. If you’re going to be giving this out to other people inside the organization, you need to make sure that the organization’s point of context have approved those people to receive a copy.
These things should not be sent out to a mass email distribution list, but instead, you should always minimize the transmission of that report down to only those people who need to have access to it. Any time the report is being copied, it also needs to be kept track of using some sort of an audit trail. Otherwise, additional copies can be made and those can start growing legs and going all over the place too. Another issue when you’re creating and storing these reports is maintaining version control.
And again, using digital signatures can help with this because you’ll have an exact timestamp of when that report was signed and no changes can be made after that thing was signed. If you’re using other versions of version control, you’re going to want to make sure you’re using a point notation like version 1.0, version 1.1 for a minor change, or version 2.0 for a major change. Now, when it comes to storing the reports, we’ve already talked about the fact that you want to make sure they’re encrypted so that nobody can access it that doesn’t have the proper rights or the proper encryption key to decode it.
But you also have to think about how long are you going to store these reports. Now, when it comes to the retention period for storing your reports, this is going to depend on the client’s industry and their compliance requirements. For example, if you’re subject to PCI DSS, there is currently no PCI DSS requirements regarding the exact retention period for all of the evidence and reports collected by a penetration tester, but it is considered a best practice that all of that evidence is retained by the tester, either internal to the organization or a third party provider for a period of time when they’re considering any local, regional, or company laws that have to be followed for the retention of evidence. So what does this mean? Well, it means you might need to store it for six months, might need to be stored for one year, it might need to be stored for five or 10 years. It really isn’t clear.
And so you want to make sure you’re checking with your client what is the time period they want you to store the reports for, or what is the period of time they’re going to store the reports for. As a penetration tester, you want to make sure you know what the life cycle is for those documents and all of that evidence because the longer you hold onto it, the more vulnerable you are. And so if you don’t have a need to hold onto it, you do want to properly dispose of it using digital shredding techniques or destruction of the evidence drives. So what do I think is a good time period to store it? Well, I think if you’re using something around 12 to 24 months, that should be an adequate amount of time for most cases. But again, this is going to depend on your customer and the industry they are in.
