Making Sense of Service Organization Audits

Making Sense of Service Organization Audits

February 14, 2012 2:00 pm

(Save to cal)


Join David Barton, Jon Long and Online Tech's Risk Management & Security Officer Jason Yaeger for a discussion to help clarify data center audit standards and assessments.




View the slides (PDF).

Our webinar was Tuesday, February 14, 2012 from 2-3pm ET to discuss the differences between AICPA's (American Institute of Certified Public Accountants) SOC (Service Organization Controls) audits and reports, other types of audits, and the difference between point-in-time, period of time, self-assesments and independent assessments.

David Barton, CRISC and CISA

David Barton, CRISC, Principal, UHY LLP
David is a Principal and is the practice leader of the Technology Assurance and Advisory Services group at UHY Advisors, Inc. in Atlanta, GA. He is Certified in Risk and Information Systems Controls (CRISC) and received his Certified Information Systems Auditor (CISA) designation in 1988.

With over 25 years practical experience in information systems and technology risk and controls, he is an expert in identifying and reducing information technology risk throughout an organization.

Read David Barton's guest blog post on our blog, "SOCs and SASs: The New Standards for Service Organization Controls Reporting."

jon-long-150Jon Long, CISA, QSA Senior Audit Manager CompliancePoint
Jon is a Senior Manager and Practice Builder at CompliancePoint as well as an enthusiastic blogger at The Risk Assurance Guy. He is a Certified Information Systems Auditor (CISA) as well as a Qualified Security Assessor (QSA) in the Payment Card Industry (PCI).
Jon has over 15 years experience in IT, and crossed over into auditing in 2006.  His background enables him to conduct audits from the perspective of having been audited by external auditors.  He is currently championing an audit approach that allows organizations to combine multiple compliance requirements into a single SOC 2 engagement.

Jason Yaeger, Risk Management & Security Officer, Online Tech


Jason Yaeger is Online Tech’s Risk Management and Security Officer. In his 3 years at Online Tech, Jason has guided the company through successful completion of many audits, including SAS 70 Type I, SAS 70 Type II, SSAE 16, and HIPAA. In addition to overseeing operations across all of Online Tech’s data centers, Jason is also the Vice President of the Southeast Michigan Chapter of 7x24 Exchange. Prior to Online Tech, Jason was Director of Internet Operations at 20/20 Communications where he spent 8 years developing the company’s wireless and internet initiatives.

Full Transcript Below. 

April Sage: Good Afternoon everyone thanks for joining us for today’s webinar. Today we will be talking about Making Sense of Service Organization Audits. My name is April Sage and I will be moderating the questions today to our guests. We are fortunate to have two of the leading voices of change in the audit industry, David Barton, Principal of UHY Advisors, LLP and Jon Long, Senior Audit Manager of Compliance Point. We will also be joined shortly by Jason Yaeger, Online Tech’s Risk Management and Security Officer who has guided Online Tech through several independent audits over the last few years. He will have first hand experience to share.

We are going to start off with David and Jon walking us through some of the background of the definitions of standard assurance certified clients. They are going to compare some of the most common audit standards and surface some of the key issues this list of audits is imposing on service organizations and their clients. We will have Jason here to weigh in the differences the various audits have made to day-to-day operations and the company culture at Online Tech and just share his experience of guiding a company through a long and winding maze of audit requirements over the years.

We welcome your comments and questions throughout the webinar. With no further ado, David, I will turn the presentation over to you.

David Barton: Thank you very much, April. Again, what we hope to accomplish today is to alleviate someof the confusion that is out there in the marketplace about these new audit standards the AICPA introduced mid-year 2011.

Let me go ahead and get started with some background information. I think everyone is aware that SAS-70 has been eliminated and the AICPA replaced SAS-70 with three new auditing standards. These are not standards for data centers and service organizations, these are audit standards. SOC 1 is commonly known within the marketplace as SSAE-16, which is a short acronym for Statement of Standards for Attestations Engagements Number 16. That report is for control and service organizations that have internal control over financial reporting. That right their is really the crux of the issue today as far as the confusion, particularly as it relates to data centers, colocation providers and in some instances cloud providers (depending on the nature of the service they provide).

SOC 2 is a controls report on a service organization relevant to security, availability, processing integrity, confidentiality or privacy. Those are the five principles of the Trust Services and Criteria that have been defined and out in the marketplace for several years. Finally, SOC 3, which is basically the old Trust Service audit that has been rebranded as Trust Services Report for Service Organizations.

Okay, so what was wrong with SAS-70 and why did the AICPA feel it needed to be replaced? First and foremost it really had limited applicability. It was really only intended for internal controls over financial reporting. It became “in vogue” as a result of some changes in the ways audits were performed which we will get into a bit later. Also it is limited distribution. It is meant only for the customers, auditors and management of customer firms. It was never intended to be a general house-keeping seal of approval which it became known as. Secondly, it wasn’t consistent with the international standards for similar types of audits that were being performed under other international accounting bodies. It was incorrectly applied almost, I won’t say universally, but in a lot of cases. It was used to provide general assurance over security and processing integrity and availability for software providers, partnership type arrangements and document management companies.

I even heard of a specific, very large, Fortune 50 bank requesting a SAS-70 report from a national pest control provider for all of their branches. Clearly it was never meant to provide assurance that pest control was going to do what they were supposed to do. There was also no standard criteria for internal controls or general IT controls. There was no criteria whatsoever. Basically SAS-70 was a blank sheet of paper. Every organization developed their description of their controls, which the auditor then proceeded to test and determine whether those controls were operating correctly. And last, but certainly not least, there was huge variation in quality from various services organizations in terms of the depth of the report and overall quality and whether it really addressed the controls a given customer was interested in.

Here is an example of inconsistent quality. This came from a specific report that I have seen and had the privilege of reviewing. The control objective said: “Controls provide reasonable assurance that data integrity is maintained through various stages of processing in accordance to the users specifications. So if you look at the control activities specified by ACME Corporation, the IT director is solely responsible for maintaining all archives and backup systems.” First and foremost, that’s not really a control. “The test performed by the Big Four was the inquired and reviewed archive and backup policies and procedures with the IT director.” Okay, that’s great, but what does that have to do with maintaining data integrity? Nothing. “Reports are generated detailing job statistics and exceptions if any with daily backups for each server. The Big Four examined the backup reports for successful completion of backup jobs.” Again, that is all well and good, but that doesn’t have anything to do with data integrity. “Media tapes are used to execute the backup procedure.” Well that’s terrific, what else would you use to execute backup procedures? I guess you could use disk to disk, but again, that’s not really a control. It’s a statement of how backups are performed. “Off-site storage is provided by Iron Butter Flying Data Services.” Well that’s great. Not really a control, it’s more of a statement of how backups are stored. “Conclusion: No exception noted.” So if you are reading this report as a customer of ACME and you are concerned about data integrity, do these controls and the tests that were performed really give you any assurance about whether your data is going to maintain its integrity?

Let’s look at some definitions. I’m going to turn the presentation over to Jon at this point and he is going to go through some of these definitions and what they mean in terms of how service organizations work.

April: David, can I interject with a question at this point? I just want to clarify that the understanding we have come to is that no two SAS-70 audit reports are the same. And because there is the opportunity or the work flow for the company being audited to specify the controls that they are audited upon, this is the inherit nature of the levels of inconsistency. So if someone wants to compare one SAS-70 report from one company to that of another company, they really have to read the full details of the complete report. You can’t just assume that because two companies have both had a SAS-70 audit report that that’s meaningful in terms of an apples to apples comparison.

David: Absolutely. Each one is unique and the only way to know whether the specific controls you are interested in have been addressed is to read the report, and trust me, they are not something that you want to read. They are painful to read most of the time. There is a lot of detail and a lot of wordsmithing in there that can gloss over a lot of controls that are either missing or not designed correctly. So yes, the understanding you have is correct. No two SAS-70 reports are identical and that’s inherent in the way the report was intended by the AICPA. It was never really meant to provide assurance of security, availability, processing integrity, confidentiality or privacy. It was meant to convey what the controls are at a service organization that impacts the internal controls over financial reporting at a customer site.

April: Thank you, David. Well I am really happy to delegate the reading of those reports to you, because they do not sound fun and sorry about that Jon, we will now turn it over to you.

Jon Long:That’s okay. Thank you for that! I just wanted to back-up a minute and say that a lot of the confusion seems to be around just some of the basic terminology. So what I did was go to and pulled a couple of definitions from there that are up on the screen. And I just wanted to go through what this means and maybe that will establish a common ground for everyone in understanding so we can all come from the same place. Standard, for example, is “something considered by an authority or by a general consent as a basis for comparison or an approved model.” Who defines the standards? And what does it mean, by way of example, to have a clean house? Who gets to decide that? Should it be the occupants of the house? Or does it need to be an independent authority or general consent of what a clean house is? Why can’t the people living in the house decide what’s clean? If you back up to the definition of standard it can’t be the person defining that themselves, it has to be someone else defining that or general consensus as to what that means.

If you define a clean house by the occupant for example, it is going to be the bare minimum that it can be. No clutter, clean floors, no food left on the counter, etc. That’s what it means to me, because that means I don’t have to clean as often and I don’t have to do as much. It’s just easier for me, because I like my house the way it is. When you start having an authority talking about what clean means, then you get several different categories of authorities defining what that means. One of those is broad objectives such as no clutter, no dishes in the sink, clean floors, no dust, no food left on the counter, everything in its place. So it’s more broad and you get to decide what that means to you. What does it mean to me to have everything in its place? It leaves it as broad as it can be so you get a little more flexibility. Not everyone is going to have the same kinds of floors. Some have tile, some have wood. So you aren’t going to use the same kind of cleaning agents for those, it gives you flexibility.

Then there is an even deeper level and definition of what a clean house is. For example, no clutter means no clothes on the floor, beds must be made, no obsessive trinket collection or pictures hanging. You can see where I’m going with this. You could get very, very detailed very, very quickly, but the problem is sometimes it’s not applicable. You call out the tile floors for example and specify that they must be cleaned daily with bleach, but what if you don’t have tile floors? Suddenly what’s right for one organization (or house) is not right for all of the other organizations. So there is an inherent problem with being too specific.

What is assurance? Assurance is a positive declaration intended to give confidence. There are various types of assurance. There is:

  • “My house is clean”
  • “His house was clean when I inspected it last year”
  • “His house is continually clean”

The difference there is the perspective. When you say my house is clean, it implies, yes, you can trust me, because my house is clean, take my word for it. There are some inherent weaknesses in that, because it is only as good as the person making that assertion and most people are going to default to how clean their house is. It’s a very low level of assurance when you take someones word for it.


Then there is the next level of his house was clean when I inspected it. That means you had somebody come in and look around and yeah that person said, “yeah it was clean when I checked it out. I don’t know anything about before I got their or after I left, but while I was there it was clean.”

The next level assurance is, his house was clean all last year. That is the best you can do at this point. To say through various methods I went to his house everyday this last year, I visited him once a week and sure enough, every week his house was clean. It is called a period-time assurance in the assurance world. Where as the second example, is a point-in-time assurance.

The utopia where everyone would love to be is to have the assurance of his house always being clean. That level of assurance is not available today, because to do that you would have to basically have to have cameras in his house all the time. It’s just impractical to have that level of assurance.

What David and I did was break down some of the standards and assurances into categories on this chart (view slides). The one you see at the top there is the Cloud Security Alliances standard which is very, very good for cloud computing. It is very, very detailed and tailored toward the cloud space. The only problem is it’s a self-assessment at this time. There are no attestation standards built around it where a third party can go and look and verify those standards and provide assurance based on certain rules.

A lot of great detailed controls, but there is no assurance component to it. Cobit, same thing. HIPAA, there are components of HIPAA that provide point-in-time assurance. ISO 2701. Some people have tried to debate that in terms of me making the assertion that it’s a point-in-time assurance, you just look at any certificate that is provided by a registrar and it will show you that there are terms that are to be followed by that organization. If they are broken, they become immediately non-certified and it is revoked. So it is a point-in-time assurance. PCI is the same thing. It’s a point in time assurance. Whenever the Quality Security Assesor (QSA) goes onsite and test the controls and checks them, if everything checks out then they say yes, they pass the test. But they don’t know anything about before they got there or after they left.

SSAE-16, SOC 1, itself is a standard, but it is an attestation standard that applies only to auditors and they are the only ones that have to comply with that standard. The actual meat or heart of that project is not based on any accepted standards and like SAS-70 it is defined by the organization that is being audited. So it is the lowest level of my house is clean and I define what clean is. That is why the AICPA says that it is not designed to provide assurance about security or availability. You have to go back to a standard for those types of assurances and that is why they have brought in this last one called the Trust Services Principles and Criteria. These are standards for security, availability, processing integrity, confidentiality and privacy.

There are over 180 controls built into those Trust Services Principle and Criteria and that is the foundation of what a SOC 2 is. Like SOC 1 it has a Type I and a Type II, which is period-in-time versus point-in-time. You can get both. It is a very interesting thing and there is no single answer, because Trust Service Principles are not as detailed as the Cloud Security matrix for example. They are very, very broad and they are meant to be broad so they can apply to various organizations. The issues this creates for companies that are trying to make people happy about their internal controls is they’re being forced to satisfy everybodys need for assurance with all of these different requirements, all of these different standards. They are wasting time scheduling and supporting external auditors from multiple firms.Sometimes they are being audited directly by their customers. There is a lack of clarity regarding the customers expectations and it just makes people want to rip their hair out. What compounds this issue, is a lack of understanding of the word certified.

The first definition of certified is “holding or guaranteed by a certificate.” In the dictionary it actually says the second definition is “to guarantee as meeting a standard.” So certification or certified implies a standard. Well what is the standard? And who grants the certificate? Better yet, I’m a Certified Information Systems Auditor, does that mean that ISACA guarantees the quality of my work? Do I get to get rid of my error and omissions insurance? What does that mean when you say you’re certified in something? I just wanted to go over some basic fundamentals there, because I have to understand everything in very simple terms and I’m not like David who understands very, very complicated issues at a granual level.

With that, David, I want to pass it back to you and you can take us from here.

David: As we defined what SSAE-16 certified really means, which assurance would certified belong to? Are you looking at a self-assessment, third party attestation, over a period of time or is it real time utopia? I think the answer we would like to see is something that covers a period of time. Which is what both SOC 1 and SOC 2 are intended to provide.

Alright, so new audits for third parties. SOC 1 (or SSAE 16) is intended for Internal Controls over Financial Reporting (ISFR). SOC 2 is one of the newest ones that the AICPA puts out, it’s based on some predefined control principles and criteria (which are the Trust Services principles and criteria) and they cover the five areas of security, availability, processing integrity, confidentiality and privacy. Now SOC 3 is basically a condensed and abbreviated SOC 2 report. It only includes the auditors opinion and managements assertion. It also includes an abbreviated description of the system it is being attested to.

At the conclusion of that report, what you basically get is the auditors opinion. This SOC 3 seal (see slides) can be displayed and that is sort of the good housekeeping seal of approval that a lot of the industry has been clamoring for. The other piece of a SOC report is its general availability. Meaning that with the appropriate controls in place the service organization can actually distribute the SOC 3 report to perspective customers and general folks who are interested, it can be published publicly. That again is something the marketplace has been asking for for quite a while and a lot of people just provide the SAS-70 or SSAE-16 reports, but they do so incorrectly.

Let’s talk about ICFR. What really is ICFR? There are a whole lot of words on this and that’s because it came from the AICPA standards itself. “The process designed by or under the supervision of restaurants, principal executives and principal financial officer....” The meat of this is: “In accordance with general acceptance of accounting principles, includes those policies and procedures that pertain to the maintenance of records that in reasonable detail, accurately and fairly represent the transactions and assets. Second, provide reasonable assurance that the transactions are recorded as necessary to permit preparation of financial statements. Third, provide reasonable assurance regarding prevention or timely detection of unauthorized acquisition, use or disposition of the registrants assets that could have a material affect on the financial statements.” Don’t be confused by that last part and think we are talking about planting equipment or in the context of what we are talking about today, a server that may be located in a data center.

The services provided by a service organization that are part of a user that needs information systems, basically the standards says, an auditor doing a financial statement audit has to understand the control environment and the information system is part of that control environment. Paragraph 2.01of SSAE-16 standard says, the guidance in section 324 of the audit standards, “a service organizations services are part of a user entities information system. If these services affect any of the following, the classes of transactions in the operation that are significant to the financial statement. The procedures, automated or manual, by which the user entities transactions are initiated, authorized, recorded, processed and reported in the financial statements. See related accounting records, whether electronic or manual, supporting information specific to accounts and users financial statements involved, initiating, authorizing, recording, processing and reporting.” Those are the same five words again. “D: How we use our entities information system captures other events and conditions that are significant to the financial statements. E: The financial reporting process used to prepare the financial statements including significant accounting estimates and disclosures.” I’m going to ask you a question, if you are a data center and you are providing racks and pipe, what impact do yo have to any of the transactions that flow through those racks and pipes that are totally and entirely controlled by your customer? I’m guessing that you all agree that it’s very difficult to get to a point where anything that a data center does, impacts those financial transactions.

Jason Yaeger: I would say the only real way we could affect it is by whether we have a lack of physical control where someone could get into the actual physical facility, break into that rack and steal data or availability of such financial data. Those would be the only things I could really think of.

David: So really, you’re talking about security and availability.

Jason: Exactly.

David: Okay, so even there, I’m going to go out on a limb and say, even if someone were to break into the facility and did somehow get access to a physical server that belonged to one of your customers, they would have a very difficult time affecting any of the financial transactions flowing through that server real-time. Unless, there was some pretty lax logical security, which I’m guessing in most cases, Jason, your clients are responsible for the logical security over the operations that process those transactions?

Jason: In some cases, clients provide their own firewall services and in some cases we provide those firewall services. In most cases the clients would be primarily responsible for hardening the application.

David: Right, the application level is where the transactions are processed. 2.02 “Other controls of service organizations may be relevant to the audit. However, services that do not affect the services described in 2.01,” in other words it has to be part in parcel to the transactions that are flowing through there, “are not part of user entities systems and service providers that provide such services would not be considered service organizations or the purposes of SSAE-16.” That doesn’t mean that there aren’t a dozen more customers and management and other people who are interested in how you are protecting their data, the security, the availability, etc. All this says is if you are talking about those kinds of items it doesn’t apply to SSAE-16.

In paragraph 2.03 it says, “as stated in the preface, prior to the issue...” Get down to the meat of it. A-E Section 324 does apply to services provided by a service organizations that are limited to executing client organization transactions that are specifically authorized by the client. It uses the example of a checking account transaction, but it also says here, for example, when the user entity retains responsibility for authorizing the transactions and maintaining the accountability. If you are a data center or colocation provider and you are not providing any managed services over the applications that process the financially relevant transactions, then I’m opening it up to a challenge to say, how is it that companies are getting SSAE-16 reports? Because clearly this says it doesn’t apply.

April: We have a couple questions here, David. One question, if you go back to the previous slide in Section D, could the IT general controls be covered there?

David: Yes, they could. But when you take the entire paragraph as a whole and look at the broader intent of the paragraph, you can find a place were you could say general controls apply to this section. Yes there are lots of those and both the SSAE-16 guidance, as well as the SOC 2 guidance, provide examples for each other if you read them carefully. My point in presenting this slide is, again, you have to understand the nature of how we got down to this level. This all started, because of audit standards that said it is the auditors duty to understand the overall control environment. Which, by the way, includes the information system, which by the way may involve a service organization.That doesn’t mean that there aren’t other ways to understand the control environment. i.e. a SOC 2 report provides better, more comprehensive information about security and availability which is what you are really dealing with when you engage the services of a data center or colocation provider.

April: Thank you, David. And I’m assuming that is the direction we are working our way towards. Here is a quick question about SOC 2 and SOC 3 that maybe we can address now. What is a best practice for providing SOC 2 reports to customers? That question actually came in when you were talking about to who and when the reports are available. So, when do you give the report to the customer?

David: It is no different than a conventional SAS-70 or SSAE-16. A SOC 2 report covers a period of time mostly if it is a Type II report. The types of reports are the same. It is either designed under the controls which is a point-in-time or designed and operating effectively under the controls which is a Type II. If you get a SOC 2, Type II report, it is going to cover a period of time and the period of time it covers is going to be part of the overall opinion letter. It will clearly state that the period of time the testing covers is from January 1 to December 31 or January 1 to June 30, whatever the dates may be. Once you have had the audit conducted and the auditors are satisfied, they develop their report. Once you get the final report from the auditor then it’s available to give to your customers as they request it or you can distribute it to your customers as they wish. There is no real standard as to how or when you distribute it.

Jon: It still is in restricted use though, right David?

David: Yes. In the definition it says it may be restricted use and there is some trust factor there. I think it may be the language and that the actual SOC 2 guidance was intended to help protect the audit firms and allow them the flexibility, but the word may is there. Whereas in the SSAE-16 guidance it is clearly not ever intended to be a general release. So, yes it is generally restricted.

April: David, does it make any difference who provides the report to the customer? Does it have to come from the operations department, management or sales? Or does it not matter?

David: It does not matter, because it is your report as a service organization. However, you should check the language in your contract with your auditor, because most audit firms will specify when those reports can be released and that they have to be released in their entirety if they are. It will also generally restrict the audience to auditors and management of your customers firms.

April: Okay, let me pose a question to you about SOC 2 versus SOC 3 reports. If you were evaluating or if you were wanting to compare quality of services provided by a service organization, would you recommend that someone compare the SOC 2 audit report or the SOC 3 audit report?

David: The SOC 3 report is not going to give you any detail as to what was tested or what the specific control activities were that the service organization put in their assertion. All you are going to get is brief description of the control environment or the system. So for that a reason, if you are really trying to compare apples to apples as far as providers, a SOC 2 would be a much better report to read. Th problem being depending on how the audit firm has restricted the use of the report it may be difficult to get your hands on it in a timely fashion.

April: Do you think that the SOC 3 report runs the risk of people looking again for that stamp of approval like they maybe did mistakenly with SAS-70? So that if two providers have a SOC 3 report, would they assume that is an apples to apples comparison and is that a reasonable assumption?

David: It is a reasonable assumption. There will be differences between SOC 2 and SOC 3 reports. But again, you don’t get the detail with the SOC 3 so basically you have to really on the opinion letter. The good news is that a SOC 3 report, if you conduct a SOC 3 audit, and you say you are going to do that for security and availability (two of the five principles that are there), then the auditor basically has to form an opinion based on the testing that they do as to whether or you truly do have controls that meet each one of those criteria. So at least you know you are comparing the same criteria from one organization to another. There may not be the same level of detail within that, because the criteria within that are not specific standards. In other words, one could state that you must perform daily backups and store those backups in a hardened facility that is capable of withstanding this and this.That is not the level of specificity. This is the SOC 3 criteria of basically placeholders and the service organization still has to fill in the blanks of how it meets the criteria. But the criteria will say you need to have procedures to ensure recoverability of the data. It won’t specify how soon, how it should be done, where it should be done, just that you have to have backups. So there is a level of distinction between the two.

April: Okay great. Thanks, David.

David:  Again with SOC 2 and SOC 3 there are over 120 controls criteria, not standards, and they are focused on four key areas:

  • Policies
  • Communications
  • Procedures
  • Monitoring

And again, it is not a “thou shalt do this and thou shalt do that.” It’s not that specific. It’s basically saying the organization should have policies and procedures around backup. The organization policies and procedures around data security. It’s fairly high level and the organization gets to determine and fill in the blanks again.

Lets talk a little bit about SAS-70 versus SOC 1 and what has changed with SSAE-16. Suppose you already meet the criteria that you are providing processing of transactions that have a direct financial statement impact. The SAS-70 was an audit. The SSAE-16 (or SOC 1) is an attestation, which changes the way the auditor will perform the work necessary. With SAS-70 there was no assertion and with SOC 1 there is now a management assertion that goes into the overall report. Management says “we hereby declare and decree that we had controls in place from this time to this time and here is how we determined what those controls were and here is how we can provide this level of assurance.”

In SAS-70 you had a description of controls. Wheres SOC 1 is an overall system description. So you should see more information in an SSAE-16 compared to a SAS-70 which basically just lays out what your controls are. In a SAS-70 you had no criteria. In a SOC 1 the auditor is supposed to identify and evaluate the suitable criteria used to develop managements assertion. Again, SOC 1 provides a system management description, but there is still no standard criteria in a SOC 1. And I can guarantee you right now that there are procurement departments all over America rewriting their policies in their contracts to require SSAE-16 certification.

New Standards.

This is a table (view slides) that comes directly off the AICPA website and describes the differences between the three, when they should be used and how they should be used. In a SOC 1, the users of the SOC 1 report are the users controls, controllers officers and users auditors. In other words, Online Tech has ABC Manufacturing Organization as a client. And ABC Manufacturing’s auditors are trying to plan the financial statement audit for ABC and they need to understand the controls around the services Online Tech provides. This is all about the financial audits of a service organizations customers, not the financial statements of the service organization. There is a lot of confusion around that and what is related to user financial controls around that.

SOC 2 is for management, regulators and anybody else interested in what the controls are at a service organization. It could a result of government risk and compliance programs, over sight, do diligence, etc. The “what “would be concerning those five principles of security, availability, process integrity, confidentiality and privacy. SOC 3 is for anybody that needs to understand that there is an assurance around any of those five areas.

When do you use each?

Basically, if it’s going to be used by your customers and their auditors to plan or perform a financial statement audit, then it should be a SOC 1. And you have to take that with a grain of salt, because in most cases, if you are talking about a data center, there are other ways to get comfort around those controls. Which report will be used by customers to gain confidence and trust in a service organizations system? A SOC 2 or SOC 3 report. If you need to make the report generally available or make a seal, then it should be a SOC 3. Again, these are straight off of the AICPA’s web page.

What are the issues with the SSAE-16?

Basically management is still providing that system description. They are still saying “my house is clean,” going back to Jon’s portion of the presentation. And you have no way to know what that really means other than to read the report in its entirety and ask yourself what’s missing. There’s still no standards, controls or criteria. All service organizations that had a SAS-70, correct or not, are likely going to default to a SOC 1. We’ll get to why that is in a minute, but I want to stress that is not correct. And in some cases you will have a lot of service organizations that will provide both simply because of this misunderstanding in the marketplace and the fact that their customers audit firm said they had to have an SSAE-16. And lastly, procurements rewriting RFP’s and contracts will require SSAE-16 certification. Again, there is no certification as we talked about earlier.

Here are example’s of press releases (view slides) that already say SSAE-16 certified that’s replacing SAS-70 certified jargon that has already been in the marketplace for several years.

Why won’t these new standards really help?

Well, until we get some educated consumers out there who drive the train, there is still going to be this issue of SAS-70. Everyone knows what they think they know in general terms what a SAS-70 is. Most of the time they are wrong and don’t really know what it’s for, but they use it anyways, because it was the only thing they had. Secondly, the AICPA came out when they announced these new standards and said SSAE-16 replaces SAS-70. So if you are a service organization and you have been getting SAS-70 reports for five years and the AICPA says SSAE-16 replaces it, you’re naturally going to go there first. The problem is they didn’t fully explain what SOC 2 was all about and how it can be used instead of or in addition to SAS-70 or SSAE-16.

The service organization still specifies the controls and there is still no criteria they have to meet. The quality is still going to be questionable, because you don’t have that standard set of criteria you’re basing your report on. And then there is still the confusion on the differences between SOC 2 and SOC 3 and SOC 1 and these other two.

And the last item, the customer is always right. What do I mean by that? Well here’s how we got to where we are today. The customer auditor, let’s say it’s one of the Big Four, wants to look at an SSAE-16, because the AICPA says that replaces SAS-70. They have been looking at SAS-70 reports for five years and their comfortable with it, they don’t want to think about anything else. So, the Big Four says we want an SSAE-16. Procurement is going to rely on what their auditors says, because they don’t want to rock the boat. They are going to ask their service organization for an SSAE-16. The service organization wants to get the business of the company, so they are probably going to agree and figure how they can address the need. And lastly, the service auditor wants the business of the service organization to do the new audit and if the service organization says, “hey I need an SSAE-16, because 80% of my large customers are asking for an SSAE-16, “ right, wrong or in-different they’re going to give it to them. So that is what I mean by the customer is always right.

Suggestions for improvement?

This is my personal opinion and I am going to stress it as such, but based on what I presented today I think you can see how I got here. If we use SOC 2 as a foundation for all SOC audits, meaning we use that standard set of criteria for all of the IT general controls that it outlines, then you can add operational aspects and any additional ICFR type controls and have a much better report. It will be more thorough, it will be more complete and I think it will be higher quality, because if you don’t meet one of those criteria you have to explain why not in the report. Whereas, in an SSAE-16 you just leave it left out and hope that the reader doesn’t notice. And lastly, it will allow a comparison of apples to apples. That is my justification for why I believe a SOC 2 is a better solution as a base line for any of these types of reports.

April: David, I’m going to have Jason weigh in on the differences between these audit reports from someone who has had to steer a company through them. Jason, you’ve been through a SAS-70 audit, an SSAE-16 audit (also known as SOC 1), the developments of SOC 2 and SOC 3 audit reports, a HIPAA audit and a PCI/DSS audit. That’s a lot of audits and paperwork shuffling (we have a great appreciation for Jason and everything he has done here). Jason can you comment on a couple of things, one did you see a significant difference in the impact of period-of-time versus point-in-time audits? Why or why not?

Jason:  Our first audit was point-in-time audit, the SAS-70 audit and then obviously HIPAA. When we initially implemented the SAS-70 in 2009 there was a big difference. Our first audit was a point-in-time audit, because we didn’t have a history of having the audit period to go back to for the Type II. So it was a big difference initially. Now, because we are always doing audits from the prior period, we’re making sure we are living up to the things we say we do all the time, doing a point-in-time audit now is inconsequential. There were some things we had to add to our control processes for HIPAA and some things we needed to cover in our employee documents and processes and procedures.

To look at it for just a point-in-time and not look at it throughout the year is a lot easier obviously, but because we are already doing it for SAS-70, SSAE-16 and now SOC 2 , we just include all of those HIPAA policies into our recurring policies and procedures. We’re not just doing them once a year when we’re getting audited. So from my perspective a point-in-time audit is good to start off with if you have no prior history, but you really should be doing the other.

April: So you feel that it’s made a valid and significant contribution to the company culture and do you feel like it has improved the quality of our service?

Jason: Absolutely. It’s absolutely improved the quality of our service and improved the communication and transparency we provide to our clients. So, we’re doing a lot of similar things that we were doing before, but we had no way to show our clients and prove to them from an outside third party that we were doing them. We did have to change some things. We had to add a couple things to our policies and procedures. But now we get say UHY to look at an entire year, an outside third party that we had come in and do this and that just gives all the more credibility with our clients and the fact that we’ll share it. That’s another thing, we’ll share it with our clients as well. A lot of companies in our space won’t share it. Maybe, because their afraid it’s a peek under the hood so to speak.

April: So for companies that do have that culture of transparency and genuinely wanting to comply instead of having it be an aggravation that you’re writing a check to satisfy bare minimum requirements.

David: Somebody sent me a question overnight and basically the question was can an organization get both reports, a SOC 2 and SOC 1, in order to hedge their bets so to speak in terms of what is going to be most acceptable. The actual standard says that you cannot mix the two, which means that you cannot produce a SOC 2 and a SOC 1 report together using the same header and everything. As to whether or not you can produce two distinct reports, the answer is, yes. There is nothing in the standards that says you cannot deliver two separate reports. One is SOC 2 and one is SOC 1.

I would precaution anyone that is considering doing that, because of the things that we have discussed here and whether or not the controls all relate to ICFR. If you are going to provide both reports, you need to ensure that the SOC 1 report has relevance to the control over internal financial reporting. If you are a data center and all you are doing is providing racks and pipes, you have a longer ways to go to justify why you are giving someone an SSAE-16 report. When you really get down to brass tacks, there is really not a lot of ICFR involved housing their servers. They control everything else. They are controlling the initiation, processing, recording, etc. All those thing we talked about in the earlier slides.

April: Thanks so much. If you come up with any follow-up questions or you would like to get in touch with our guest speakers today, I will post my email address. Feel free to send me an email and we will connect you with the right person. Jon or David, any last thoughts before we sign off?

David: I have the additional resources page slide up here (view slides). There are a couple of websites. One is the AICPA itself and the SOC section of their website. The Cloud Security Alliance is an outstanding organization that is also trying to help alleviate some of this market confusion. SOC also has a lot of good resources. Here is the contact information for both myself and Jon. We are both on Twitter and we will occasionally comment on other things. we are both active bloggers. So if you look us up, you’re probably going to see us trying to get this message out there.

Jon: Google the Risk Assurance Guy ( a personal blog there), but yes the Cloud Security Alliance for sure. We didn’t even get to the best part, which is how we combine some of the additional subject matter such as the cloud computing controls matrix in a SOC 2 engagement. We’ll save that for the next time.

April: Thank you. I think a follow-up on the Cloud Security Alliance is definitely in order. We;ll schedule a follow-up and let everyone know when that is. One last question here to squeeze in, do you recommend that a business conduct an audit annually or lessor more often?

David: Well, the standards for the SSAE-16 is annually. It can cover less than a year, but for the purposes of complying with Sarbanes-Oxley, when you have a justifiable need for an SSAE-16, it has to be done at least yearly.

April:  Is that the same for SOC 2?

David: For SOC 2, you don’t have to do those on a yearly basis, but it doesn’t make much sense create one one year and then let it go the next, because you want your customers to have assurance more than that spot check. You have to go back to those slides from earlier and try to find out the right type of assurance level and definitely a coverage for a period of time is much better and you want to renew that at least annually.

April: David, Jon, Jason thanks so much for sharing your expertise with us today. We look forward to talking with you all again soon. Feel free to email us, find us on Twitter, LinkedIn and we hope to connect with you all soon. You can email me at Thanks for you participation today everyone.


Webinars    |    Online

Get started now. Exceptional service awaits.