Forums: Why Most Security Audits Are Crap. - Forums

Jump to content

  • (2 Pages)
  • +
  • 1
  • 2
  • You cannot start a new topic
  • This topic is locked

Why Most Security Audits Are Crap. (Confessions of a consulting whore)

#1 User is offline   Blake 

  • Former Commander In Chief
  • Icon
  • Group: Retired General
  • Posts: 7,316
  • Joined: 24-September 02

Posted 07 October 2004 - 11:36 AM

I admit it I have been a security consultant for over a year now and unfortunately I don't feel that I have brought much benefit to my clients. My open conclusion is that the overwhelming majority of all security assessments are garbage. The only time these assessments hold any water is when used to verify that current security measures are in place. Why you ask? Well most would respond, You are obviously not doing a good enough job, if you bring no value, in response I would have to say yes.

Time is Money
For any consulting company to maintain profitably this day and age, (especially in a mid-sized banking sector) The consultant must perform multiple jobs at the same time, as well as perform with as little resources as possible. The idea of a security team coming in with experts from each genre of security from policies to technology is overwhelmingly gone. In most cases you are performing the test yourself with maybe some phone support from your peers. Though this produce the most cost effective results I doubt it is giving that much value to customer.

True Security Pro's are Scarce.

It is difficult to find true security professionals this day and age. Since security has become the latest sexy field the market has become flooded with professionals who are not exactly the best at the craft. I find the most common flaw to be the inability to actually think dishonestly. These may seam easy to some but others have such strong moral fiber they fail to realize that other do not. When testing many applications and showing the developers or customers the first response from them is, Why would you do that?. My answer is always , because I can. We need security pro's with the ability to think that way rather then recite what they have read form a book.

Who is at Fault?
Well, frankly it is everyone. Customers don't have the money to spend and consulting companies are eager to make up lost earning from the great economic drought we have been having. Instead of refusing work because it is not cost effective, consulting companies simply cut down quality to meet the demand price.

What is the solution?

If you are a mid sized company that does not have federal or business partner requirements for a penetration test or security assessment then your many is best spent in highering a network or systems administrator with a strong security background. You will get more for your money of you take that 20 grand and add it to the budget you are going to spend on your next position. Remember you get what you pay for.

GSecur
0

#2 User is offline   Nick W 

  • Master Sergeant
  • Icon
  • Group: Members
  • Posts: 1,250
  • Joined: 12-August 03

Posted 07 October 2004 - 03:21 PM

/agree.

Man, you nailed the head with this one:

Quote

If you are a mid sized company that does not have federal or business partner requirements for a penetration test or security assessment then your many is best spent in highering a network or systems administrator with a strong security background.


The problem is that security auditing is an "instance" solution to a "constant" problem. It's not that security auditors don't know what they are doing as much as the field is constantly changing. What might be a good exploit today might not be in use tomorrow. A good example of this is the HTA object data exploit. Once it became widely used, AV software started putting in heuristics scanning on .hta and .vbs files, Microsoft started patching methods to get binaries to the workstations, and companies like PiVX with their "solutions" dried up literally overnight.

Another problem is web security. For the most part, IIS/MSSQL and Apache/MySQL are now at the point where they are secure from random external attack, but attacks on code and even custom code have evolved and are thus, "the new thing". It would be better to keep your private code private and have an interal security auditor review the code.

Every single point you addressed you found the real problem. Time is indeed money, bringing in a team to learn how you do business takes too long and costs far too much. A good deal of security experts are really just security consultants that read from a script on what should be done. They don't actually know WHY it should be done.

All the "good" consulting companies hire a great deal of individuals (power is assumed just by #s) that can be trained to follow a book or course. It doesn't solve the ingenius hacker that will use a combination of social hacking and ethically see no problem in throwing up signals on one computer to throw the administrators off the heist he/she is doing on another.

In the meantime, the really good security experts are being skipped over for promotions and such because they "waste too much time on unimportant items".
0

#3 User is offline   uko 

  • Private First Class
  • Icon
  • Group: Members
  • Posts: 42
  • Joined: 05-September 04

Posted 07 October 2004 - 04:12 PM

Interesting thread GSecur. In a broad kind of sense you’re absolutely right. This is a topic that can be delved into very deeply. I go through this pain every 6 months when my organisation has bi-annual security audits and pen testing (remember there is a difference).

Being involved in many audits and pen tests I’ve come to see the difference in a good audit and a poor audit. I’ve found there are many ways to get the most out of an audit. One of the most obvious is you need clear objectives and goals in mind. Be it from the Consultant’s point of view or the organisation’s view. And I’m not talking about a goal like “I want a more secure network”. There has to be a bit more substance to it.

I remember one of the better security consultants I worked with. He said that he wasn’t going to audit the network on industry best practices but rather on what we classed as best practices. The audit went horrible, we got a poor review, we couldn’t define what was good or bad. As a result we created a 100 page IT Security Policy. We now have structure and a clear definition on what is classed as good and bad practice. We now get high praise from security consultants and pen testers find it hard to hack our network.

But I think I’m starting to drift from your topic…

Out of curiosity GSecur, what do you do to make an audit more successful for the customer? What would you define as a good network and a poor network from a consultant’s point of view?
0

#4 User is offline   beardednose 

  • Forum Moderator
  • Icon
  • Group: First Lieutenant
  • Posts: 1,834
  • Joined: 23-May 03

Posted 08 October 2004 - 04:22 AM

I just received a pen test from an outside firm and found it very helpful. It has been more helpful in cleaning up some outstanding issues, even more so that y2k or Sarbanes-Oxley.

Some of the issues I already knew about, others I did not. The best part was the issues that we discovered while gathering the info for the test. Those were the biggest issues we found, and we found them internally (they were outside the scope of the test, that's why the pros didn't find them).

The presidents and CIOs literally "pushed out bricks" when they saw the results.*

We paid a heck of a lot for the test, but it was worth it. The issues I had with the whole thing are as follows:

- I knew about many of the issues and suspected others, but until management saw it from an outside firm, they would not act on it. Even after I hacked systems and gave them some of their most personal info.

- Although we told management that doing the test would only result in large, expensive projects that disrupt the business, they seemed surprised when the results came back. They are complaining about the cost, resources consumed, and disruption to the business that the projects are causing. Surprise!

- Although the firm were pros (everyone would recognize their company), I had to stay on top of them. In addition, the spelling and format of the final report wasn't as good as I'd hoped. Althougth I know better, I was hoping to not spend as much time managing them. They were very good and earned their money, no doubt.

- I was dismayed to find that some people on our internal security team were using poor passwords. That really pissed me off. If the security folks don't care...I get tired of constantly checking up on everyone and then being the bad guy whacking people. Ya, I know, that's my job.

- Management is too knee-jerky. I've been able to talk them out of most of their overkill reactions. They don't realize how much $ I saved them there (and business disruption). It's hard to tell management that won't give your resources and $ most of the time that they're trying to throw too much resources and $ at a problem when it can be done cheaper with less work, but I'm trying. <_<

- Now that I'm managing the remediation of worldwide systems, they can't imagine why I can't get much else done.

And finally, I'm still trying to get them to see that if we don't enforce our standards and policies and review servers before they go on the Internet, we'll be doing the same thing again in 6 months (the remediation). We're making progress in that area, but of course it just means more work for me. But eventually, it will mean less. In the meantime, I'll keep outsourcing some of my security projects.

Time to reorganize the security team too....

* You know what got em? One of the presidents recognized, in the list of files that could have been compromised, a document re: a pending acquisition! (Now I know their hot button, so I'm looking internally for more <_< <_< <_< ).
Don't post just a THANKS! Here's why...

Forum Rules you need to know...RuLeS
0

#5 User is offline   Skydriver 

  • Private First Class
  • Icon
  • Group: Members
  • Posts: 102
  • Joined: 27-December 03

Posted 08 October 2004 - 04:58 AM

i agree with GSecur most of the security managers in the big companies don't want to spend allot of money on security but want to be secured and most of them even don't know what to test and they are spending there time and money on the wrong places.
it remind me that I was in one of our clients a few days ago and he wanted to test is network so in the time I had to start with a scan of all the network from there I found some vul and used them to got into one of the major servers from there I jumped to other major servers and from there to the DC of the Org. and became a administrator of all the network so after he saw that he wanted me to hack into the network devices I did that too and found one router with SNMP broke from there and got the enable pass but this wasn't enough from him so he wanted me to hack to the Unix machines and my hours were over already so I scan the machine and found some vul that I need to rewrite an exploits in order to hack it
so I told him look if u want me to hack to this it will take me some time but believe me the server is vul and it will be more easy for u to upgrade the version, but no he wanted me to prove it like it wasn't enough for him that I am administrator of all the network and I hacked almost to every server in the network and to every network device.
I think the main problem that most of the people that are Company managers Security don't know what to do and sometimes just care about the paper that saying that the network or the application is ok even if they have been told that the job will take (lets say 30 hours) they will take the job 10 hours and just want the paper that saying that everything is ok and wouldnt care about the real security of there company.
0

#6 Guest_mongo_*

  • Group: Guests

Posted 09 October 2004 - 08:18 AM

Well infosec is still a mystery for most organizations. Most executives believe the CIO is responsible for all aspects and that the solution is the addition of another firewall or ids; a pen test will verify that I am secure and can take it easy. It takes a a major incident to shake an organization out of it's complacency and begin to look at the fundamentals of securing the organization.

There are a broad range of consultants with varying levels of expertise in the security field out there who provide value in some areas while avoiding other areas that need to be addressed. Not every consultant is a hacker (in the true sense of the word) capable of uncovering every obscure risk and addressing them.

Pen Testing plays an important role is assessing the current state of the environment BUT is limited to the technical aspects. Audits can provide an excellent accessment of an organizations security posture BUT once the report is in, it is the organizations responsibility to accept the findings and implement the changes.

The challenge is finding a way to convince the organization that the dollars spent to identify and mitigate risks/threats/vul'ns is less than the dollars required to respond to recover from and remediate a major incident. There has been some great work on the development of a ROSI metric that translates security costs to quantifiable returns. This seems to be having an impact on executives (particularly in light of the legislated requirements) and shining a light on the need for incorporating security into all aspects of the organizations.

Of course, as a consultant, the trick is for find an organization that understands this and is willing to invest in and implement security related initiatives.
0

#7 Guest_SyN/AcK_*

  • Group: Guests

Posted 09 October 2004 - 09:36 AM

Hmm... this may be one of the first times I've had a severly contrasting oppinion on something you've posted on the site GSecur. Your comment about money being better spent on a Network or System Administrator with a strong security background is very true, but not feasible. I mean, take that a lot of Security PROFESSIONALS are not really even qualified to handle a network's security (or more importantly a company's security posture), its even harder to find Network and System Admin's with respectible enough skills to secure a network. I mean, remember why Security Professionals are here in the first place, there was all these insecure networks.

Something does need to be done about the quality in the industry... all to often I hear about a "company" simply running a Nessus scan and calling it good. Yes, I do the same, but I go into huge amounts of detail, I offer to help reconfigure or setup new, more secure services, and besides that, I feel like I provide a measure of education. I hope that when I leave an audit, the entire companies security posture is stronger, and even if they don't know where all the problems will arise, at least they have there eyes open.

Think of yourself as a doctor.... a lot of times, your job is not to cure something, but to provide a better quality of life. As a security professional, it is not your job to ensure that a company stays secure forever, thats impossible, but you try to instill policies and procedures that will do the best job they can to make sure that the company is at least not making decisions that will make them more insecure.
0

#8 User is offline   Blake 

  • Former Commander In Chief
  • Icon
  • Group: Retired General
  • Posts: 7,316
  • Joined: 24-September 02

Posted 09 October 2004 - 11:12 AM

Most of the time many of the security issues discovered could have easily been found if the engineer had just run nessus.

The times that an audit actually is worth it's money is when an engineer needs to have proof to get his point across to managment. An audit should also be performed on Security Policies, since regulations quickly chnage and can be difficult to stay abreast of.
0

#9 User is offline   TK_man 

  • Private
  • Icon
  • Group: Members
  • Posts: 9
  • Joined: 23-April 04

Posted 10 October 2004 - 06:26 PM

I agree with ya Gsecur. Personally, I find the whole industry very amusing. To many CISSP's not enough injuns! I personally know CISSP's that don't understand subnetting, or have trouble differentiating between TCP and UDP. Cmon people wake up and smell the coffee. The root cause of the madness, is Yuppie fu*ks that make good money in the industry yet truly haven't got a clue. If it was up to me, nobody gets a job in computer secrurity, unless they were, are, or are striving to be a hacker. INstead of studying countless volumes of worthless info for months at a time in the purely selfish endeavor of getting certified, perhaps one should sit down with a few machines in the lab and hack the crap out of them. Now that is time well spent. Talk about being able to defend a network! Another big pet peev I have, although I don't blame them, is the infamous security product vendor and their never ending array of smoke and mirror technology! Cmon, IPS the next big thing? Give me a break. YOu get my drift.... Enough for now, gotta get back to my well paying security job! B)
0

#10 User is offline   Spookie 

  • Staff Sergeant
  • Icon
  • Group: Specialist
  • Posts: 293
  • Joined: 21-December 03

Posted 11 October 2004 - 10:29 AM

This is really a great thread as I've been watching it grow over the past few days. One thing that I think was mentioned by uko was

Quote

One of the most obvious is you need clear objectives and goals in mind.
This is so true as one has to remember as the concultant your position is to so to speak "Consult" In the end the company providing the paycheck will state what objectives they wish to accomplish. And yes it can be frustarting when you "advise" the elements of discoveries made while accomplishing the goals and objectives of the "pen-test" and 6 months later the problem still exists. You've done your part- you mad enote of it, reported it, documented it, presented it. It's up to them to fill in the blanks, as your job was to consult. One must remember their is no such thing as a 100% secured network.

Yet you will have those within the various disciplines of InfoSec and NetSec that will say you can. When you ask them what principle security model they invoke and you get the deer in the head like look from them you already know in your mind your in for a long ride.

I also agree to a certain extenet with TK_Man when he said

Quote

To many CISSP's not enough injuns!
There are CISSPs out there who have the ability to have been able to sit in a classroom and be able to retain the flood of information thrown at them, that have absolutely no clue as to the differences of malware. Sad but true.

Then you have those in the InfoSec field that try to diminish a certification for their own companies profits because they have the ability to be seen or heard. As an example some time ago there was a comment regarding the C|EH certification.

One magazine in specific had a write up that was down right scalding of the class. But if you read between the lines you would know
1. That the author of the column is part of a company that charges an arm and a leg for "their" own Red Team to come in and do an assessment.
2. That at the end of the actuall class work, he bailed and failed to particiapte in any of the classroom activies or war games.

That same author wrote a scalding article on one of the few Pen-Testing tools out there.Once again the same company has their own pen-test tool that they also charge an arm and a leg for ( Your up to 2 arms and 2 legs now) So here you have one author writing 2 columns both critical of the competition and they feel they've done there part to continue in the "fruits of labor" in the good name of their company and rate this years christmas bonus.

But the backlash was more interesting as most companies didn't even know what was the C|EH certification. Now it is one of the most sought after certs in the IA field, Next to Wireless and Forensics. Who wants to pay the uge costs of a outside team coming in to tell you your babies ugly" They already know that so they'll save a few bucks and hire there own guy, who has the basic hand on or knowledge. In some cases the new hires, a lab and equipment is still several grand cheaper than some of these test for big companies. As for Forensics, I'm all for the part of breaking the cert into 3 parts and having the student recovery their cert to pass the class. Thats after the written test. Can't recovery the cert, no problem you fail try again in 6 months.
Fail again come back in a year. This way you have the book smarts to know the rules and meathodolgy and the know how to physically do the job.

When you have conferences, majority of the time all you hear is about what the future holds and all the new toys and gadgets coming out that will help secure the networks from the "bad guys". You hear the " We've done this and we're doing that" when in reality they've been yanked over the coals and embarassed several times.

The biggest problem I see in the various disciplines of IA is the lack of communciations. No one really talks to each other- Job Fear, lack of knowledge, or plain don't know is the culprit but yet when you ask a directed question you always get the same response from most of the people in the positions to make a difference.

" I know"
Beauty is only a light switch away
0

#11 User is offline   Ragabash 

  • Private
  • Icon
  • Group: Members
  • Posts: 7
  • Joined: 28-September 04

Posted 12 October 2004 - 11:59 AM

Ok, as most of you posting here are already working in the security-branch, i'ld like to ask you guys something.

I just started studying informatics, and going for networks and security...

Can you give me some advice as to what i definately should do, and on the other hand, what i definately shouldn't ? For instance, like TK_man said, be good at hacking systems to know their weaknesses..

This would be really helpful and very much appreciated

I already stepped back from gaming all the way (you've got to do something in high school ;) ) to maybe like 1% entertainement now, and the rest of my spare time reading and learning on the subject.

grtz
0

#12 User is offline   Blake 

  • Former Commander In Chief
  • Icon
  • Group: Retired General
  • Posts: 7,316
  • Joined: 24-September 02

Posted 12 October 2004 - 02:42 PM

What the hell is "informatics"???
0

#13 User is offline   TK_man 

  • Private
  • Icon
  • Group: Members
  • Posts: 9
  • Joined: 23-April 04

Posted 12 October 2004 - 03:01 PM

Main Entry: informatics
Pronunciation: "in-f&r-'ma-tiks
Function: noun plural but singular in construction
Etymology: International Scientific Vocabulary information + -ics
chiefly British : INFORMATION SCIENCE
0

#14 User is offline   Tyrano 

  • Staff Sergeant
  • Icon
  • Group: Members
  • Posts: 296
  • Joined: 13-February 04

Posted 12 October 2004 - 09:08 PM

I've been pondering for a few days on how to respond to this. There are certainly some valid points, but I think a few key areas have been missed here. Or perhaps I have missed something and this entire post is bogus. Regardless, onwards ho!

Firstly, consultants are hired after an issue or problem has been identified. This may very well be your "It's audit time, lets get someone to test our security policies" scenario. If there have been no recent security issues that management has identified, its highly unlikely they will see any benefit in paying consultants to harden their networks further. Or at all. :P If they do hire consultants, its going to be because they don't have the staff, the staff doesn't have the expertise, or the company can not afford to delegate time to this issue. This means that even if security consultants are brought in, they will probably be very limited in their scope, unless they've been contracted to perform a complete overhaul. So obviously, some issues may be overlooked, or if they are discovered will have to be left up to the staff to remedy. These guys care about the bottom line, returns on investment, and the buck stopping here. So yeah, it may be the case that your security audit is going to be crap, but thats just something you will have to deal with. Obviously advise and document everything you stumble upon outside of your assigned task that you believe should be changed, but leave it at that. Who knows, it may become a recurring contract. It's just something consultants need to deal with. Document document document.

For application auditing, I don't know much about it so I can't really comment on it. I guess just be proud that you are one of the few who can think outside the box when it comes to breaking stuff. It's definately not something you can learn in a class, and I think the cert-horde is beginning to realize this. Hopefully this will help weed out the certified from the talented. I think it's going to be very difficult to prove to people an auditor can do this, except with experience. I don't think placing "I THINK LIKE A CRIMINAL" on ones resume is too bright.

Personally I think the industry is at fault. The FUD (fear, uncertainty, and doubt) a lot of these security companies create to sell their products and services, the plethora of useless certifications that can show you know how to study, and the slow patching processes of most vendors are more at fault than cash-strapped businesses. As for consulting companies lowering their quality, the effects of their previous work will surely show up when some attack occurs. And then their rep goes down, and it is all about rep. :blink:

I totally agree with GSecur though on hiring a security conscious administrator though. You can pen-test all day long but if the IT department can't fix the problems, or prevent them from ever happening, you need a new IT department. Sure, some things are unavoidable, but getting slammed by the latest worm because your patches weren't up to date because you were too busy doing something else is just irresponsible.

</rant>
0

#15 User is offline   Ragabash 

  • Private
  • Icon
  • Group: Members
  • Posts: 7
  • Joined: 28-September 04

Posted 13 October 2004 - 12:59 AM

Gsecur, on OCt 12 2004, 10:42 PM, said:

What the hell is "informatics"???


TK_man, on Oct 12 2004, 11:01 PM, said:

Main Entry: informatics
Pronunciation: "in-f&r-'ma-tiks
Function: noun plural but singular in construction
Etymology: International Scientific Vocabulary information + -ics
chiefly British : INFORMATION SCIENCE


Ty :)

I'm not sure but i think the correct translation would be IT, you know, programming, networks, OS, ...
0

  • (2 Pages)
  • +
  • 1
  • 2
  • You cannot start a new topic
  • This topic is locked

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users

  • Share



Our Sponsors:


SwiftLayer Affiliate Web Hosting