so true, so true.

There is no standard SQA/SQC organization or individual. There is barely even a standard for developers. I know of at least two major SQE certifications that differ wildly. There are at least two major Java developer certifications, but really only one .NET and one PHP (although multiple Oracle and MySQL ones).

All of these are better than Software Security Engineering, which has just about nothing -- even worse than nothing if you include CSSLP or GSSP.

There are a multitude of issues that arise with comparing developer-testing to quality testing to security testing, all of which are highly dependent on several factors. I would argue that since security is a requirement -- that if a quality test group exists, then that group would be somewhat responsible for testing for those security properties -- and that this would obviously be considered positive testing (not negative testing). However, this is a very popular argument, and somewhat obvious.

This is really uninteresting unless you are talking about case studies. Microsoft Press has a book on "How We Test Software at Microsoft". They have a lot of standards around development, testing, security testing, et al. Many other organizations are not as mature as Microsoft. They are somewhere else, but there is a lot of variety, even between market horizons -- ISV's and Enterprises -- as well as industry verticals -- Financial, E-Commerce, Health, Energy, Government, et al.

Some organizations don't have a well-defined software test group, or it's completely outsourced. Some like Google call their developer-testers SQE's, when really they are developers. I'm not sure that you can make these generalizations like you do.

Certainly, there is a lot of overlap between quality test automation tools and security testing tools, just like there is a lot of overlap between development tools and quality testing tools / security testing tools. For example, you'd see many a web developer using FireBug, but you'd equally see it in the hands of a software security engineer (even if it's used a bit differently).

I consider the quality testing tools to be better than a lot of the security testing tools today. For example, I much prefer Sahi, Selenium, Ghost Train, or Chickenfoot to the Ajax testing engines in AppScan, WebInspect, or OWASP Sprajax. However, it's probably much easier to use a tool like RATS or Fortify on the Javascript source code.

In the same way, fuzz testing tools such as Sulley or PaiMei (even file fuzzing) works a lot less efficiently than would using Klocwork K7 (defunct?). Even when using lcov or other code coverage statistics, it's much better to go directly to the source code for the bug hunt (or at least have it available to utilize these kinds of code comprehension techniques).

No, quality testing can and should focus on whatever developer-testing and/or security testing is missing. Of course it should be mostly positive, functional testing based on the use cases. However, using quality testers to augment or replace developer-testers and/or software security engineers is not something that I recommend.

Keep the three groups working on their separate goals, although they most certainly should be working together and cross-training on the areas/expertise where they do overlap. In other words, SDET's, SQE's, and SSE's all perform basically the same function and require very similar technical/tactical skills. But SDET's are rewarded for defect reduction before integration, while SQE's are rewarded before acceptance testing, and SSE's are also rewarded before acceptance testing -- but only on security-related defects. Strategically, they are working on completely different goals and business drivers -- and each requires its own set of background experience and knowledge that is vastly different than their neighbor.

Tools have nothing to do with this. There shouldn't be a DevInspect, QAInspect, WebInspect, etc. There should only be one. Can SDET's, SQE's, and SSE's all use FindBugs? Of course. Coverity? Yes, definitely. Canoo WebTest? Why not?

As you can see, I don't disagree with you, but I just think differently. Top X "bad stuff" lists are usually great for the tools, not the individuals or teams. Instead, those teams should index their findings by method/person/tool (probably using some sort of reputation system), and decide on their own which set of process, talent, and technology to put towards any given defect-prevention cycle or bug hunt.

"Developers by nature are detail oriented and typically (the good ones anyways) have a deep understanding of flows, and processes from start to finish" - so is a good tester

"The majority of QA people aren't interested in becoming security engineers or having a thorough understanding of vulnerabilities such as sql injection, os commanding, or http response splitting" - I wish I could disagree with this as a lot of the testers I hang around love knowing the techy details. Sadly though they are in the minority and I keep coming across questions from people wanting to know how to do security testing wihtout knowing even the basics

Interesting article apart from the phrase "QA tester" which I hate...

I'm going to echo my colleague Dennis Hurst's "so sad" comment - but I'm going to go a little deeper.

I've written some articles and have been giving [educational, not sales] presentations to QA teams for just over a year now and feel strongly (as you seem to) that QA doesn't necessarily need to understand security... but therein lies the problem of compartmentalization of issues.

Software "security" has typically been the *security team's* problem, and with that every time someone in a different group hears security they automatically assume security will take care of the problem. Unfortunately, those of us IN security understand that's simply not possible. This leaves us with a bit of a problem - but I'll address that when I publish my next article... for now I'll make 2 points

1) When you talk about terminology remember to use "defect" instead of "vulnerability". I've discussed this at length so I won't re-state but ask yourselves how those words are perceived by someone who is a QA engineer.
2) Consider workflow-based "security defects"... which 'scanner' products are historically tragic at finding (for good reason). QA testing is critical-path here... as they understand the possible use-case paths through the application.

Anyway, without getting too long-winded (oops) you're 100% correct. There needs to be more on this topic, and we need to reach QA departments with more than just tools and "gotta-do" orders.

Cheers
./Raf

@Phil

I don't disagree that good testers exist and are capable. I was making a point about the culture/skills/drive of 'average' QA people in my experiences.

We have been doing this training QA people around 8 years now with good/excellent results, so a bit suprised this is realigning. We are Fortune Global 200 oompany (don't want to be more specific as it reveals the company

@Anonymous
I never said that nobody was doing does this correctly. The general vibe from the security industry is to teach QA people a list of vulnerabilities/attacks from some 3rd party top ten list and either say they've done their job, or if it doesn't work out blame them for not absorbing the material as they'd hoped.

Gunner posted an entry today echoing this

"Good training programs should use wording that makes sense to QA", totally agree, they are not going to read the whole OWASP guide as much as we might like them to."

at http://1raindrop.typepad.com/1_raindrop/2009/02/allies-are-where-you-find-them.html .

Your company should talk about this as not many people are. Use the contact form to contact me directly so we can chat.

Having worked for companies supposed to be meeting DoD, FAA, FCC, or FDA standards - I can tell you little is being done to implement "Structured Testing" (Google "Structured Testing" +NIST to find), much less data driven testing or fault injection testing - and absolutely not mathematical modeling. Every manager I have worked with (or interviewed with) has said something to the effect of "Those things are way over our heads here - maybe someday...".

Since there is not a single QA/QC organization that requires an engineering degree to join, I see the situation as hopeless (IEEE/CS started a “Certified Software Engineer” program, but got scared and dropped “Engineer”). Good thing we have all the Six Sigma Black Belts to save the day (look at the MOT and BAC stock prices for the last few years to see how that worked out). Read up on Deming’s rules about “Drive fear from the organization” (20/70/10 rule) and “No slogans” (Six Sigma).