The Twenty Laws of Testing Computer Software

As a software development project manager, I conduct, plan, organize and cajole the software engineering efforts in companies large and small. During the course of this work, I have never ceased to be amazed at the lack of understanding of both the importance of properly testing a software product or products, and the lack of knowledge around how to correctly conduct the testing effort.

This holds true in corporations both large and small that I have worked for during my fifteen year professional career. In my opinion, a Project Manager should have a complete understanding of the software testing process, and should also have experience not just scheduling and planning the resources conducting the testing effort, but actual personal testing experience.

It occurred to me earlier on in my career as a Project Manager that in order for me to be a better Project Manager, I was going to have to learn and research everything I could get my hands on about testing computer software. I took courses, I bought books and read them; I related the information I gathered to my experiences as a developer and in some of the ecommerce companies I had worked for and built early on in my career.

I found that this desire to learn the ins and outs of testing was over half the battle towards becoming a more accomplished PM. The Project Manager who appreciates the importance of testing, has been a tester, knows and respects the testers on the team, and has a deep seated, fundamental respect for testing is a Project Manager who commands respect from his project team.

One of my favorite books is “Microsoft Secrets”, by Michael Cusomano. In the book, he describes how early on in the history of the company, testing became a career path on the same level as programming. Knowing, from my extensive reading about Microsoft and Bill Gates, the high altar upon which programmers are placed at Microsoft, I found this to be extremely significant.

Great software development teams and great software engineering companies take the testing of their software seriously. They don’t cut corners, and they don’t have to, because they began with the end in mind.

So without much further adieu, here are my twenty laws for testing computer software. Look for me to expound upon each of the twenty laws in more detail on these pages very soon:

1. The sole goal of testing software is to find errors. Software testing is defined as the method of running a computer software program with the intent of discovering errors in the computer software program.
2. The definition of a good test case is that a good test case is one that has been written in such a manner that it has a great chance of discovery of previously undiscovered errors.
3. A successful test case is one that has been used to discover a previously undiscovered error.
4. Only a high quality software testing process will result in a high quality software testing effort.
5. Testing computer software is a professional discipline that must include skilled and trained professional computer software testers.
6. Someone must assume full responsibility for the improvement of the software testing process.
7. It is vital to foster a 100% positive, inclusive and team-oriented approach with a “test to break” mental attitude.
8. A test case for testing a computer software program must include a definition of the expected result of the computer software program being tested.
9. A computer software programmer should not test the computer software program they have coded themselves.
10. By extension, a computer software programming organization or engineering department should not test its own programs; This is the work of an independent testing organization.
11. The results of each test case should be reviewed with great care.
12. Test cases should be written in order to include unforeseen and invalid user inputs, as well as foreseen, valid user input.
13. Testing a computer software program to insure it performs as it should is only fifty percent of the testing effort. Another fifty percent of the testing effort should be expended in order to insure that the computer software program does not perform in ways in which it should not be performing.
14. Avoid one-time, spontaneous, disposable test cases.
15. A testing effort initiated under the assumption that no errors will be found will not be a successful computer software testing effort.
16. The proliferation of errors in a computer software program can be prevented through the employment of testing during the early stages of the software development lifecycle.
17. Software testing tools can be and should be a key element of a software testing effort.
18. Although perhaps counterintuitive, the probability that more errors will be found in a section of a computer software program in which errors have already been found increases with the number of errors discovered in that section of the computer software program.
19. Testing computer software well is an extremely mentally challenging exercise that requires creativity and perseverance from the testers in order to succeed.
20. The perception (oftentimes forwarded by management) that “not enough time exists to test the product properly, so let’s just ship it anyway”, because the “rewards of shipping the software outweigh the risks of shipping the software with undiscovered errors” may still be common practice in many software development and engineering organizations, yet such an attitude will lead to catastrophe, as software quality is intrinsically linked to customer requirements and customer satisfaction.

Want to know more?

You’re reading Boston’s Hub Tech Insider, a blog stuffed with years of articles about Boston technology startups and venture capital-backed companies, software development, Agile project management, managing software teams, designing web-based business applications, running successful software development projects, ecommerce and telecommunications.

About the author.

I’m Paul Seibert, Editor of Boston’s Hub Tech Insider, a Boston focused technology blog. You can connect with me on LinkedIn, follow me on Twitter, even friend me on Facebook if you’re cool. I own and am trying to sell a dual-zoned, residential & commercial Office Building in Natick, MA. I have a background in entrepreneurship, ecommerce, telecommunications and software development, I’m the Director, Technical Projects at eSpendWise, I’m a serial entrepreneur and the co-founder of Tshirtnow.net.