Abstract: The Testing Manifesto is an encapsulation of a what some of us, context-driven testers, believe the role of “Tester” to be. The skill-craft of testing can be too blurred in many environments, such that we thought this was necessary to put out there. While we’ve used this internally for a while now, we were prompted to share this after seeing a recent tweet gain multiple likes and retweets, which on the surface seems noble, but is actually part of the misinformation out there on what the role of testing actually entails. Testing is not Product Management; Testing is not Programming. At my company, this is used as a guideline, in conjunction with the Agile Manifesto that drives the higher level team processes.
A while back, we all got a good look at the Agile Manifesto, along with the twelve principles (or combined into one PDF here) which puts the focus on coming together to collaborate and solve problems. Now, while collaboration is an excellent way to generate solutions, as leveraging the wisdom of the crowd is a valuable practice, this can sometimes become a driving forces that puts too much emphasis on cross-functionality and push craftsmanship to the back burner. The Agile shops we’ve witnessed fall into two main categories, collaboration vs craftsmanship. Don’t be confused, these are not completely opposed camps, and the latter is not devoid of the former. A good Agile shop that centers around craftsmanship will of course also leverage collaboration as a part of that, but the implementations that make us wary are those that attempt to push everyone to do everything, when a team of specialists with some overlap may actually be a much healthier approach to solving engineering problems.
The Testing Manifesto
Long story short, in order to put some emphasis back on the craft of testing, and be sure that the role testers fill is clear and does not get pushed to the edges, we have put together this Testing Manifesto (PDF).
This is meant to compliment the Agile Manifesto as the two are to be used in parallel; one is not meant to replace the other. We hope that you find value in this and can use it to help discussions with some people who could be considered more Agilistas, and need help finding that balance between the focus on collaboration and craftsmanship. You can have both, but there’s a healthy balance that many do not attain. We hope that putting this out there publicly, will help teams move the needle more toward a balanced perspective. If we’re truly context-driven, then we must admit that craftsmanship can’t always be a low priority; it matters, thus a mix of generalists and specialists may be needed to effectively solve many of the modern engineering problems we face in the industry.
4 thoughts on “Testing Manifesto”
After reading the manifesto, my first impression is that the tester is considered as a “humble servant” of the developer. It seems to me that the developer is considered as the “center of the universe”, around which everything revolves, including the services of the tester.
We have to be aware that it is not the developer who takes the final decision on releasing a product. In the end this decision will be taken by senior management in deliberation with the business unit. In order to make a well-founded release decision, these people need valid, usable and independent information regarding the quality level of the product that is about to be released. It is the tester who can and should provide this information to them, so these final decision makers are a very important stakeholder for the tester and the test results.
I miss this completely in the manifesto.
[Connor’s reply: Yes, as testers, we operate in a role of service to the team. This includes programmers, Product Owners, Scrum Masters, other internal stakeholders, external stakeholders, etc. There’s a big difference between providing service and being subservient and this manifesto makes no claims to, nor supports, the latter.
To speak to your second point, nothing in here contradicts the fact that Product Management makes the release decisions, and we’re in full support of that fact; knowing where the role of tester ends. In fact, this supports that in the main principle by speaking how we (testers) are not gatekeepers, so I am unsure how you would have missed that. This is a testing manifesto after all, not a blog post about the relationship between testing and product management. Perhaps that is what you wanted? If so, feel free to read more on that here.]
Basically I agree with most of the statements that are mentioned in the Manifesto.
I fully agree on the statements in the first part of the manifesto (above the “twelve commitments” to the developer).
Regarding the 12 principles of software testing; these are presented as “commitments to the developer”: Personally I wonder why the focus is only on commitments to the developer, and not to other stakeholders like the product owner, project lead, and quality lead.
[Connor’s reply: First of all, let me state that I think every tester is a “test lead,” regardless of whatever title your company has given you. Also, since quality is the job of the whole team, there is no “quality lead” in my opinion. I am being very intentional about delineating between the terms “testing” and “quality” since they mean very different things.]
Basically I agree with most of these statements, however for some I do not fully agree. I will elaborate on those:
(1) I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied.
[CP] This might not always be the case. There are situations in which the developer can be perfectly satisfied with the result of his work, while the tester is not satisfied because the quality is not on the required level. In that case there is a conflict of opinions between the tester and developer, which might require escalation.
[Connor’s reply: Again, you keep talking about quality, but we’re talking about testing here. In this point, we’re specifically talking about satisfaction with the testing that has been done, not satisfaction with the product. In other words, I as a tester will not stop doing my job of testing (learning, exploring, informing, experimenting, questioning, etc.) simply when I feel like it if you, the developer, still have concerns about the status of the product, the methods I used to test or the quality of that testing.]
(3) I will test your code as soon as I can after you deliver it to me. I know that you need my test results quickly (especially for fixes and new features).
[CP] This is indeed a good starting point, but it is not always possible. Especially if the tester has to test the deliveries of several software teams or perform consolidation testing, it is not always possible, from a planning point of view, to test the code on short term. The tester has to plan this test activity in his own activity schedule. A developer cannot simply assume that a tester is available every moment of the day to start testing his code.
[Connor’s reply: Saying “As soon as I can” already suggests it is not “always” possible; however, we’re context-driven, so obviously we’re talking about guidelines and heuristics here. This is a “when possible” statement, so you could phrase it that way if it makes you feel better. That does not change the meaning. As far as schedule and not being available – Why? Testers are in an act of service, and as a tester, I am available at most any time to pair with a developer for the purpose of learning and exploring the product together. Many times I do this before even a single line of code is written. This has many benefits, because not only does the developer know where I am headed with my testing, and thus codes more proactively, but he/she makes the code much more testable because now there’s a shared understanding of the end game. Much less back and forth post-code commit; much less legacy style thinking of testing as a “phase” that comes “after” coding. We’ve moved away from that, and now embrace a coding/testing overlapping paradigm. Remember, testing is not just the act of using the software; it is questioning, discussing, challenging, etc.]
(4) I will strive to test in a way that allows you to be fully productive. I will not be a bottleneck.
[CP] Also here it is implicitly assumed that the tester is always available to test the developer’s code at any time, in order to avoid being a bottleneck for the developer’s activities. This is not always realistic, since the tester has to address the needs of other teams or has to perform other activities.
[Connor’s reply: Again, read the language here. Nobody is saying “always” or “never”. There are no best practices, so read this for what it actual is and not perhaps what you’re filtering it to be. In this case we say “I will strive to…” Notice, it does not say “I will always…” Be sure to read with that in perspective. We’re not coming at this from a standards-driven mindset. We have an allergic reaction to best practices, and already use language that supports that.]
I hope this clarifies a bit why I made the statements in my first comment.
[Connor’s reply: Let me know if it helped at all to point out that the language/wording used in this is in fact not as black and white as you may have originally taken it. And of course, all comments are welcomed.]