How confident are you that your organisation’s approach to testing will deliver quality outcomes and solutions that perform as required?
It’s a ‘piece of string’ question: how much testing is sufficient for your organisation? The amount and type of testing undertaken within an organisation is often as much a habit as it is any risk-based assessment.
The key questions are: Does the level of testing historically performed within your business give you the confidence that projects will achieve high-quality deliverables expected by the business? Are you confident that the quality standards will be met and that the end solution will perform to the level required?
Software Testing – A Dark Art or Just Misunderstood?
Software testing is often misunderstood, underestimated and under-appreciated. In some organisations, it’s considered something of a ‘dark art’.
This lack of understanding can have a massive impact on the quality of the end deliverables and increases the risk of releasing a substandard product onto the unsuspecting end user.
Depending on the sector this can be a minor inconvenience (beta testing of apps) or a catastrophe event (banking security failure, defence systems crash).
So how do you assess the right level of testing for your project? We explore below some of the key considerations that should help establish a threshold for testing.
Do You Have the Right Team?
The first step is to ensure that there is sufficient experience at the test strategy and planning table.
Testing is a qualitative – not quantitative – exercise. There is simply no replacement for someone with the requisite knowledge and experience, who has made mistakes, had successes and knows where the risks and issues are when developing the testing strategy and plan.
RBT (Not Random Breath Testing)
Risk-Based Testing (RBT) is fundamental to developing a sound test approach. Understanding the acceptable risk level for the project and getting business input is crucial.
The higher the risk tolerance, the lower the emphasis is on quality. It is essential to understand the level of risk the business is willing to accept and then demonstrate to them that risk and quality have an inverse relationship.
This can also be sector and compliance driven, for example, the risk and quality approach for testing a fighter plane’s navigational systems is vastly different from a simple iPhone app.
You Will Always Run Out of Time and Money
As budgets go, you can always bank on the fact that there will never be enough time or money to achieve everything you planned to do from a testing perspective. Knowing the time you have and what financial constraints exist ensures that you prioritise the key focus areas.
Cost and Impact if it Goes Wrong
Cost and impact is part of the risk-based assessment but worth a special mention.
The cost and impact of failures (large or small) at each stage of the project changes, for example:
- At the development stage the costs and impact of a bug is minimal.
- At internal testing phase a bug will increase both cost and impact but is in a controlled environment.
Once live and open to the public the cost and impact can be significant. Just ask any bank that has had a national failure of its transactional systems. As with RBT, this is something that the business must be informed of and decide on the risk level they are willing to accept.
The complexity of the project will impact the nature and level to which you test. Complex projects may require several test beds across various platforms and environments and need to cater for multiple scenarios. Additionally, the performance of the system may be critical and require additional forms of testing, such as load or stress testing.
The next consideration is the capability of your team, and you must ask three critical questions:
- Have you had a chance to hand pick your team or are they inherited?
- Do they have deep experience with this type of project and are they skilled and experienced in the forms of testing you will undertake?
- Are they engaged and available?
These questions will enable you to assess if you’re resourcing model is fit for purpose and the approach and level you will be able to test to.
More is Not Always Better
If and when you do run out of time, throwing more people at a testing problem can help.
But it is not a linear relationship and getting more people involved in testing does not guarantee the speed and quality of the outcome is improved. If they are the wrong people in the wrong context their involvement could in fact make a bad situation worse.
Testing – A Necessary Evil or Fundamental to Success?
Some project managers and stakeholders see testing as a necessary evil that they “have” to do that just soaks up valuable time before getting the system into production: it will always take too long, cost too much, doesn’t help build the product. In this scenario ensuring these people help decide the level of risk using a risk-based testing approach is ideal.
Other project managers and business stakeholders understand that testing is about quality and test teams are integral to achieving a quality outcome. This attitude gives you more latitude and scope to dive deeper for longer where necessary.
Either way understanding the landscape and constraints you are working within always helps when planning the appropriate type and depth of testing for your project.
We believe that quality thought leadership is worth sharing and encourage you to share with your colleagues. If you’re interested in republishing our content, here’s what’s okay and what’s not okay.
To speak to our team about how we can help your business deliver better projects, please contact us.