MCV/DEVELOP News, events, research and jobs from the games industry
QA has come a long way in recent years, but with issues around representation, automation and underappreciation, have things come far enough?
For many in the games industry, a job in quality assurance represents the first rung on the development ladder. It was one this humble correspondent was about to step on when his head was turned by an offer to join the staff of a leading games magazine instead. In the 25 years since that fateful decision, he’s often wondered if he made the right choice. Now, having heard from seven dedicated and passionate QA professionals, he realises he probably did and that the games he might have had a hand in were all the better for someone else’s being there instead.
We’ve checked in with seven top QA professionals to see where things stand. Next up is Neil Soane, UK MD for Quantic Lab.
Leaving aside matters of personal taste, critical acclaim and popularity, and wearing only your QA hat, which game – other than your own – has impressed you the most over the last 12 months?
So not a really recent release but I have been playing a lot of ‘Inside’ by PlayDead recently and I am really impressed with the general quality and polish on this title. The art style is incredibly beautiful (in its own way), engaging and draws the player into the dark (and frankly) depressing world. The mechanics are slick, and puzzles really need the player to think about what their role is and how they can influence their environment. It’s great to see titles like this getting the plaudits they deserve.
What’s been the biggest challenge that QA teams have had to face in recent years?
Being applicable. By that I mean fitting into the development or publishing pipelines and still providing meaningful feedback on both quality and quantity. Long gone is the time when a QA team could be engaged at a fixed stage of development and know their role would be done just before release. With games as a service, ongoing DLC and patching the profile of a game’s release is now stretched out over many months. The teams have to understand that they are likely in for the long run with the developer or publisher and a release window could be months rather than just days. In addition, the type of feedback that clients want is not just “how many bugs do we have?”, but also “is my game relevant?” through to “do you think our UI works for multiple markets?”. It’s a very different environment to operate in now.
QA teams have always been frontline combatants when it comes to crunch. How far along are we from the c-word falling out of use?
Crunch was something that became the norm rather than the exception and I worked through many evening of drinking sugary soda and eating pizza to get the game done. We have seen a reduction in teams pushing us to crunch for them. Companies are now looking at extended crunch periods as something to be avoided if possible and I tend to agree with that approach. When you see ads for new studios that specifically say “we don’t do crunch” that is great to see that aspiration. Whether that truly happens is another question, but we will always be flexible with our clients and work around their schedules. We try to assist our clients in their planning so that crunch never becomes a requirement. Stuff happens during development; things don’t always go to plan so we will always be there to assist where we can. If crunch is avoided then we’re happy too, it’s more about the people to us.
Unionisation has recently become a hot topic? How has that come about and do you foresee increased levels of unionisation across the industry – in QA especially – as inevitable?
As a company we are pro-people and invest heavily in the hearts and minds of our staff. Our company like many others operate in multiple countries and each have different employment laws that offer varying degrees of protection to their staff. In some territories we find that unions are not necessary as local laws offer excellent protection to staff, but in others if unions are needed to protect staff rights, then we would support that. It can be a more complex question than many might initially realise, a scenario of ‘one out, all out’ is something I think may struggle in the environment in which development and publishing resides. The unions supporting QA staff are still learning how they would fit into the modern games industry, so it may be a little early to know for sure how the situation will pan out.
What is your favourite commercially-available tool that you discovered or made use of this year?
Because of the relationship we have with our tools providers we cannot get into detail about which ones we use however, that said, we are seeing an increased interest from our clients for visibility and transparency of our processes. We are actively looking at giving our customers ways to see, in real time, the status of their projects via active process management tools with external and internal tracking. This would be an end-to-end process that involves our clients from as early as possible and gives them the ability to see and interact with their project. We want to remove the concept of a passive process to a fully immersive pipeline with touch points throughout. Some tools will allow us to achieve this, and we are actively working with those companies to understand how we can be involved.
How embedded are AI/automated tools in today’s QA environment and how have they made things better, or perhaps worse?
AI tools are becoming more and more important within QA however I believe we are still at a very early stage of understanding how AI will help us with our pipeline. There are some thoughts that AI bots will take over huge chunks of the testing pipeline in the future and to some extent I can see that happening, however I believe that is a little way off and would be instigated and implemented by the developers at a low level. The QA service team would then have access to the ‘bots’ within the game and can be instructed to perform specific tasks for a duration or numeration depending on the team’s need. We would always support the clients with new technologies where it is possible for us to be involved, there will always be limitations. Of course, engine tech companies such as Unreal and Unity may well be already going down this road, people much clever than me will know what is possible but we will always be here to support the clients in their preferred choices and be agile in responding.
What other challenges do you see on the horizon for QA and how can teams and individuals best be prepared to meet them?
The shift to automated testing has a very strong place in our industry and this movement is something we are seeing more as developers start to understand what can be done in this way. We already work with some clients on their plans, however there is a huge gap between knowing that automation is needed and actively supporting it in the games they are producing. It’s almost becoming a bit ‘buzz wordy’, and teams know they need it, they just don’t know how. Also, automation is not a ‘one size fits all’ approach, it works for some areas of a game but not for others and the personal/human element would always be needed. Being able to offload tedious tasks like bounds checking to a bot has got to be a better use of the technology rather than getting a tester to sit and check them all.
Are you optimistic about the future of QA? Why?
It would be weird for me to say ‘No’ wouldn’t it? Well, I am optimistic and for the following reasons:
Quality will always be king. I play games some now and wonder how did this ever pass format checks etc? The concept of ‘buyer beware’ quality control is a weak argument and I feel that at some point the consumer will start to push back. Releasing effectively pre-beta games on consoles and other platforms is, I hope, a short-term trend. QA teams will always try to push back against releases that just are not ready.
Our services are always changing. We consistently roll in new ways of doing things and offer those to our clients thus making us relevant. Our communication with the client is key to deciding what is needed next and making sure we can be agile enough to work around those needs. Internal QA teams are great. However, we offer new eyes on each title and can give a fresh view where the team are living it every day.
We can de-risk the QA process for the client and integrate our teams into theirs as seamlessly as possible. It’s a huge win-win for them. In addition, an external service provider has the distinct advantage to learn by being involved in multiple projects from differing partners. Why stop at QA? QA service providers do not just test games, we all provide a range of services that allow clients to address certain questions in their development pipeline. This point alone ensures that QA and the satellite services offered will always be held in high regard by the clients.
