Many organizations today are moving towards automated testing for the right reasons, but unfortunately the vast majority experience a spectacular failure. Often at Meetups or talking to potential customers who have failed at one or more automation implementations, I continue to hear the puzzlement of why they failed. I often hear what great diligence they put into the planning and the talented people they hired to build the automation, so the questions begs why are they all failing?
After working with a number of clients to not only install an automated testing framework, but also implementing their agile process, I have consistently observed what I term a “killer dozen” automation implementation killers. Over the next few weeks I will discuss each of these issues in more detail.
1. Unrealistic Planning and Expectations
3. Standalone Teams
4. Starting Automation in the Wrong Place
5. Implementing in a Silo
6. Failure to Deal with Culture Issues
7. Choosing the Wrong Tool Set
8. Lack of Transparency
9. Not Truly Integrating with your Agile Practice
10. Failure to Take a Leap of Faith
11. Sorry, QA does not own Automation
Unrealistic Planning and Expectation – It never fails that shortly after meeting with a client who is planning to implement automated testing, the planning team rolls out the vision of having “x” amount of tests automated by a certain date and “x” amount by another date. My very first question is always what baseline are you using to determine how many tests cases can be written by these dates? The answer is always the same, they either have no baseline or just took a best guess. The problem is that they are now setup for failure, once they don’t hit the planned number of test cases the organization will start to lose faith in the automation. Determining how many test cases can be written should use the same practice as scrum teams use for planning, after an initial ramp up you can start to measure velocity and also start pointing test cases in order to start adding timelines for test case completion.
Training – A simple truth exists in the software industry today that most organizations fail to fully understand, their traditional QA staff does not possess the required skill set to work with today’s agile automation test tools and they are not going to find individuals with these skill sets through their traditional recruiting process. I often find myself called in by clients to participate in interviews with potential automation candidates, but the interviews always end the same way. After one or two questions I am able to determine that the candidates supposed expertise in automated testing is limited to record and play automation. Organizations at some point must come to the realization that true automated testing requires at a minimum a set of basic skills which include programming knowledge, understanding object oriented concepts, development tools and unit testing concepts.