Skip to main content

Good-bye manual tester, hello crowdsourcing!


In the 90s when I got my first job as a software engineer I remember that being a software tester
required very little.  Some people got jobs testing software with only two qualifications: you knew how to use a computer and you liked breaking stuff.  Testers would sit in labs and invest long hours into pressing buttons and looking for bugs.  The quality assurance discipline has changed a lot in the past 20 years.  It’s no wonder that currently companies often times are looking for SDET (Software Design Engineer in Test) or a QA Automation Engineer.

As a QA Engineer today you are expected to know automation tools like Watir, Selenium or Cucumber.  You are expected to know how to write up a test plan, prioritize test cases, schedule test passes, coordinate with the deployment team and many other things.  Less and less time is left over to the good old fashioned bug bashing or ad-hoc testing.  Today you can bet your career on the fact that manual regression passes will not last for a long time.  Sure, you can probably do it when you are working on the 1.0 version of your product, but as soon as you start releasing consequent versions you will never get $100K engineers allocated to do the same thing over and over and over again.

However there is still value in ad-hoc testing and you can’t leave bugs undetected because you were too busy automating primary scenarios.  One of the ways to solve this is crowdsourcing.  We have been working with Appirio’s CloudSpokes (now TopCoder) for some time and I can tell you that crowdsourcing bug bashes seems like the way of the future.  It’s an equivalent of using a Mechanical Turk to find bugs for you.  We create a challenge and have folks compete in finding more and better bugs.   We usually get half a dozen testers from across the world banging on early builds of new applications.  The total cost is about $2,000 per bug bash and it yields bugs at an average cost of $50-$100 a bug.  Some portion of the bugs are minor, not important or aren’t actual defects and a QA Engineer with domain expertise still needs to sort through the findings. Regardless of that, the ROI is pretty clear.  One of the best things about these crowdsourced bug bashes is that you get a fresh perspective on your software.  The folks who sign up to do these competitions have a “beginners mind” that the team has lost due to being around the product every day for months.

For people who are looking to get into Software Testing this could be an easy way to get their foot in the door.  You can be a part of the crowdsourcing bug bashes for some time, learn the tools and then before you know it you are a QA Engineer with some experience.

Comments

Popular posts from this blog

SDET / QA Engineer Interview Checklist

After interviewing and hiring hundreds of engineers over the past 12+  years I have come up with a few checklists.  I wanted to share one of those with you so you could conduct comprehensive interviews of QA Engineers for your team. I use this checklist when I review incoming resumes and during the interview.  It keeps me from missing areas that ensure a good team and technology fit.  I hope you make good use of them.  If you think there are good questions or topics that I have missed - get in touch with me! SDE/T or QA Engineer interview checklist from Mike Borozdin If you like this checklist you might want to check out these posts: Emotional Intelligence in Software Teams   and  Good-bye manual tester, hello crowdsourcing!

Code versus Configuration

At Ethos we are building a distributed mortgage origination system and in mortgage there is a lot of different user types with processes that vary depending on geography.  One of our ongoing discussions is about how much of the logic resides in code vs. being in a workflow system or configuration.  After researching this topic for a bit, I have arrived at a conclusion that the logic should live outside of code very infrequently, which might come as a surprise to a lot of enterprise software engineers. Costs of configuration files and workflow engines First thing that I assume is true is that having any logic outside of the code has costs associated with it.  Debugging highly configurable system involves not only getting the appropriate branch from source control, you also need to make sure that the right configuration values or the database.  In most cases this is harder for programmers to deal with.  In many FinTech companies where the production data is not made readily acce

Intuitive Programming - Comments

Comments are a topic of vibrant discussion.  Ever since programmers could leave some text in the program that was ignored by the machine the debate started: “what’s a good comment, what’s a bad comment, why comment?” There are endless instructions to programmers that say many of the following things: 1) Describe your function! 2) Don’t write in the comment what you wrote in the code. 3) Tell people why you are doing what you are doing. What I think has been missing from this discourse is the audience for comments and through those audiences there is intent.  The code is being read, skimmed or analyzed by people and tools.  So what are the audiences and reading modes? 1) Maintaining and enhancing the code 2) Skimming through the entire module or file to figure out what the overall structure is 3) Reviewing the test files to check out the test coverage and edge cases 4) Seeing the docstrings of functions while being in a separate file altogether 5) Reviewi