Skip to main content

Good-bye manual tester, hello crowdsourcing!


In the 90s when I got my first job as a software engineer I remember that being a software tester
required very little.  Some people got jobs testing software with only two qualifications: you knew how to use a computer and you liked breaking stuff.  Testers would sit in labs and invest long hours into pressing buttons and looking for bugs.  The quality assurance discipline has changed a lot in the past 20 years.  It’s no wonder that currently companies often times are looking for SDET (Software Design Engineer in Test) or a QA Automation Engineer.

As a QA Engineer today you are expected to know automation tools like Watir, Selenium or Cucumber.  You are expected to know how to write up a test plan, prioritize test cases, schedule test passes, coordinate with the deployment team and many other things.  Less and less time is left over to the good old fashioned bug bashing or ad-hoc testing.  Today you can bet your career on the fact that manual regression passes will not last for a long time.  Sure, you can probably do it when you are working on the 1.0 version of your product, but as soon as you start releasing consequent versions you will never get $100K engineers allocated to do the same thing over and over and over again.

However there is still value in ad-hoc testing and you can’t leave bugs undetected because you were too busy automating primary scenarios.  One of the ways to solve this is crowdsourcing.  We have been working with Appirio’s CloudSpokes (now TopCoder) for some time and I can tell you that crowdsourcing bug bashes seems like the way of the future.  It’s an equivalent of using a Mechanical Turk to find bugs for you.  We create a challenge and have folks compete in finding more and better bugs.   We usually get half a dozen testers from across the world banging on early builds of new applications.  The total cost is about $2,000 per bug bash and it yields bugs at an average cost of $50-$100 a bug.  Some portion of the bugs are minor, not important or aren’t actual defects and a QA Engineer with domain expertise still needs to sort through the findings. Regardless of that, the ROI is pretty clear.  One of the best things about these crowdsourced bug bashes is that you get a fresh perspective on your software.  The folks who sign up to do these competitions have a “beginners mind” that the team has lost due to being around the product every day for months.

For people who are looking to get into Software Testing this could be an easy way to get their foot in the door.  You can be a part of the crowdsourcing bug bashes for some time, learn the tools and then before you know it you are a QA Engineer with some experience.

Comments

Popular posts from this blog

Quality of Code is Quality of Life

About 20 years ago when I started working in technology companies I remember “the best” engineers had similar patterns:
-They worked crazy hours
-They knew the systems no one else knew
-They could react and deliver something faster than anyone else
You could always hear other employees say: “Bob is really smart, no one knows how to get anything done in system X besides him!”

This reinforced optimization around being the only person who knew how to do something in some part of the code.  That in turn reinforced job security and bargaining for those engineers, but also chained them to a particular system.  We had big code bases of C++ or Java code where some “Bob” hacked up features as soon as he possibly could.  “Bob” would have occasional nuclear disasters where he’d sleep in the office or through the weekend and then everyone would thank him for how he “saved the day.”  “Bob” sacrificed his quality of life to get praise when he hacked stuff up quickly and then the second time when n…

SDET / QA Engineer Interview Checklist

After interviewing and hiring hundreds of engineers over the past 12+  years I have come up with a few checklists.  I wanted to share one of those with you so you could conduct comprehensive interviews of QA Engineers for your team.

I use this checklist when I review incoming resumes and during the interview.  It keeps me from missing areas that ensure a good team and technology fit.  I hope you make good use of them.  If you think there are good questions or topics that I have missed - get in touch with me!


SDE/T or QA Engineer interview checklist from Mike Borozdin
If you like this checklist you might want to check out these posts:
Emotional Intelligence in Software Teams  and Good-bye manual tester, hello crowdsourcing!

Why you should take the software job in San Francisco (or not).

Silicon Valley is an iconic place for technology.  Many people say this is the place for the “best and the brightest.”. Apple, Google, Facebook, Salesforce, Twitter and other top companies draw a lot of talent form all over the world and the largest chunk of VC capital goes to companies in the Bay Area, so it seems like moving here is a no brainer!

The real situation is actually not that simple, I believe there are three scenarios where it makes sense, but in many cases living in the Bay yields disappointing results.  The cost of living, housing situation, homeless catastrophe make places like San Francisco a lot less appealing to a lot of people.  So in what situations does it make sense to move to SF?

Startup founder raising millions There are many places to be a startup founder, but if you are looking to raise capital the largest pool of VC money is in the Bay Area.  There is an established network, events and conferences which give founders an opportunity to pitch more people th…