I just received a "blog kit" about a new QA company called uTest, and I'm looking for information from people in the community who have dealt with them.
From first glance, it appears to have a bit of a unique set of strengths and weaknesses compared to traditional QA firms.
On the upside, they're trying to take advantage of the "many eyes" philosophy of testing and they seem to be trying to match testers to products within their areas of expertise. It does seem to be a nice, low-impact means of getting your foot in the door of software QA.
On the downside, they're a "pay-per-bug" place, which leads one to ask who determines whether or not it's a bug, do you get paid for duplicate bugs, and are all bugs created equal in the eyes of the payor? (In other words, am I going to get the same for finding a crash bug as I am for a misplaced comma?)
For the unknowns, how do they handle build security/watermarking for firms, can we remove testers from our products that don't "get" QA, how do they handle firms that reject bugs to prevent payment while fixing them anyway, etc.?
A lot of questions, and hopefully some good answers.
1 comment:
Good questions indeed.
Allow me to clarify on behalf of uTest:
Bug Duplication issue - Testers do not get paid for duplicate bugs however the uTest platform has a built in algorithm that highlights potential bug duplicates in real time while testers are starting to fill in the fields associated with a bug report. This algorithm minimizes testers frustration. In addition Testers may add information pertaining to an existing bug (a bug stub) which if valuable will be considered for compensation.
Not all bugs are created equal, bug price is determined by severity, complexity, the release I am testing and my own testers ranking among other parameters. For example, the higher my testers ranking the more I get for a bug.
In regards to the other questions : You can remove testers from products testing cycle, Testers can also dispute bugs that don’t get approved and that leaves it to uTests discretion to approve or not. Firms that consistently reject bugs for no good reasons (also based on statistical model) May not be allowed to use the uTest services in the future.
And in regards to security – It is understood that this is harder to control, if your application cannot be shared with the outside world (not even in the last cycles of QA prior to market release) then uTest may not be right for you. We are building some security mechanisms in the platform but more on that later.
Roy Solomon
VP Marketing
uTest.com
Post a Comment