Lightning and Thunderbird testing week, a post mortem.
Last week we had our last (to date) community testing effort and the focus was Thunderbird 10 and Lightning 1.0. This was a first for me, dealing with two products during the same week and dealing with a product I don't use too much. I was not too confident that the test cases available for Lightning were up to date - but I thought that some testing was better than none. So I did My usual call to testers using the too many ways we have to do that :
- Mail to testers
- Newsgroup post
- blog post
- mozillazine post
And I didn't do that on the same day but on the course of two weeks. And I got a bit stressed and worried because the numbers of answers I got was very low. And then Philipp posted on the Calendar blog and my mailbox got filled with people willing to spend some time testing.
The plan was to send instructions when I would get back from mozcamp in Berlin and start the testing effort. Unfortunately I came back sick and was in bed for most of the testing week. I did send instructions and then left my computer unattended for the week as I was sleeping and fighting fever. The instructions I sent did the job for most of the volunteers that had signed up for Thunderbird testing but were confusing to say the least for the many new comers who had signed up for Lightning.
I would like to apologize to the people who wanted to help but got confusing instructions f from me.
I would like to thanks standard8 for getting into #tb-qa and answering people.
And I would like to point out our results :
- Thunderbird - 81% coverage and 15 bugs found
- Lightning - 38% coverage and 4 bugs found
And I've learned a lot on something I thought I mastered so next time it won't happen the same way for sure.
Syndicated 2011-11-22 10:09:48 (Updated 2011-11-22 10:35:08) from Ludovic's weblog