What's going on in Fedora QA

I apologize for not updating this blog for a bit - there's always one more thing that needs doing! However, just because I've been quiet doesn't mean Fedora QA has been inactive - far from it. We managed to crawl across the Fedora 16 finishing line back in November, and we all took a few days to get our breath back before jumping right into Fedora 16 damage control (I'm sure there must be a more diplomatic phrase for that) and Fedora 17 planning. Helping to write the Fedora 16 Common Bugs page has been an interesting experience, and quite educational - it's interesting how wide the gap can be between understanding an issue well enough to deal with it in terms of release validation, and understanding it well enough to document it for a notional audience that knows nothing about it, particularly in the cases of problems booting from GPT-labelled disks and issues with the size of grub2's core image on older partition layouts. The F16 common bugs page is longer than we'd ideally want it to be for any release, but we knew going in that F16 would be a challenging release with the grub2 and GPT migrations. It certainly points up some issues we'll be trying to fix as soon as possible for Fedora 17. I've also been hanging out in the Fedora forums and trying to help out with F16 issues where I can; the other forum denizens are doing a great job of getting on top of the F16 changes and passing on the news (and the workarounds...) to others, though, so kudos to them. Now Fedora 16 work is mostly out of the way and I've caught up with all the todo items that piled up while F16 validation was taking up all of our time, I recently managed to work through all the feedback that had been added to the Fedora 16 QA retrospective page on the Wiki and come up with an action plan. The retrospective is a great initiative of James Laska's - it's a single Wiki page where we can throw thoughts that grow out of the QA process on a specific release. If we notice something that's not working great, or think of a way we could improve the process, we can quickly throw it in the retrospective and get back to work, knowing it'll be developed into a proper action plan later. So I went through each of the thoughts contributed by a wide range of people in the QA team, took direct action on several of them, and turned the rest into trac tickets for the QA and release engineering teams. That gives us a nice clear view of the work we can do in the 'quiet time' before Fedora 17 Alpha lands to improve our processes ahead of that validation cycle. So what can you look forward to in terms of new and improved QA procedures (as I know that's just the highlight of your life)? Well... * Better testing of EFI installations / upgrades * Specific testing of EC2 installation / deployment * More comprehensive coverage of installation from images written to USB * More testing of different upgrade scenarios * Expanded testing of Xen virtualization * Artwork coverage in the validation testing * Completion of the work on ensuring there are test cases for all release criteria * Earlier (and more) Test Composes for each release point, after successful testing of this for Fedora 16 Beta and Final And probably a few more changes besides. Of course, the AutoQA train keeps on a-rollin'. The team recently completed deployment of AutoQA 0.7 and have already started AutoQA 0.8 planning. Some of the goals for the Fedora 17 cycle are to complete work on the ResultsDB system for managing AutoQA test results, and to have complete daily automated testing of Fedora 17 installation after branching. We've also been talking to the anaconda team to try and ensure we work together to start testing the currently in-development anaconda UI rewrite as soon as possible, in order to try and make it as robust as possible and avoid the last-minute crises during the validation process we had so much trouble with in relation to Fedora 16's major installation changes. It's going to be another exciting release cycle - but then, it always is, in Fedora land! Myself and Tim Flink will be at FUDCon Blacksburg in January, together with several members of the QA community. Sandro Mathys has been working on planning a hackfest to bring the anaconda and QA groups together to work on new anaconda UI testing, we will probably be giving a few presentations, and Tim will be available to talk about AutoQA and any other QA-related tooling stuff. See you there!