Testing in Sessions
The first thing which you will realized in your effort to reinvent exploratory test management was that testers do a lot of things during the day that aren’t testing. If you wanted to track testing, you needed a way to distinguish testing from everything else. Thus, “sessions” were born.
In practice of exploratory testing, a session, not a test case or bug report, is the basic testing work unit. What you call a session is an uninterrupted block of reviewable, chartered test effort. By “chartered,” we mean that each session is associated with a mission—what you are testing or what problems you are looking for. By “uninterrupted,” we mean no significant interruptions, no email, meetings, chatting or telephone calls. By “reviewable,” mean a report, called a session sheet, is produced that can be examined by a third-party, such as the test manager, that provides information about what happened.
In my team, sessions last 90 minutes, give or take. We don’t time them very strictly, because we don’t want to be more obsessed with time than with good testing. If a session lasts closer to 45 minutes, we call it a short session. If it lasts closer to two hours, we call it a long session. Because of meetings, email, and other important non-testing activities, we expect each tester to complete no more than three sessions on a typical day. What specifically happens in each session depends on the tester and the charter of that session. For example, the tester may be directed to analyze a function, or to look for a particular problem, or to verify a set of bug fixes.
Each session is debriefed. For new testers, the debriefing occurs as soon as possible after the session. As testers gain experience and credibility in the process, these meetings take less time, and we might cover several sessions at once. As test lead, my primary objective in the debriefing is to understand and accept the session report. Another objective is to provide feedback and coaching to the tester. We find that the brief, focused nature of sessions makes the debriefing process more tractable than before we used sessions, when we were trying to cover several days of work at once.
By developing a visceral understanding through the debriefings of how much can be done in a test session, and by tracking how many sessions are actually done over a period of time, we gain the ability to estimate the amount of work involved in a test cycle and predict how long testing will take even though we have not planned the work in detail.
If there’s a magic ingredient in our approach to session-based test management, it’s the session sheet format: each report is provided in a tagged text format and stored in a repository with all the other reports. We then scan them with a tool we wrote that breaks them down into their basic elements, normalizes them, and summarizes them into tables and metrics. Using those metrics, we can track the progress of testing closely, and make instant reports to management, without having to call a team meeting. In fact, by putting these session sheets, tables and metrics online, our clients in the project have instant access to the information they crave. For instance, the chart in figure 1 shows that the testers are spending only about a third of their time actually testing. That corresponds to two sessions per day, on average, rather than three sessions. Since the chart represents two months of work, it suggests that there is some sort of ongoing obstacle that is preventing the testers from working at full capacity.


<< back                                                                                                                                                    next >>