TY - JOUR

T1 - Comparing distributed and face-to-face meetings for software architecture evaluation: A controlled experiment

AU - Babar, Muhammad Ali

AU - Kitchenham, Barbara A.

AU - Jeffery, D. Ross

PY - 2008/2

Y1 - 2008/2

N2 - Scenario-based methods for evaluating software architecture require a large number of stakeholders to be collocated for evaluation meetings. Collocating stakeholders is often an expensive exercise. To reduce expense, we have proposed a framework for supporting software architecture evaluation process using groupware systems. This paper presents a controlled experiment that we conducted to assess the effectiveness of one of the key activities, developing scenario profiles, of the proposed groupware-supported process of evaluating software architecture. We used a cross-over experiment involving 32 teams of three 3rd and 4th year undergraduate students. We found that the quality of scenario profiles developed by distributed teams using a groupware tool were significantly better than the quality of scenario profiles developed by face-to-face teams (p < 0.001). However, questionnaires indicated that most participants preferred the face-to-face arrangement (82%) and 60% thought the distributed meetings were less efficient. We conclude that distributed meetings for developing scenario profiles are extremely effective but that tool support must be of a high standard or participants will not find distributed meetings acceptable.

AB - Scenario-based methods for evaluating software architecture require a large number of stakeholders to be collocated for evaluation meetings. Collocating stakeholders is often an expensive exercise. To reduce expense, we have proposed a framework for supporting software architecture evaluation process using groupware systems. This paper presents a controlled experiment that we conducted to assess the effectiveness of one of the key activities, developing scenario profiles, of the proposed groupware-supported process of evaluating software architecture. We used a cross-over experiment involving 32 teams of three 3rd and 4th year undergraduate students. We found that the quality of scenario profiles developed by distributed teams using a groupware tool were significantly better than the quality of scenario profiles developed by face-to-face teams (p < 0.001). However, questionnaires indicated that most participants preferred the face-to-face arrangement (82%) and 60% thought the distributed meetings were less efficient. We conclude that distributed meetings for developing scenario profiles are extremely effective but that tool support must be of a high standard or participants will not find distributed meetings acceptable.

KW - Architecture evaluation

KW - Process improvement

KW - Controlled experiments

KW - Groupware support

KW - Scenario development

UR - http://www.scopus.com/inward/record.url?scp=37649012568&partnerID=8YFLogxK

U2 - 10.1007/s10664-007-9052-6

DO - 10.1007/s10664-007-9052-6

M3 - Journal article

VL - 13

SP - 39

EP - 62

JO - Empirical Software Engineering

JF - Empirical Software Engineering

SN - 1382-3256

IS - 1

ER -