A Comparison Between Performing a Heuristic Evaluation Based on a Formal Process Using a System and the Traditional Way: A Case Study

The heuristic evaluation belongs to the usability inspection methods and is considered one of the most popular methods since it allows the discovery of over 75% of the total usability problems involving only 3 to 5 usability experts. However, certain problems and challenges have been identified during their execution. One of the identified problems is the lack of validation scenarios that demonstrate the contributions and benefits of incorporating systems that provide support for heuristic evaluation. Although there are case studies that could support the problems described above, the literature has shown that these are quite scarce and, for the most part, are software products that are used as support tools for certain elements to be evaluated, such as, for example, visual clarity of the system, page links validations and readability. For this reason, since there are no validation scenarios that demonstrate the contributions and benefits of the automation of heuristic evaluations through software products, the result is that there is a low level of confidence in usability evaluators to use a system that supports the heuristic evaluation process, for so the inspection continues to be carried out manually. As a part of previous research, a software product was implemented to support the five steps (planning, training, evaluation, discussion, and report) of a previously selected formal process to perform heuristic evaluations. To test the previously developed software product, a case study that demonstrates the contributions and benefits of incorporating the web application developed as a support tool for heuristic evaluation was proposed, and it has been of the utmost importance since it has allowed performing a comparative analysis between the evaluation when is performed by using the selected formal process through a template contained in an MS Excel format document, and by using the web application developed as a result of the aforementioned previous research. The results demonstrated that a higher average result was obtained, for the team that used the implemented system, in all the established criteria. The interpretation of the results can be summarized with the fact that the system obtained a better perception by the team who used it than the results obtained by the team that used the evaluation template.