Developing and Validating a Set of Usability and Security Metrics for ATM Interfaces

In previous works, the authors evidenced the lack of specific metrics for UX improvement in the ATM domain, and the importance of the Usability and Security of ATM interfaces, principally because it is a channel where people interact with cash. According to this, covering the lack of specific metrics for the ATM domain is very important. For that, we proposed developing and validating a set of Usability and Security Metrics for the ATM domain. To do this, first, the Security and Usability metrics that were used by other banking domains were taken as input. Second, we reviewed the metrics that the ISO 25000 standard provided. With this input, we adapted the metrics to the specific characteristics of the domain. The result was a proposal of metrics for ATMs; divided metrics focused on quantifying aspects of Usability and metrics focused on measuring aspects of security in the interfaces. It was subjected to expert judgment to establish whether the proposed metrics had what was necessary to be validated later. We consulted experts in ATM interface design and domain experts from various banks in Peru. We adapted the proposed metrics to a survey format and asked each of the experts to place a score on four aspects. Finally, we prepared and conducted User Tests containing tasks related to the withdrawal operation for the validation scenario, and we prototyped the ATM interfaces of the four most iconic banks in Peru. A SUS questionnaire followed the User Tests. The results finally obtained were compared with the results obtained from SUS in order to validate if they both gave the same trend as output. We could conclude that the Metrics proposed were validated by expert judgment and by the validation scenario previously described.