Check out the new USENIX Web site. next up previous
Next: Qualitative Results Up: User Study Previous: Formal User Testing

Task Completion Time and Error Rate

It took longer for users to create image portfolios than to create passwords and PINS. Photo portfolios took longer to create than Random Art portfolios, because people spent more time browsing and looking at each image.

Users also required more time to login with image portfolios compared to passwords and PINs. It took slightly longer for users to login using Random Art compared to photos, suggesting that people can recognize photographic images more quickly than abstract images.

After one week, however, there was a greater degradation in performance with PINs and passwords compared to portfolios. Table 1 shows the average creation and login times. The reason for the longer than expected login times for passwords and PINs is that several users required multiple attempts. (Note that login times include multiple attempts, but do not include those who could not login at all).



Table 1: Average seconds to create/login

A number of minor and major errors were made with PINs, passwords and portfolios. During the first session all users were able to recover from their errors and to login successfully with portfolios, but this was not always the case with PINs and passwords, no matter how long or how many login attempts were made.

Even after one week, the number of unrecoverable errors made with images was far lower than that of passwords and PINs. If we imposed more secure password and PIN restrictions (e.g., restrictions on character length and type, limited number of attempts), we suspect that the number of failed logins with passwords and PINs would increase. In contrast, all users were able to remember at least four out of five of their portfolio images on the first attempt.

Further study is needed to discover how frequency of use and long term memory effects will influence performance and error rates in portfolio authentication.



Table 2: % Failed logins (# failed logins/20 participants)

next up previous
Next: Qualitative Results Up: User Study Previous: Formal User Testing

Adrian Perrig
Thu Jun 15 15:16:10 PDT 2000