It is common for usability professionals to conduct tests with participants in 60 to 90 minute sessions. In many cases, participants are seeing an interface for the first time. When the results from a usability test are negative, it is common for project team members to wonder aloud whether or not participants would “eventually” learn how to complete a task if given enough time and/or training.
Depending on the application or audience, it is one that is consumer facing or one that someone will need to use over and over again. Consumers have the opportunity to give up or move onto a competitors application, website, etc. to complete a transaction (e.g. buy X). In this case, they are not seeking to learn how to use something. Instead, they are looking to achieve something as quickly and easily as possible. On the other hand, employees at a company might have to use an application over and over again in the course of their job. In many cases, they can “overcome” a bad interface and develop work arounds. They must learn to use an interface in order to achieve what their employer has asked them to do.
In the case of an application for employees at a company, it would be best to set up a usability test where we have different cells of participants: a) those who have never seen the application before b) those who have been exposed to it for one day, c) those who have been exposed to it for one week, d) those who have completed training. In this way, we can measure the impact of learning by doing and formal training.