What are the limitations of usability testing?

Many users perform tasks while performing several unrelated tasks on the same computer at the same time. This may have a strong influence on how they interract with a given product, but it is difficult to simulate.

Similarly, users may normally interact with a product at particular time of the day when they are subject to certain distractions or a certain state-of-mind. These may also have a significant effect on the task but are difficult to simulate.

Usability testing is often confused with market research, but the two are very different.

Usability problems can easily be detected from a sample of about five test participants. Very different kinds of people will tend to encounter the same usability problems. More than seven participants is just a waste of time.

Market research issues, things like design-preferences and whether they would actually want to use the product, are completely different. These vary wildly from person-to-person and different personalities/backgrounds. You need big sample sizes to judge them accurately.

For example, a participant will be much more careful about a purchase they are genuinely making than about a purchase they are pretending to make for the sake of a usability test.

Usability testing typically last 45 minutes, but many web interactions take place over several days, especially when they involve interacting with an online community

Yet this is what many social media apps are all about