Just wondering if it’s common and practical to show our participants another version of our design to get their thoughts? My concern is validating version A without observing version B wouldn’t be a useful approach.
That would be preference testing. A/B testing can only be done on a live site with 1000s of visitors.
When preference testing you show on participant version A and then version B. The next participant you show version B first and then version A to reduce bias.
This is my favourite article on this topic:
Repeat after me: Preference testing is not A/B Testing
Researchers sometimes ask participants which of two alternative designs they prefer. The data from these studies comprise opinions that have little predictive value. In contrast, multivariate A/B testing involves target users doing real tasks. The data from these studies comprise behavioural observations that predict real-world behaviour.
Welcome to the forum @Cam_Rogers!
Yes, definitely not the same. And there are specific ways to run preference testing to avoid bias.