I’ve been in a situation, early in my career, where we ran some unmoderated tests on a prototype before any moderated testing. The tests failed spectacularly. The participant totally missed the point, went down a rabbit hole, and got distracted and preoccupied by some parts of the prototype that were features of the prototyping software and not actually related to our product.
This was a good thing. It made us realise that we had totally missed the mark with the direction of the product and had made a bunch of assumptions that weren’t true. It also highlighted that we needed to refine the scope of our test and make prototypes that were more contained and focussed—and to use better prototyping software that didn’t get in the way. It cost us less than $50 to learn this—probably cheaper than running a local moderated test. It was a cheap way to fail and learn.