IMO, this is a definite YES.
Getting feedback at the wireframe stage is key to avoid going down the wrong road, but all of the visual stuff you add in later will also fundamentally shape the user experience in a lot of ways: visual hierarchy, perception of clutter, emotional response, etc. So you should be doing some testing with higher-fidelity prototypes as well.
Difficult part of course is finding time and resources to test in all of the relevant points in the design process, and building a tool stack that will make it possible.
Jakob Nielsen talks about this a lot - here’s a primer on his “discount usability” concept that he came up with way back in 1989 (I hope it’s ok to post links in here):
I tried my hand at building a tool stack to function as a modern version of this, which you can find here:
Hope it helps!