I use a script that describes the user-story in terms of what the user wants to do (for instance “I want to create a recurring payment module for a customer”). Within the script, I provide few technical info like “as a user pretend to be interrupted by a phone call and you want to save the process to start it again in a couple of hours”.
The most important UX KPIs I would check are:
- Navigation/action pattern - is it clear the naming convention?
- System notification - is the user able to understand what will happens next, for instance, after she/he saves the form?
- User interactions - how the user is guided by the system in case there’s an error?