From my perspective, I actually didn't see any problem. Instead, a lot of opportunities which requires a good investment. Investment in terms of skills. Currently, your skillset is good enough to sell it to big clients. But you need to polish it.
The Big Billion Boys
Before your polish and upgrading your skill, It's better to understand who requires this skill? Why they requires the skill? How much they paying for it? For more concrete answers to these questions, pull out a market auditing workshop along with your team members. After this workshop, you will get a clear picture of about your future clients. With the clear picture, you can take skill upgrading decisions and move forward for it. For getting the grip of the market, you can start with websites like angel list, we work remotely, etc
the current market you can see a lot of requirements for visual designers who can do the job of a webmaster also. And a lot of UI/UX designer and Front-End engineering requirements. If you combine your skill and the market requirements, it's clear that you are on track. But you need to fill some gaps.
Restructuring the thought, not the title
"Is it a good idea to switch from Web Design to UX Design?", I prefer you to restructure the sentence to "Is it a good idea to include Human-centric approach while designing for the Web?". what I mean is you don't need to switch, just start practising.
Which mountain should I climb first
User Experience is a big topic with a lot of subjects like usability, research, visual design, etc. So first you focus on learning and understanding Usability. Being a Web Designer it will easy to get into the grip.
What gear I need
For baby steps, I prefer to start with the book Don't make me think by Steve Krug. Also, read Design of everyday things by Don Norman. These two books are considered as the holy books.You can find more books on a big book list thread in UX mastery, But I prefer to start with these two books and rest you can refer once you get the grip.
Just reading books and articles about user experience won't help you
becoming a master. For that, you need to think, engaging, observing and implement, iteratively.
While studying and practising new things don't forget to showcase the knowledge that you explored and acquired through different channels. You will get a lot of constructive feedbacks, which will help you in gaining confidence, exposure and the drive to the next level.
These are just my thoughts.
Validate it more and you can do it.
All the best for the team.
Have a Nice Day!