You can do this today, though you would definitely be breaking copyright (you need to strip the DRM from the epub), and if you're cloning someone's voice without their permission you're probably breaking some more laws. You're pretty safe though assuming you don't distribute it or try to make money.
Check out https://github.com/aedocw/epub2tts for creating an audiobook from epub. Take a look in the utils directory for notes about fine-tuning a voice clone. I can tell you I've done some voices that are close enough to the original to be pretty shocking.
Feel free to get in touch if you have any questions, it's pretty fun making your own audiobooks with the reader of your choice!
Check out https://github.com/aedocw/epub2tts for creating an audiobook from epub. Take a look in the utils directory for notes about fine-tuning a voice clone. I can tell you I've done some voices that are close enough to the original to be pretty shocking.
Feel free to get in touch if you have any questions, it's pretty fun making your own audiobooks with the reader of your choice!