I don't see the problem with using interior mutability (Rc<RefCell<T>> or Arc<Mutex<T>>)
With Slint [1], we just embrace it, and rely on interior mutability for the shared state, and that works well.
[1] https://github.com/slint-ui/slint