I think the core issue with CalDAV (and by extension, the protocol it's build on top of, WebDAV) is one somewhat endemic with most W3 approved standards: it's way too overengineered to do too many things at the same time rather than a lean format.
The result is a protocol that's essentially become a furball of different ideas mashed together, which inevitably leads to the situation where a lot of services just... don't try to be full spec compliant since it's out of scope for what they actually want to do.
I don't think I've ever seen a W3 data format spec that didn't end up this way, it just always seems to happen. (It doesn't seem to happen with their lower level protocol specifications.)
WebDAV is what it is because it's designed (largely by Julian Reschke who also took over the HTTP spec from TBL back then I believe) to be addable conflict-free and routable without complex payload inspection to existing HTTP services/practices. Its use of XML can be regarded over-engineered, but it follows what was available and considered adequate at a time that had much much more consideration for interoperability and multi-party buy-in than is even conceivable today. I wouldn't dismiss it so quickly, especially considering WebDAV is an IETF RFC not W3C spec to begin with.
The result is a protocol that's essentially become a furball of different ideas mashed together, which inevitably leads to the situation where a lot of services just... don't try to be full spec compliant since it's out of scope for what they actually want to do.
I don't think I've ever seen a W3 data format spec that didn't end up this way, it just always seems to happen. (It doesn't seem to happen with their lower level protocol specifications.)