I don't imagine stepping through the ODE with finite differences at the same time step as drawing should be very expensive, essentially it's a handful of sums and products. In fact, using a closed form solution with an exponential is probably more expensive.
A middleground is to use fixed timesteps with the closed form solution. Then the integration factors can be statically precomputed as constants, and every frame becomes just a 2x2 matrix multiplication.
And even if you are to calculate the integration factors based on dynamic frame rates, they only need to be computed just once per frame for all objects that share the same damping configuration.
Ah, interesting. Serves me right for slacking in engineering mathematics classes. I think the method you're describing is one of those numerical methods approximations that I'm supposed to know about.
You're right, it's not that expensive! The main limiting factor for me is that not all animations play monotonically forward in time. I'm also interested in using these in animation software where you have a timeline and can seek anywhere. So a closed form is helpful in handling that case too.
