Arbitrarily precise is not the same as exact. There are many possible examples to illustrate why this matters. Here's one: let's say that you compute floor(f(x)) where f(x) falls right on an integer n. If you evaluate f(x) in floating-point arithmetic, you probably get n +/- epsilon and not n exactly because of rounding errors. This makes the floor function go to either n or n - 1, so you sometimes end up with an error of 1 no matter how precise the floating-point numbers.
Right, and with a little extra effort, you can make an arbitrarily small error in one of your terms give a result that can be anywhere from negative infinity to positive infinity.
What kind of lousy CAS does not support arbitrarily precise numbers? For example Fricas/Axiom/Scratchpad gives arbitrarily precise results.