That is an astute connection. The "recursive" part is that you do not need to keep all the history of observations and actions (often denoted y and u. in control theory). You can summarize the past with sufficient statistics (the mean and the variance for linear quadratic gaussian (LQG) problems).
You could imagine keeping all of the data and fitting a model to that. For LQG problems, you can use dynamic programming [1] to solve the problem faster.
You could alternatively imagine keeping a finite window of data and fitting a model to that. That filter would have a finite impulse response.
