That’s a fascinating idea, but in practice I think we Americans have completely messed it up. Some U.S. states like Georgia centralize investigative responsibilities (often when municipalities are too small to budget these services). This strategy hasn’t lead to great results. The Georgia Bureau of Investigation is notorious for falsely indicting and imprisoning minorities. I suspect the same is probably true in other states.
We do still have the FBI and federal Dept of Justice that is supposed to provide some semblance of oversight. However, as we have seen over the last 4 years, they are equally susceptible to political influence as local law enforcement agencies.
There is an interesting fact that's seldom overlooked over here. We have comunidades autónomas, that might be similar to states, only smaller. They're relatively new, less than 50 y.o. and accumulates a disproportionate number of corruption cases. Why is that?
Central government was developed around the statute of the public officer. In the 1800s there was an unending flow of public officers in and out (the "cesantías") caused by political parties putting their people in public jobs and firing the others' people when every elections turnover. At a certain moment a system of merit access was imposed to provide stability. If you were under the line that divides technical from political, you're safe from firing. Still it's possible that you want to cooperate for promotions but, at a certain level, you just don't care.
Comunidades were developed from scratch, with a hight proportion of "external" workers (that don't have the protections of the public officer statute), with "merit points" distorting the exams (being the merit having been working for years without exams because you're friends with someone) and with very little judicial oversight.
TL;DR: to avoid local corruption, make the police come from as far above as possible. To avoid central corruption, make technical lead not political and involve judges not elected by politicians. Everything elected by politicians gets corrupted.
We do still have the FBI and federal Dept of Justice that is supposed to provide some semblance of oversight. However, as we have seen over the last 4 years, they are equally susceptible to political influence as local law enforcement agencies.