Judges are elected or appointed, they can't be directly fired, and it would take the federal government to call for impeachment. This is a non-trivial matter, and the fallout would require a special appointment, or special election.

To institute a policy that increases fairness of the whole system instead of targeting a specific individual makes sense. After all there is nothing to say that someone else (or several someones) wouldn't have the same issue.

This solves the problem, without having to literally make it an act of congress.