Today's AI does exhibit some intelligence, period. It is absurd to claim that an intelligent-looking entity doesn't have intelligence, only because we might not be able to determine which part of the entity has one. The superintelligence is an entirely different problem though because there is no clear path from intelligence to so-called superintelligence, everything has been just a speculation so far.
> It is absurd to claim that an intelligent-looking entity doesn't have intelligence
Is it? I am pretty sure biology will solve good old "are viruses alive?" sooner than we agree on definition of intelligence. "Chinese Room" is at least 40 years old.
And so do tons of counterarguments against the Chinese Room argument.
Practically speaking, the inherentness of intelligence doesn't really matter because both intelligent-looking entity and provably intelligent entity are capable for societal disruptions anyway. I partly dislike the Chinese Room argument for this reason; it facilitates useless discussions in most cases.
In that case there was still some intelligence. It turns out that a composite entity of Hans and its trainer was intelligent, and people (including the trainer) unknowingly regarded that as Hans' own intelligence.
