An AI running on a computer is bound by the rules and limitations of computing. Biological brain is not a computer, it has been wired differently and hence is not bound by the limitations of a computing device. Forget about Human, an AI running on a computer cannot even match the brain of a cockroach.
It is almost unlikely that the computers as we know them will ever become conscious because of their inherent limitations. This is another matter that computers can do faster and accurate calculations compared to a Human brain, but then the biological brain isn't designed for this purpose.
To create something which is capable of matching Human Intelligence, we will first need to create something which may mimic a Human Brain.
You reminded me of this:
https://en.wikipedia.org/wiki/Theory_of_mind
https://en.wikipedia.org/wiki/Theory_of_mind_in_animals
Not only the AI but also many animals lack the "mind". Many AI experts have postulated that an AI created by the Human will not be able to grow a mind of it's own.