The same goes for the AI that pretended to be blind to get the information to enter the captcha. It’s fair enough comment to point out that it didn’t really know what it was doing, it just mimicked what it calculated to be a successful move in that situation to reach its goal. But that really is a red herring.
I get your point about Chess computers, but that IS the point though - it's not really AI. There is no intelligence, no reasoning - it can't put 2 + 2 together and get 4 (for real, you can convince it that 5 is the right answer). It can't deduce or combine information unless someone else has already done that.
It's a great tool, it is a step up from spell checkers and grammar checkers when writing articles, but it can't write anything truly original.
Plus, it can be hugely wrong but supremely confident. That is where it's dangerous - when people use it for something they don't have expertise in, and trust what it tells them.