I imagine it can now, or will soon be able to, create something that sounds like it expresses human emotion. But the big point is, I think, that it doesn't know where that might have come from, or what it might lead to, so it will lack emotional and even logical context. Like an alien blurting out 'the kind of thing humans say when they're happy/sad/in love', without knowing what situations might - typically or unexpectedly - elicit such emotions in us.

It can pick flowers, in other words, but cannot transplant them into fertile soil, and they will wither.

It also seems to have little understanding of poetic voice. One of the first tasks I set ChatGPT when messing around with it last year was to 'write song lyrics in the style of Leonard Cohen, about an old priest comforting a boy whose father has been conscripted to fight in a war'.

It came up with a piece of doggerel that made sense, but was pointlessly shallow, as expected.

I then asked it to do the same in the style of Bernie Taupin, and the Beastie Boys. It just rewrote the same style, with no sense of what imagery or stance might be appropriate for either, despite presumably having accessed all lyrics ever written by all three of them, as they are openly available online.

I should maybe repeat the experiment with the current version.