ChatGPT isn't AGI, but it's certainly AI and many people who've spent years working in the field believe that we're taking on extreme risk by not building in adequate guardrails. What it's doing already is surprising everyone, including the people who built it.

We don't need to understand intelligence at all. The machines are coding themselves and we don't know what's going on in there--we're just shovelling in training data. And a system doesn't have to be self-aware to be destructive.