What AI chat tools are, and what they get wrong

You type a message in plain words. That message is called a prompt. The tool writes a reply, usually in a few seconds. Then you can reply to the reply, like a conversation. It can feel odd the first time, and that’s normal. Claude’s help pages suggest talking to it the way you’d talk to a colleague or a friend.

CheckpointYou know that a prompt is just the message you type.

They’re good at first drafts: emails, letters, notes, plans and lists. They can explain something in simpler words, and suggest ideas when you’re stuck. OpenAI, which makes ChatGPT, says to use it as a first draft, not a final source. That’s a fair way to think about all of them.

CheckpointYou can name two jobs you might try an AI chat tool on.

Both companies say plainly that their tools make mistakes. OpenAI lists the kinds: wrong facts and dates, made-up quotes, studies and sources, and confident answers to questions that aren’t simple. The usual name for this is a hallucination. The tool sounds just as sure when it’s wrong as when it’s right, and that’s the real problem.

123View full size

Step 3: They can be confidently wrong

CheckpointYou know an answer can sound certain and still be wrong.

Anthropic, which makes Claude, says not to rely on Claude as a single source of truth, and to check any high-stakes advice carefully. Take that seriously for health, money and legal questions: check those with an official source or a professional, every time.

12View full size

Step 4: Don’t treat it as your only source

CheckpointYou know which kinds of answers always need checking.

These tools learn from text collected up to a certain date. Unless they search the web, they may not know about recent events. If something could have changed lately, like a price, a rule or an opening time, check it.

CheckpointYou know to check anything that might have changed recently.

If a chat tool says it has sent an email, saved a file or booked something, check. Anthropic says Claude can wrongly claim it has sent emails or made documents, and can give links that don’t work. Don’t take its word for it.

CheckpointYou won’t take a claim like “I’ve sent that” at face value.

Treat a chat like something other people might read. Anthropic recommends being careful about sharing bank and card details, health records, passwords and confidential documents. OpenAI says it reviews conversations to improve its systems and to check they follow its rules. So if you wouldn’t put it in an email to a stranger, leave it out.

1234View full size

Step 7: What never to type in

CheckpointYou know the four kinds of information to keep out of an AI chat.