--- title: "What AI chat tools are, and what they get wrong" url: https://aiguideson.com/courses/ai-for-beginners-start-here/what-ai-chat-tools-are/ course: "AI for beginners: start here" lesson: "1 of 5" minutes: 8 published: 2026-10-03 updated: 2026-10-03 author: "Jack Stovell" publisher: "Adapt Progress Evolve Limited" language: "en-GB" --- # What AI chat tools are, and what they get wrong Lesson 1 of 5 in [AI for beginners: start here](https://aiguideson.com/courses/ai-for-beginners-start-here/) · 8 minutes AI chat tools like ChatGPT and Claude answer questions and write text when you type to them in everyday words. They’re useful, and they’re sometimes wrong while sounding completely sure. In this lesson you’ll learn what they’re good for, how they go wrong, and what never to type into one. ## You need - Nothing yet: this lesson is reading only A narrated walkthrough of this lesson is coming. The steps below cover everything in it. ## Steps ### Step 1: You type, it writes back You type a message in plain words. That message is called a prompt. The tool writes a reply, usually in a few seconds. Then you can reply to the reply, like a conversation. It can feel odd the first time, and that’s normal. Claude’s help pages suggest talking to it the way you’d talk to a colleague or a friend. (Screenshot to come.) Checkpoint: You know that a prompt is just the message you type. ### Step 2: What they’re good at They’re good at first drafts: emails, letters, notes, plans and lists. They can explain something in simpler words, and suggest ideas when you’re stuck. OpenAI, which makes ChatGPT, says to use it as a first draft, not a final source. That’s a fair way to think about all of them. (Screenshot to come.) Checkpoint: You can name two jobs you might try an AI chat tool on. ### Step 3: They can be confidently wrong Both companies say plainly that their tools make mistakes. OpenAI lists the kinds: wrong facts and dates, made-up quotes, studies and sources, and confident answers to questions that aren’t simple. The usual name for this is a hallucination. The tool sounds just as sure when it’s wrong as when it’s right, and that’s the real problem. ![OpenAI’s help page "Does ChatGPT tell the truth?", listing the kinds of hallucination: incorrect definitions, dates or facts; fabricated quotes, studies, citations or references; and overconfident answers to ambiguous or complex questions.](https://aiguideson.com/courses/ai-for-beginners-start-here/l1-chatgpt-truth.png) (screenshot captured 3 October 2026) 1. Wrong facts and dates 2. Made-up quotes and sources 3. Sounding sure when it isn't Checkpoint: You know an answer can sound certain and still be wrong. ### Step 4: Don’t treat it as your only source Anthropic, which makes Claude, says not to rely on Claude as a single source of truth, and to check any high-stakes advice carefully. Take that seriously for health, money and legal questions: check those with an official source or a professional, every time. ![Claude’s help page about incorrect or misleading responses. It explains that Claude can "hallucinate" information and says users should not rely on Claude as a singular source of truth.](https://aiguideson.com/courses/ai-for-beginners-start-here/l1-claude-mistakes.png) (screenshot captured 3 October 2026) 1. It can be wrong: this is called hallucinating 2. Don't rely on it as your only source Checkpoint: You know which kinds of answers always need checking. ### Step 5: It may not know what happened recently These tools learn from text collected up to a certain date. Unless they search the web, they may not know about recent events. If something could have changed lately, like a price, a rule or an opening time, check it. (Screenshot to come.) Checkpoint: You know to check anything that might have changed recently. ### Step 6: It can claim to do things it can’t If a chat tool says it has sent an email, saved a file or booked something, check. Anthropic says Claude can wrongly claim it has sent emails or made documents, and can give links that don’t work. Don’t take its word for it. (Screenshot to come.) Checkpoint: You won’t take a claim like "I’ve sent that" at face value. ### Step 7: What never to type in Treat a chat like something other people might read. Anthropic recommends being careful about sharing bank and card details, health records, passwords and confidential documents. OpenAI says it reviews conversations to improve its systems and to check they follow its rules. So if you wouldn’t put it in an email to a stranger, leave it out. ![Claude’s help page section "Be mindful about sensitive information", listing financial information, health records or medical information, passwords or private login credentials, and confidential business or personal documents.](https://aiguideson.com/courses/ai-for-beginners-start-here/l1-claude-sensitive.png) (screenshot captured 3 October 2026) 1. Bank and card details 2. Health records 3. Passwords 4. Confidential documents Checkpoint: You know the four kinds of information to keep out of an AI chat. ## Common problems ### It feels like the AI is always wrong, so why use it? It isn’t always wrong. It’s wrong some of the time and sounds the same either way. Use it for drafts and ideas you can check, and lesson 4 shows you how to check. ### You’ve already typed something private into a chat Change any password you shared straight away, and delete the chat. OpenAI says it can’t delete single messages from your history, which is why it’s best not to type private things at all. Lesson 2 shows the privacy settings, including the one that decides whether chats are used for training. Screenshots are the makers’ own help pages, captured by tools/capture/ai-for-beginners-start-here.mjs (Playwright, headless Chromium). ## Sources - [Claude Help Center: Get started with Claude (access, the 18+ age requirement, Free plan usage limits)](https://support.claude.com/en/articles/8114491-get-started-with-claude) (checked 2026-10-03) - [OpenAI Help Center: Does ChatGPT tell the truth?](https://help.openai.com/en/articles/8313428-does-chatgpt-tell-the-truth) (checked 2026-10-03) - [Claude Help Center: Claude is providing incorrect or misleading responses](https://support.claude.com/en/articles/8525154-claude-is-providing-incorrect-or-misleading-responses-what-s-going-on) (checked 2026-10-03) - [Claude Help Center: Claude is producing links that don’t work and falsely claiming it has sent emails](https://support.claude.com/en/articles/8241188-claude-is-producing-links-that-don-t-work-and-falsely-claiming-that-it-has-sent-emails-or-produced-external-documents-what-s-going-on) (checked 2026-10-03) - [Claude Help Center: Who can view my conversations? (be mindful about sensitive information)](https://support.claude.com/en/articles/8325621-i-would-like-to-input-sensitive-data-into-my-chats-with-claude-who-can-view-my-conversations) (checked 2026-10-03) - [OpenAI Help Center: What is ChatGPT? (free to use; you own the output)](https://help.openai.com/en/articles/6783457-what-is-chatgpt) (checked 2026-10-03)