I'm going to add to this with advice for any teacher running into this situation.
Ask to borrow the kid's computer for a second, and use the AI. Pick a word, then pick a letter that is not in that word. Ask chatGPT how many times that letter appears in said word. (Avoid "how many Ns in Mayonnaise" because that went viral and got trained out.) Hell, give ChatGPT multiple tries. Ask it to demonstrate each time that letter appears in a word.
Let the entire class witness chatGPT fail. Because it cannot count. It cannot spell. It cannot think. Please put your lesson plans aside for a class and use it as a learning opportunity.
To add to your arsenal for educating these kids, please look into the concept of AI hallucination. AI cannot perceive things and has no ability to think critically, which means it cannot tell what's real and what's not. Really drill into these kids that they are better off asking advice from a toddler.
I used the characterAI bot instead of chatGPT in this case, but chatGPT has the same issues, because neither bot is capable of thinking about what it's saying.
Calling these things "artificial intelligence" is a core part of the problem. They are not intelligent in any sense of the word. They are less intelligent than the spell check and grammar check functions in Microsoft Word circa 2010.
