"Scheherazade, you easily make up stories of Egypt or any country you please."

Reading Plato's Phaedrus¹ again I was struck by this line in relation to discussion of fabrication or hallucination in tools like ChatGPT.

Phaedrus Socrates, you easily make up stories of Egypt or any country you please.

Musing on that line I tweeted:

I ran the question through various generative search and search-like systems (pre-pending only "Please answer this question:") and was surprised at how they all got it "wrong":

• ChatGPT-3.5 & ChatGPT-4: Scheherazade

• Claude 2: Scheherazade

• Google's Bard: ChatGPT

• Inflection AI's Pi: Scheherazade

• llama-2-70b-chat (at Perplexity Labs): [DECLINED TO ANSWER]

• Perplexity AI: Scheherazade

  • w/ Copilot: Scheherazade
    • Note: I declined to answer Copilot's clarifying question (which was great, something like 'Which do you think it is?').
      • Aside: Note the Copilot question is not retained for review by the user.
    • Note: I generally have the AI Model (in Perplexity's Settings) set to Perplexity, rather than GPT-4 or Claude 2.

• You.com's YouChat: Scheherazade

  • w/ GPT-4: ChatGPT

Bonus: ChainForge

Here is a ChainForge flow, where I show various prepends and appends and include Google's Bard as an option: https://chainforge.ai/play/?f=iflfrrosd4j4

Search: [chainforge]

Footnotes

1. Via The Perseus Digital Library Project [@crane2001perseus] ↩