The before
Why the comparison point decides what a result means
Every result is a difference between two situations. Change the 'before' and the same AI can look remarkable or ordinary. The shopper and chat on this page are a teaching reconstruction, not a real participant, record or the chatbot's wording.
How the comparison was set up
Researchers randomly assigned 44,614 consumers to a chatbot group or an unavailable-support group in September–October 2023. This separates the tested response from customers choosing which service to use. The comparison supports the specific intervention in that setting.
What it did not compare
With no live answer as the before, the result says nothing about AI against a capable human team. A business that already answers questions well starts from a different place and may see a different, possibly smaller, change at the same moment.
Finding your own before
Application, not a study finding: the same shape shows up in a quote request left until morning, an after-hours call that rings out, or a booking form nobody reads until Monday. Write down what happens today, who handles it and what the customer does next, before changing anything.
Before judging any AI result, including your own, name the exact 'before' it was measured against and check that it matches your business today.