“To err is human,” the saying goes. But it seems that making factual errors and creating facts out of thin air isn’t an exclusively human characteristic. AI chatbots ChatGPT and Google Bard both have documented histories of fibbing. Now it seems Microsoft’s Bing AI has the same bad habit.

In a blog post, AI researcher Dmitri Brereton documented the numerous errors made by Microsoft’s new chatbot during its demo—mistakes nobody noticed at the time. Brereton starts out by highlighting factual inaccuracies displayed by Bing AI in making pet vacuum recommendations. The chatbot stated that Bissell Pet Hair Eraser Handheld Vacuum had limited suction power, a short cord, and was noisy enough to scare pets. However, none of that is true. The pet vacuum in question has no cord, and the cited review doesn’t say anything about alleged noisiness or limited suction power.

Additionally, when the AI chatbot generated an itinerary for a five-day getaway in Mexico, it gave totally inaccurate recommendations, according to Brereton. Night club recommendations seem to have been made out of thin air, with actual critical information missing.

But, perhaps the most concerning errors came in Bing AI’s Summary of Gap Inc’s finances. In the demo, the AI displayed Gap’s reported gross margin as 37.4%. However, that’s the unadjusted gross margin. The adjusted gross margin was 38.7%. That might not seem like a big deal for people who don’t understand financial jargon, but it’s a whopping error for those who deal with dollars and cents every day. Bing AI also misreported the company’s operating margin, diluted earnings per share, and expected net sales growth. Many of the figures yielded by the AI appear to be created from whole cloth.

Bing AI also summarized Lululemon’s finances, and according to Brereton, the results are just as bad. The AI misreported the company’s gross margin as 58.7% when the real value is 55.9%, its operating margin as 20.7% when it’s actually 19%, and diluted earnings per share were reported as $1.65 when they’re actually $2.00. Again, these might seem like nitpicks to some, but when you’re dealing with financial documentation, accuracy is beyond critical. And again, many of these figures seem to be outright fabrications rather than errors.

However, all this should be taken in the context of brand-new tech. Both Google and Wikipedia faced harsh criticisms for yielding unreliable information when they first arrived on the scene. It will take a while for the engineers to figure out how and why AI chatbots produce inaccurate results. But until then, don’t trust anything they say without checking for yourself first.

Source: DKB Blog