Since Google Gemini has an iPhone app, I thought it would be a simple matter to snap a picture of my grocery list and have it give me an estimate as to total cost of my upcoming grocery store jaunt. It refused with a terse message. At first, I thought Gemini was being too strict, unable to respond due to Google’s too strict controls. Other Gen AI models were able to work, why not Gemini? It turned after a bit more exploration, suggested by Mark Loundy’s comments on Mastodon (thank you, Mark!), to simply be a problem with my prompt.

Note: I’ve edited this blog entry mostly on my phone. This update cleans up the text and has been edited (by me) for length and redundancy. Shorter is better.

The Grocery List

Prompt: Analyze picture and give me a breakdown on total price or cost

It should have been an easy job, but Gemini balked. It gave me this terse message:

I am sorry, but I cannot provide a total price or cost breakdown for the items in the image. The image is a handwritten shopping list, and it does not contain any pricing information. To get a cost estimate, you would need to know the specific brands, sizes, and current prices of each item from a particular store.

Having relied on Perplexity AI countless times, I switched over to BoodleBox (running in a Safari browser window on my iPhone), and gave it the exact same prompt.

BoodleBox with Perplexity Bot loaded came through!

BoodleBox Results

Here’s an estimate of the total price of the items on your shopping list, based on average prices at HEB in San Antonio, TX:

Estimated Total: $125 – $190

Perplexity also offered these caveats, which Gemini certainly could have done and I was OK with:

• Prices Fluctuate: Prices can vary based on location, sales, and specific product choices.
• Weekly Ads: Check HEB’s weekly ads for potential deals and discounts 3.
• Online Ordering: For the most accurate pricing, check HEB’s website or app for curbside pickup or delivery options.

Now, don’t get me wrong. Gemini has its awesome moments. But in this day to day one, on the fly via mobile, it failed me. It worked perfectly but I still wanted it to try and give me something.

It failed because it didn’t try…which it continues to do again and again. I’ve run into this problem on Gemini several times, where it says it can give me a response, then stops…as if someone had put its hand on its figurative shoulder and whispered, “No, you can’t.” I know the answer it gives might be imperfect, even inaccurate. I want it’s best guess.

A Best Guess

But what I was looking for was a best guess, a ballpark figure, based on available information, not a perfect estimate. Honestly, to do that, I would need a spreadsheet, accurate pricing off the HEB grocery store website, and…wait. Why can’t “King of Search” Google’s GenAI product, Gemini look that up?

Ain’t nobody perfect. I should adjust my expectations and stop expecting Gemini to do something it couldn’t get right because it’s not programmed for that. Worse, as so many point out, LLMs might have some difficulty with arithmetic.

Explaining LLMs

Mark Loundy via Mastodon points this out, patiently:

@mguhlin When a large language model gets an arithmetic function right it’s just a matter of chance. They do not have that function built into them.

@mguhlin This isn’t a “fail.” It’s an example of a user trying to use a tool for something for which it was never intended. Large language models are not calculators. They do not do arithmetic.

@mguhlin Prices are also regional and extremely dynamic. A price listed in the morning might be different by lunchtime. Plus, there’s no way for it to know which provider you’re going to use. What you describe is a good idea for a specialized app.

But I suppose Mark is right. I am expecting too much. Or, am I?

Wait, What About ChatGPT?

Sure enough, ChatGPT did a great job estimating costs, etc. The math was on par with Perplexity.

This leaves me wondering, “Why didn’t Gemini give it a go?” Probably because someone at Google restricted it, making it impossible for it to provide what other GenAI models could.

Another Try for Gemini

After Mark’s feedback, I decided that I should just ask Gemini, why? Of course, I asked it a little differently and also gave it a new prompt. You can read about it here yourself. An excerpt from that response:

Total Estimated Cost for your Grocery List: $136.25

Disclaimer: This is an estimate based on general HEB pricing. Your actual total at the store could be higher or lower depending on sales, specific brands purchased, exact quantities (e.g., weight of chicken/produce), and any store-specific pricing variations on the day you shop. It’s always best to check the HEB app or website for the most accurate current prices at your specific store location.

So in the end, Gemini didn’t fail, I did. I failed to provide a prompt that would get the desired result. Here’s my new prompt:

Prompt: Ok, transcribe this image (a grocery list) and give me a total based on individual items using pricing for Texas HEB grocery store.

Next time I want to use Gemini, I will need to use a prompt similar or exactly the same as this one.

Or…I could just start with BoodleBox’ via Perplexity, Claude, or ChatGPT bots. Or simply use another GenAI tool.