Brittbot

Upgrading from Data Center Tech to Software Engineer

A chatbot sitting on a computer chair typing on a keyboard connected to a desktop computer sitting on a desk. The chatbot looks confused.

Gemini Does Make Mistakes

—

by

Brittany Gates
in

Yesterday, I wrote a post about my great experience using Google AI, especially Gemini (which you can read here). Well, I ran into a problem using the chatbot today, which reminded me that Gemini does make mistakes. However, the mistake I ran into forced me to research and learn so there is a silver lining. I’ll explain that in more detail a little later in this post.

When It Comes To Code Snippets Gemini Does Make Mistakes

Allow me to set up the scene: I’m writing code for my backup program (which I will discuss in a future post), and I wanted to know the following:

My prompt in Gemini: "Is there a Python module I can use that checks for a Linux-compatible file system?"

This is Gemini’s reply:

The reply from Gemini that shows incorrect Python code.

I read through the explanation about the code provided:

The explanation of the Gemini reply about the Python code snippet it supplied.

I wasn’t going to copy the code and paste it into my Integrated Developer Environment (IDE) without knowing what it did first. That’s tempting, don’t get me wrong, but I want to understand what the code does. That way I can learn and use that knowledge to writing better code in the future.

So after understanding what the function did I customized the code for my project. Then I ran into a problem: My IDE (PyCharm) didn’t have “f_fstype” in the autocomplete. I thought I made a typo so I retyped it. That didn’t work. I turned to Google Search and searched to see if this was a problem with my computer or even PyCharm. There wasn’t a problem with my setup.

I finally visited the official Python documentation and found the entry regarding os.statvfs(), read about the options the module used and “f_fstype” wasn’t one of them.

I went back to Gemini and asked if “f_fstype” was part of os.statvfs() and the chatbot said it was:

My question to Gemini about the Python code it provide, and its incorrect reply.

As you can see Gemini does make mistakes and can double-down on them. So what did I do?

I Left Feedback

I left negative feedback about this answer because it’s incorrect. Then I provided a link to the official Python documentation that shows all the available options for the os.statvfs() module.

This is important because Gemini, like any other Large Language Model (LLM) learns. There’s a possibility it learned incorrect information just like you or I would. And when I find out I learned wrong, I go to learn the correct information. Gemini needs to do that too. Thus, leaving feedback is so vital.

So Gemini Does Make Mistakes…And Makes Me Think

When I got the incorrect information I had to rethink my solution for my problem. I did use Gemini again for assistance, but first I wanted to think about how to approach the problem. I used some Linux commands on my computer like “stat” and wondered if I could use that to gather the information about the file system on a disk.

While this setback could have discouraged me, I didn’t let it. Writing code, programming a computer, developing software, whatever you want to call it is about solving a problem. Sometimes the problem is straight-forward, other times it’s not. Becoming a great code or programmer or Software Engineers is about finding a one or more solutions to a problem. That’s what I learned from this situation.

So thanks, Gemini!