Careful with AI tooling

AI tooling

 Since some period I started working with AI tooling. Mostly I use Gemini and Visual Studio's Copilot. The experience is a bit of mixed feelings about this. Gemini had some good suggestions but also failed many times. Copilot has good code completion suggestions but misses the mark also occasionally. Copilot's function name suggestions are very welcome.

 Some examples of AI tooling failing are listed here:

• I asked Gemini for camera sharpness algorithm. It came up with a good algorithm but the actual OpenCV function calls and parameters were incorrect.
• I asked Gemini to get the real sample time from an 'IMediaSample'. It hallucinated and suggested to use the non existing 'GetSampleTime'. There is a 'GetMediaTime' function but this returns the stream time; i.e. the time since the graph was running and not the time from the start of the video. 
• I asked Gemini of conversion from UCS-2 to UTF-16. It wrongly suggested to use wstring_convert but wstring_covert is hardbound to std::string as byte_string
• I asked Gemini for a natural sort algorithm. It came up with a good implementation and a clever trick to circumvent the conversion from string to integer numbers. It lacked though the removal of leading zero's which is essential for the clever trick to work.
• I had to work on CMake lately but suggestions were often contradictory or using old conventions. In the CMake world things have shifted and perhaps the AI models are trained on old data.

 AI tooling can suggest plain bugs. I was implementing a swap of width and height and Copilot's code complete came up with the following code snippet:

// NOTE: incorrect 

Size sz = ...;
if (sz.GetWidth() < sz.GetHeight())
{
   sz.SetWidth(sz.GetHeight());
   sz.SetHeight(sz.GetWidth());
}

This doesn't swap but sets the width and height on the old height value.

 Conclusion

 AI tooling are helpful; mostly they give better answers than a Google search. It guided me lately through difficult CMake paths some very good answers but also ones who missed the boat. They are still not on the level to be trusted blindly. They also have limited scope for software engineering; e.g. code blocks; algorithms and functions. I am not aware if they can help in refactoring and extending architecture wide solutions.