Fleet managers should treat general purpose AI tools with ‘high degree’ of caution

Popular, general purpose artificial intelligence (AI) tools should be treated with a ‘high degree’ of caution by fleet managers the FleetCheck is warning.

The fleet software specialist says generative AI such as Claude and Chat GPT can be extremely effective but also frequently highly misleading.

Callum Haymon-Collins, COO, said: ‘We are now seeing fleet managers across our user base start to use AI quite extensively and getting a good feel for the reliability of its results.

‘There is no question that sometimes, its responses are excellent and rapidly creates insights from data that would be difficult to achieve through any other means. However, there are also times when its outputs lie somewhere on a line between misleading and dead wrong.

‘For example, I recently uploaded some fuel card data into a popular AI tool and its key finding was that I should closely question the provider about pump price rises between February and March, strongly implying there could be some commercial malpractice underway. It didn’t spot the war in Iran had caused the issue, which is obviously a fundamental error.’

Problems of this type were only likely to become more prevalent over time, he added, as the number of hallucinations produced by generative AI increased.

‘We are in an odd situation where the AI models are getting better but the data they use is getting worse because the information on which they train is now largely based on AI output. 

‘OpenAI’s own publicly published research shows that its latest reasoning model now creates hallucinations 48% of the time compared to just 16% in older versions. This is not a bug that can be solved but a feature of the technology. Especially, it is poor at admitting when it can’t produce a credible answer and tends to often present supposition as fact.

‘The only solution is closer human involvement and, for fleet managers, that means sense checking the output and not taking anything at face value. It is worth bearing in mind that the hallucinations can often be subtle but still sufficiently wrong to cause issues.’

Callum added that FleetCheck was carefully ringfencing its own use of AI, particularly when it came to customer facing technology.

He said, ‘Many fleet use cases for AI can be constructed but there has to be a very high level of reliability. Where we are offering products that use generative AI, such as document processing, we can limit the potential for any issues to a tiny degree. Also, these are relatively mechanical tasks and even if there is an error, the real world impact is small.

‘However, if you are a fleet manager employing AI for wider strategic insight and advice, the risks are much greater. It would certainly be possible to make decisions that are expensive, time consuming and embarrassing unless you exercise a high degree of caution.’