Raising the quality of insights in agentic analytics

In just a few years, generative AI has changed the face of enterprise analytics, notably by making analytics accessible to users across the organization. Agentic AI promises to do even more, by automating entire workflows and taking actions based on agent insights. But there is a problem: The quality of insights that agents currently generate leaves much to be desired.

Fewer than half of agent-generated insights at the organizations surveyed for this report lead to business improvement. And while most respondents praise the quality of their agents’ descriptive insights, reviews of their prescriptive insights—those on which actions are based—are decidedly mixed.

DOWNLOAD THE REPORT

Failure to produce insights that improve the business deprives organizations of their desired return on investment in agentic analytics. Worse, flawed insights that trigger incorrect actions can cause reputational or financial damage. This report gauges how companies view the quality of the outputs produced by their analytics agents and explores how organizations are seeking to boost insight quality and get their agents working harder for the business.

Some key findings from the report include the following:

Agentic analytics is only just starting to deliver. On average, 48% of insights generated by analytics agents at the surveyed companies lead to actions that improve the business. That is a marked advance from the 37% average the respondents say generated such actions one year ago. But it means that more than half of the insights generated by analytics agents today are still failing to help businesses.

Better quality of insights leads to clear business advantage. At a small cohort of organizations, which we term “insight leaders,” AI agents perform much better than the average. In this group, more than 70% of agent insights lead to actions that improve the business. The insight leaders also give the quality of their agents’ insights higher marks than do the total respondents overall. The quality gap is particularly wide when it comes to prescriptive insights.

Insufficient context reduces insight quality. When analytics agents deliver insights of substandard quality, the main reason is a lack of business context, as cited by 55% of surveyed executives. Fragmented or inaccessible data is the other major factor, according to 45%. Fewer insight leaders say their agents struggle with access to context.

Insight overload further inhibits agent effectiveness. At a majority (52%) of surveyed organizations, analytics agents generate too many insights to be properly evaluated and therefore acted upon. Less than one-quarter (24%) are able to evaluate all agent insights; most do so on a case-by-case basis, and that process is currently human-intensive.

Governance and semantics are areas of focus for improving insights. The surveyed companies are prioritizing initiatives that enable better agent insights by strengthening AI and data governance (cited by 60%). They are also adding a semantic layer to their data architecture (45%). The experts we interviewed agree that the semantic layer is crucial to ensure high-performing agents.

Download the report

This content was produced by Insights, MIT Technology Review’s custom content arm, not its editorial staff. It was researched and written by humans, with any AI tools that may have been used limited to production processes under human oversight.

News is gathered automatically from public robotics & AI feeds on a schedule.