Generative AI in Data Analytics: Applications, Use Cases, and Best Practices

Whether you’re a technology consumer or a software provider, there’s never been a technological innovation that has enthralled everyone — from young children to teenagers, adults, and workforces in organizations of all sizes — like generative AI has. In fact, 78% of the world’s 5.16 billion internet users have personally used ChatGPT!

Open AI’s ChatGPT was the catalyst that brought the world’s attention to generative AI, with Microsoft, Google, AWS, and other leading companies quickly jumping in and stoking the flames of excitement. Since then, its rapid infusion into all kinds of technology has been astounding.

Understanding Generative AI Technology

Generative AI, or gen AI, is a transformative technology that has quickly become a focal point in various industries, including data analytics. At its core, generative AI refers to a class of algorithms that can create new content, be it text, images, music, code, or even complex data models. Unlike traditional AI, which primarily focuses on analyzing existing data to make predictions or classifications, generative AI goes a step further by generating new data that mimics the characteristics of the data it was trained on.

Generative AI is powered by deep learning, a subset of machine learning that utilizes neural networks to model and understand data. Two of the most prominent types of generative AI models are:

Generative Adversarial Networks (GANs): GANs consist of two neural networks — a generator and a discriminator — that work in tandem. The generator creates new data samples, while the discriminator evaluates their authenticity compared to actual data. Through this adversarial process, the generator improves its ability to produce increasingly realistic data over time.

Transformers: Transformers use self-attention mechanisms to process and generate sequences of text. Self-attention mechanisms allow models to weigh the importance of different words in a sequence relative to each other, enabling the model to capture relationships and dependencies within the content. These models are capable of producing coherent and contextually relevant text, as they do in GPTs (Generative Pre-trained Transformers).

The Speed of Adoption in the Workplace

As McKinsey’s annual global survey points out, 2023 is the breakout year for AI. The survey found that one-third of respondents already use gen AI in at least one business function, and 40% of respondents plan to increase their overall investment in AI based on its potential. Never before have we seen a new technology garner so much organizational attention, to the point where it’s now a key topic on 28% of board agendas.

Most companies see generative AI as a breakthrough technology to boost worker productivity by augmenting human activities, not replacing them. Its potential to make workforces more productive is why organizations are making it a priority investment. Microsoft has its CoPilot for Office 365 in early access and sees it as a paradigm shift in how people will work.

Generative AI can accelerate workflows and eliminate routine tasks. Research shows that its most significant impact lies in initiatives that drive top-line growth, such as personalized marketing and, more importantly, enhancing worker productivity. This is achieved by automating time- and resource-intensive tasks, generating content (including code) in seconds, and empowering employees to work with larger, more complex data sets.

To quantify generative AI’s impact on productivity, MIT conducted a research study on two groups. The study demonstrated that the group using generative AI tools improved worker performance by as much as 40% compared to those not using it. It also showed it can reduce the need for highly specialized skill sets by enabling anyone to generate basic code, analyze data, author a business process, and more.

Since this piece is about analysts and how generative AI can boost analytics productivity, let’s look at a case involving working with data. Typically, to access data, a data steward with specialized skills would write an SQL statement to query data from a data source and then hand it off to the analyst to perform analytics.

Now, with gen AI, instead of relying on a data steward to query data, the analyst can just enter a prompt that commands the creation of a customer list with details like columns for ‘first name,’ ‘last name,’ ‘age,’ ‘address,’ and ‘monthly spend is greater than $200 a month’. It’s that simple; anyone can do this with the right AI-powered analytics tools.

Whether you’re an operations, finance, sales, marketing, or HR analyst, anyone can author a prompt to access the data they need using natural language and generate analytics-ready data sets. And it takes just seconds.

Applications of Generative AI in Data Analytics

It’s clear that gen AI has made its impact felt across industries and business functions. Let’s briefly explore how it does so specifically within the context of data analytics. 

Finance

Generative AI is transforming the finance industry by automating data analysis and reporting. It can quickly analyze large datasets to detect patterns, anomalies, and trends, providing valuable insights for financial forecasting, risk management, and fraud detection. AI-driven algorithms can generate reports and dashboards that offer real-time insights, enabling financial analysts to make informed decisions faster.

Accounting

In accounting, generative AI can streamline processes by automating routine tasks such as transaction categorization, reconciliation, and report generation. Taking these repetitive tasks off the shoulders of accountants enables them to focus on more strategic activities like financial planning and analysis. AI also improves accuracy by reducing human error and ensuring compliance with regulatory requirements through automated checks and balances.

Tax

Generative AI enhances tax preparation and compliance by automating data collection and analysis. AI systems can sift through vast amounts of financial data and offer recommendations on tax-saving opportunities, regulatory compliance, and avoiding potential issues before they become problems. 

Marketing

AI can analyze customer data to create highly personalized marketing strategies and campaigns, crafting tailored messages that resonate with individual preferences and behaviors. It can generate content for emails, social media, and advertising, ensuring that marketing efforts are both targeted and efficient.

Sales

Gen AI analyzes customer interactions and behavior patterns to provide sales reps insight into customer preferences and needs. AI can also automate the generation of sales reports, forecasts, and proposals, enabling sales teams to focus more on building relationships and closing deals. 

Gen AI for Accessing Data — The First Step in Analytics

Given the power of gen AI for data access, it’s not surprising that data platform vendors have incorporated it into their products with good uptake.

Databricks recently announced LakehouseIQ, which enables users in any organization to use natural language to query, search, and understand data. Snowflake announced Document AI, which provides an LLM-based interface to understand and interpret PDF documents and convert data into data sets. Oracle infused gen AI into its Oracle Cloud infrastructure with a focus on use cases like fraud detection, where gen AI is used to sift through masses of financial data and generate new data to identify fraud patterns.

If you’re a data platform provider, adding gen AI capabilities to simplify access to data is more than just logical — it’s required.

Gen AI for Interpreting Insights — The Last Step in Analytics

We’ve evolved from visual dashboards to business intelligence (BI) tools that add natural language generation (NLG) to help interpret insights. Several years ago, we saw an onslaught of these NLG-powered BI dashboards.

In 2021, Gartner advised companies sitting on unexploited unstructured data to use NLG-powered tools to extract differentiating insights. Usage was encouraged for intelligent document processing and insights interpretation, albeit using very structured syntax. 

Fast forward two years, and gen AI is enabling a leap forward in interpreting insights. Top BI vendors like Tableau provide gen-AI-powered data visualization dashboards to give business stakeholders an easier way to surface and interpret insights.

With gen AI, a dashboard can now tell you what to focus on by automatically generating content that says: “Of the 12 metrics, two are unusual for this week. The unusual insights are this week’s ‘Sales by Region’ and ‘Top 5 product sales’.” You can then probe this further, and the gen AI will surface more discoveries on the unusual insights. Undoubtedly, it is a huge step forward for business stakeholders consuming dashboards daily.

The Middle — Where Is Gen AI in Solving the Hardest Part of Analytics?

Analytics processes first start with data access and end with insights to stakeholders. The infusion of generative AI into data platform solutions and BI tools helps to simplify these two ends of the process.

When you consider how analysts spend their day, though, the most significant time sink is the middle — the complex, repetitive tasks of prepping, transforming data, writing analytics logic, performing calculations, setting up reporting, and endless hours of analytics process iterations and maintaining dashboards.

While about 30% of analysts’ time is spent accessing data from data platforms or Excel files to create analytics-ready datasets, 70% is spent in painstaking cycles of assembling the analytics process, managing hand-offs, performing calculations, and maintaining insights.

Today, 60M+ business analysts in the workforce spend endless cycles doing manual, repetitive work like data prep, data blending, writing analytics logic, and wiring insights into dashboards, repetitively summarizing them in emails and static PowerPoint reports. They use Excel or other desktop tools. The manual nature of the job is tedious, time consuming, and naturally leads to errors that result in the business paying the price. 

How Specifically Can Generative AI Boost Core Analytics and Analyst Productivity?

It’s not hard to imagine that the place where analysts spend the bulk of their time could benefit tremendously from generative AI-driven automation. A recent Harvard Business Review article highlighted that finance, sales, and marketing analytical roles are amongst the jobs best positioned to use generative AI to improve analytics productivity.

From data extraction, cleaning, prep, and blending, to analytics logic and insight delivery, generative-AI-powered analytics solutions are proving to accelerate analytic results. Simply put, their applications cover the entire analytics development process.

The opportunity that lies ahead is that of a purpose-built solution for analysts that uses the power of the cloud, GPT conversational interfaces, and analytics automation to improve productivity by automating mundane tasks. Imagine a seamless, prompt-driven experience with drag-and-drop automation where you need it. No special skills required, no writing code or scripts, and it is simpler than Excel, making it approachable to anyone who needs to work with data. It takes self-service analytics to a whole new level.

Unlike previous generation tools, the limitations that plagued legacy analytics platforms no longer impede modern analytics automation platforms. With modern cloud-native architecture at its core, many advantages make it better suited for the contemporary data stack.

Here’s how analysts can simplify and speed up analytics outcomes:

AI-Assisted Data Extraction

Gen AI prompts can extract data from various sources without requiring manual SQL statements or scripts. They can also make recommendations and help avoid common mistakes.

‍

AI-Assisted Data Cleansing

‍Complex data cleansing can be simplified using gen AI prompts. Given that data cleansing is otherwise a tedious, time-consuming process, analysts can now achieve results in record time on large data sets.

AI-Assisted Data Preparation

‍Gen AI simplifies transforming and organizing data. Users can perform data prep tasks without the need to write complex expressions, understand syntax, or work with SQL. They can describe the task in natural language, and AI will generate the required code.

‍

AI-Assisted Analytics‍

Enables analysts to create complex analytics without manually authoring the logic, resulting in increased efficiency and reduced time spent on analytics workflow development.

AI-Assisted Insight Delivery‍

Users can specify where and how to publish the generated insights or data outcomes, providing greater control and flexibility in sharing valuable information.

Use Cases of Generative AI in Data Analytics

As you know, we’ve found a myriad of ways to put gen AI to use across industries, and data analytics is no different. Below are just a few examples of use cases of generative AI in data analytics:

Natural Language Processing (NLP)

Generative AI, through advanced NLP techniques, can analyze and interpret human language, making it a powerful tool for extracting insights from unstructured data sources such as text, speech, and social media. The technoogy can also be used to process large volumes of documents, extracting key information and generating summaries, eliminating the need for humans to sift through extensive documentation to extract relevant insights.

Predictive Analytics

Generative AI significantly enhances predictive analytics by analyzing historical data and learning patterns to forecast future trends and behaviors. Traditional predictive analytics relies on pre-defined statistical models, which can be limited in capturing complex relationships within data. Gen AI, with its deep learning capabilities, can analyze vast datasets, identify subtle patterns, and generate accurate predictions for everything from customer behavior and product demand to cash flow and market trends.

Data Augmentation

Data augmentation is all about generating synthetic data to enhance the quality and quantity of training datasets. Gen AI can create synthetic data that mimics the properties of real data, helping improve the performance of AI models, especially in scenarios where acquiring large amounts of real-world data is challenging, such as in medical research and cybersecurity.

Scenario Simulation

Scenario simulation uses generative AI to create different hypothetical situations, enabling organizations to analyze possible outcomes and strategize accordingly. This is particularly valuable in strategic planning, risk management, and decision-making processes across industries like finance and supply chain. 

Best Practices to Overcome Challenges of Generative AI in Data Analytics

Implementing generative AI in data analytics requires careful consideration and strategic planning. Here are some best practices to ensure successful integration and maximize its benefits:

Define Clear Objectives

Before deploying generative AI, clearly define the objectives you aim to achieve. Whether it's automating data processing, enhancing predictive analytics, or generating insights for business decisions, having specific goals will guide the development and implementation of AI solutions.

Start With Pilot Projects

Begin with small, manageable pilot projects to test the effectiveness of generative AI in specific areas. Assess its impact, refine models, and address any challenges before scaling up. Pilot projects also help attain buy-in from stakeholders by demonstrating early successes.

Ensure Data Quality and Governance

The effectiveness of generative AI heavily depends on data quality. Implement robust data governance practices to ensure data integrity, accuracy, and consistency. Regularly clean and update datasets to prevent AI from learning from or generating insights based on inaccurate or outdated information.

Incorporate Human Oversight

While generative AI can automate many tasks, you’ll still need human oversight to validate outputs, make informed decisions, and handle exceptions. Establish a feedback loop where human experts can review AI-generated insights, provide corrections, and refine models over time.

Focus on Explainability and Transparency

Stakeholders should understand how AI arrives at its conclusions to build trust in the system. Utilize model interpretability tools to make AI decision-making processes more transparent and easier to understand.

Ensure Compliance and Ethics

Adhere to data privacy regulations, such as GDPR or CCPA, and implement ethical guidelines for AI usage. Generative AI systems should be designed to respect user privacy, avoid biases, and operate within legal and ethical boundaries.

Invest in Training and Skill Development

Equip your team with the necessary skills to work with generative AI. Provide training on how to interact with AI tools, interpret AI-generated insights, and integrate AI into existing workflows.

Monitor Performance and Continuously Improve

Regularly monitor the performance of generative AI systems to ensure they meet business objectives. Use metrics and KPIs to measure the impact on efficiency, accuracy, and decision making. Continuously update and refine AI models based on feedback and evolving business needs.

Leverage a Multidisciplinary Approach

Engage cross-functional teams, including data scientists, domain experts, IT professionals, and business stakeholders, to collaboratively develop and implement AI solutions. Such a multidisciplinary approach helps keep AI applications technically sound and aligned with business goals.

Scalability and Integration

Plan for scalability from the outset. Choose AI solutions that can scale with your organization's growth. Ensure that generative AI integrates seamlessly with existing systems, tools, and workflows to maximize its effectiveness and minimize disruptions.

These best practices will help your organization effectively integrate generative AI into data analytics strategies, leading to more accurate insights, streamlined operations, and enhanced decision-making capabilities.

The Future of Generative-AI-Powered Analytics

Imagine a world where analysts can use simple prompts to create complete end-to-end analytics workflows — from defining datasets and specifying how they should be combined (data blending) to outlining the analytics logic and determining where insights will be published.

Frameworks like LangChain show great potential for achieving full automation of workflow creation with generative AI. For those unfamiliar with LangChain, it allows large language models (LLMs) to be linked together, with each model trained to handle a specific task.

At Savant, we’ve built an underlying analytics automation infrastructure that leverages LangChain and an intuitive user interface (UI) to enable the creation of entire workflows using gen AI. This is in addition to the discrete parts of the analytics process, such as data cleansing, data prep, data blending, and analytics logic described above.

We, alongside many organizations and analytics leaders we’ve spoken with, believe that providing generative-AI-assisted support for each discrete part of the analytics process, as well as the capability to generate end-to-end workflows using prompts, is essential to boost analyst productivity.

An AI-assisted analytics automation platform that offers both these capabilities fulfills our long-standing vision: that any analyst or business user should be able to pick up the tool and create their analytics processes within minutes. With this approach, not only can over 60 million analysts simplify and enhance their daily work, but even those who never imagined they could perform analytics can now do so with ease.

Wouldn’t it be exciting to feel the power of such a platform firsthand? Now, you can. Visit Savant and start your free trial to see for yourself how next-gen analytics automation can help your organization fulfill its potential.