Big Data Analytics in Insurance

Insurers are turning to big data analytics to strike a difference in the highly commoditized insurance market and improve risk management in the context of growing regulations. This move is helping the industry reap rich benefits across the value chain. In this white paper we examine some of the emerging practices in the Property and Casualty insurance sector particularly in relation to product pricing, underwriting, claims handling, customer relationship management, and reinsurance.


What’s Included?
  • Data: The Insurance Asset
  • Insurance in the Age of Big Data
  • Application of Big Data Analytics in Insurance-P&C
    • Product Design and Pricing
    • Underwriting
    • Claims Handling
    • Customer Relationship Management
    • Reinsurance
  • Bringing Efficiencies in Call Center Operation—A Use Case
  • The Way Ahead

Data: The Insurance Asset

The insurance industry has always been data-driven. It relies heavily on data to make strategic business decisions across the value chain, from marketing to sales to making the right investments. It performs its core function of protecting customers from economic risks by transferring those risks onto itself, a proposition that is advantageous to both parties only when the risks are well understood and adequately distributed. The industry has gone by empirical knowledge and historical data to strike a balance between risks and returns. Big data analytics in insurance is about fine-tuning that balance.

Insurance in the Age of Big Data

Until recently, the ambit of insurance data—most of it structured—has been small. The explosive growth and pervasive presence of digital technologies with several points of convergence has resulted in data buildup that is big not just in terms of volume but also in its speed of accumulation and variety. Such massive data, much of it unstructured, would have been confounding to say the least, but for the advanced storage and analytical capability that has coevolved with it. Although the insurance industry as a whole is only beginning to take a position vis-à-vis big data, the achievements of pioneers in the industry suggest that the old business model is no longer sustainable. A recent study posits substantial growth in the use of new data sets for pricing and underwriting within the next couple of years1.

Big Data Analytics in Insurance: Trends in use of new data sets in insurance

Trends in use of new data sets in insurance

Big data solutions derive value from their ability to tackle the four imposing dimensions of data—volume, velocity, variety, and veracity—and transform hidden data into actionable insights much faster than was previously possible.

Of the four dimensions, veracity, or the accuracy of data, is an overarching issue for insurance given its impact on pricing effectiveness. Wrong data can skew rating and result in disproportionately lower premiums. Given the economies of scale, this could amount to a loss of billions of dollars each year. For a market-facing industry that survives on slender margins, this could spell disaster.

For long, insurers have assessed risks using data captured at various points of the transaction (such as policy and claims data) and external data (such as credit reports, driving records, and CLUE reports). The digital ecosystem within which insurers operate today offers more bulwark against the risk of wrong data leading to wrong decisions. Apart from claims history, insurers can now access data on their customers from social media, shopping sites, emails, and call centers and adjust premiums based on more accurate assessments.

While the advantageous use of data is expected to help smooth risks across all insurance sectors, the Property and Casualty (P&C) industry, which is susceptible to large aggregate losses resulting from catastrophes, could have even more significant gains. The hurricane seasons of 2004 and 2005, which resulted in seven million insurance claims and $100 billion in insured losses2, highlighted the deficiencies in the data used for catastrophe modeling. Risk assessment practices within P&C have undergone a sea change following these landmark events. With the emergence of cutting-edge modeling solutions, location intelligence and visualization tools, insurers can correlate coverage to risks more effectively.


Application of Big Data Analytics in Insurance-P&C

Big Data Analytics in Insurance: Application of Big Data Analytics in P&C Insurance

 Product Design and Pricing

Insurers depend on the rich analytical capability of the actuarial function to obtain a comprehensive view of their business, reduce loss ratios, and inform future pricing. The volatile market that exists today has compounded the challenge for actuaries who now have to demonstrate a greater understanding of capital management.

With the help of advanced tools, actuaries can now perform multivariate analyses and modeling, leading to effective pricing decisions. From developing custom policies at competitive premiums to real-time pricing based on market conditions, actuaries can leverage big data to keep firms firmly entrenched on the efficiency frontier.


Profitable underwriting depends on accurate risk evaluation, which comes from empirical experience in underwriting similar risks as well as from theoretical analyses of data. With big data, underwriters can advance their decision-making capability by assessing risks against a range of factors in more granular detail.

Statistical models can be built by integrating multiple data sets to understand and quantify risks with greater accuracy. Using predictive modeling based on location analytics, insurers can estimate potential losses from natural catastrophes and thus facilitate efficient claims management in the event of a disaster.

Automobile insurers are turning to telematics to fix premiums on par with the risk exposure and improve their risk management profile while motivating customers to earn lower premiums by adopting safe driving practices.

Claims Handling

Every insurance company is judged by its claims-handling efficiency. While making speedy payouts is important, identifying deserving claimants is doubly important. Despite safeguards, fraud accounts for 10% of the loss incurred by insurance companies each year3. This has cascading effects for both the insured and the insurer.

Claim adjusters are trained to spot deceptive claims; however, due to the paucity of trained resources, the desired level of rigor is not always maintained while processing the mounting claims. As fraud becomes sophisticated and new members enter the fraud ring, insurers have to tighten their vigilance a notch higher.

With the help of text analytics or natural language processing, insurers today can mine the semi-structured information in claims applications, adjuster notes, and social media to identify patterns or discrepancies. Based on their initial findings, insurers can alter their line of inquiry or flag claims for further investigation.

Customer Relationship Management

With the arrival of customer-specific products such as pay-as-you-go insurance and the proliferation of competing products, building and shoring the customer base has assumed critical importance. In a commoditized marketplace, customer relationship management is no longer only about building good relationships, but also about gauging customer sentiment and meeting customer expectations.

Using big data tools, insurers can build a unified, accurate, and robust customer profile from a distributed data pool (call center logs, social media, emails, website visits), optimize their resources by choosing the right channel of communication, and personalize messages for a meaningful customer engagement.


With the paradigm shift from attracting new customers to strengthening the existing customer base and earning long-term loyalty and profits, the ability to cross-sell and upsell relevant products will have a huge impact on the company bottom line. By mining the rich customer data available with call centers, insurers can make predictions about additional or future customer requirements and pitch their sales more effectively.


From wider risk exposure to alternative capital flow into the market, the reinsurance industry is besieged by challenges, both old and new. Reportedly, a third of the data that is returned to the industry is incorrect, leading to significant loss of operating revenue. To safeguard against such a fallout, the reinsurance industry needs access to consistent and reliable data. Establishing a centralized database that can straddle the volume and variety in insurance data and support decision-making through knowledge gained from risk modeling, trend analysis, predictive modeling, and so on, gives reinsurers an advantage amidst the competing market forces.

Bringing Efficiencies in Call Center Operation—A Use Case

Big Data Analytics in Insurance: Bringing Efficiencies in Call Center Operation—A Use Case

The Way Ahead

With the emergence of advanced risk rating mechanisms, custom products, and other technology-enabled innovations, the competition within the insurance industry is set to hot up. Upcoming insurance regulations, with their emphasis on stringent capital management, could soon impose additional burden on insurers, forcing them to weigh their decisions more carefully. Intelligent use of data can help insurers gain the resilience they need to thrive in this dynamic environment.

To leverage big data, the industry has to address the intermediate technical challenges involved in breaking data silos and streamlining the various technologies. It is equally important to build consensus across the board to embrace the change. These seemingly daunting tasks can be achieved if insurers start small by piloting projects with clearly defined objectives and mainstream those concepts depending on the outcomes.


1. Earnix. (n.d.). UK Modelling Data Acquisition and Usage Trends 2015 Survey Results.
Retrieved from
2. National Association of Insurance Commissioners. (2015, June). Natural Catastrophe Response.
Retrieved from
3. Insurance Information Institute, Inc. (2015, March). Insurance Fraud.
Retrieved from