What Makes Big Data, 'Big Data'

This post has been slightly adapted to a blog post from an APA paper written by the author. Download the APA paper here!

Executive Summary   ~1 min

The concept of Big Data has evolved beyond its traditional 'Three Vs' model—volume, velocity, and variety—to include newer aspects like exhaustivity, resolution, indexicality, relationality, extensionality, and scalability. These characteristics allow firms to take a more comprehensive approach in leveraging Big Data for a competitive advantage.

The volume of data refers to the quantity of data collected, with larger datasets providing more valuable insights. Velocity refers to the rate at which data is collected and processed, while variety pertains to the format of the data, from structured to semi-structured and unstructured.

Exhaustivity ensures comprehensive data collection, while resolution allows for granular analysis. Indexicality pertains to unique data identification and storage, and relationality facilitates comparison through common fields. Extensionality provides flexibility in adding or dropping fields, while scalability ensures the data set can rapidly expand as a firm grows.

Big Data can be leveraged in numerous ways, including product development, predictive maintenance, customer experience improvement, fraud prevention and compliance, machine learning, operational efficiency enhancement, and innovation driving. Firms can use predictive models derived from Big Data to develop new products, maintain systems proactively, and tailor customer experiences based on specific behaviors.

Big Data can also reduce fraud and enhance legal compliance, as illustrated by PayPal's automated fraud-blocking systems. Machine learning benefits significantly from Big Data, with systems like ChatGPT improving their efficiency with more extensive and diverse datasets. Analytical tools utilizing Big Data can increase operational efficiency by anticipating demands and reducing outages, and innovation can be driven by analyzing trends and market needs.

In conclusion, a comprehensive understanding of Big Data's characteristics and potential applications can significantly impact a firm's competitive edge and long-term success. Leveraging Big Data strategically allows organizations to adapt to changing market conditions, enhance customer experiences, and drive industry innovation, leading to increased profitability and sustainable growth.

Introduction   ~5  min

Big Data has become a buzzword that is hard to understand, as it can mean so many different things to different people. While most think of Big Data as the traditional ‘Three Vs’ model, Big Data has grown beyond that and now must be defined as things such as “volume, velocity and variety (the 3Vs), but also exhaustivity, resolution, indexicality, relationality, extensionality and scalability” (Kitchin, 2016) in order to get a more comprehensive view of Big Data and what it is. Not only is defining what Big Data is important, but also how Big Data is important for firms and can be leveraged to give these firms an advantage over their competitors. Some of the ways in which Big Data can be leveraged by firms on a daily basis are things such as, Product Development, Predictive Maintenance, Customer Experience, Fraud and Compliance, Machine Learning, Operational Efficiency, and Drive Innovation (Oracle, 2023).

The Characteristics of Big Data

The traditional measures of Big Data can be defined as volume, velocity, and variety. Volume refers to the amount of data collected, and in Big Data, the more data that a data set holds, the more valuable it is, as you will be able to gather greater insights that you may not have been able to see with a smaller data set. Velocity refers to the rate at which data is collected, such as real-time to periodic. Some metrics also take into account not only the velocity at which data is collected but also how quickly data can be processed and used (Oracle, 2023). Variety refers to how the data set is stored, such as “being structured, semi-structured [or] unstructured” (Kitchin, 2016). Typically, as velocity increases, the organization and structure of data goes from structured, to semi-structured, to unstructured as there is simply too much data coming into the data set all at once, and there is not enough processing power to properly sort it in real-time (Callender, 2023).

Moving away from the traditional model of the ‘Three Vs,’ which was characterized in 2001 by Doug Laney, additional ways in which large data sets can be measured if a data set is truly ‘Big Data,’ is through looking at its exhaustivity, resolution, indexicality, relationality, extensionality, and scalability. Exhaustivity is the measure that ensures that the data set is collecting all available data. Examples of this are social media platforms that measure likes, comments, saves, shares, as well as watchtime. Resolution means that the data can be broken down to the granular level, enabling further insights into what is being measured and collected. Indexicality means that it can be uniquely identified and stored in the data set. Relationality means that there are common fields in which all data can be compared against. This greatly helps in analytics where you want to compare two things to each other using similar or the same fields to see what differs from one set of data to the next. Extensionality means new fields can be added and dropped easily, allowing a firm to quickly add or delete important data that arises in the future. Finally, Scalability means that the data set can expand in size rapidly as a firm grows exponentially. This is important because startup firms grow rapidly, going from no users to millions per month, such as ChatGPT going from 0 users to over 1,000,000 in only five days and from 0 to over 100 million active monthly users in just two months (Hu, 2023). 

How Firms Can Leverage Big Data

Big Data can be leveraged in many ways, such as Product Development, Predictive Maintenance, Customer Experience, Fraud and Compliance, Machine Learning, Operational Efficiency, and Drive Innovation (Oracle, 2023). Big Data in product development can be used to “build predictive models for new products and services by classifying key attributes of past and current products or services and modeling the relationship between those attributes and the commercial success of the offerings” (Oracle, 2023). This has been used by companies such as Netflix and Procter & Gamble to “plan, produce, and launch new products” (Oracle, 2023). By using these large data sets, firms can gather insight into their target demographic and design new product offerings around these gained insights, potentially even before customers know that they want those specific aspects of your product. Big Data’s use for predictive maintenance can save your firm thousands of dollars, as you can find trends for crucial system breakdowns and fix them before they fail catastrophically, causing your firm downtime and increased costs due to emergencies.

This can also be done in a client-facing environment, where you install sensors on customer products, which tell them when they need to maintain their product in order for it to stay operational for longer based on your firm’s Big Data sets found in your servers based on historical maintenance data, an example being the sensors found in the automotive industry (Oracle, 2023). The customer experience can also be improved using Big Data by allowing your firm to paint a clearer picture of what your customers do and how they use your products, services, and communication channels (Oracle, 2023). Examples of this in action are special promotions that are customer-specific and target exactly what that customer is most interested in buying based on their past purchases.

Through the use of Big Data analytics, organizations can decrease the amount of fraud and increase their legal compliance. A specific example of this is when PayPal introduced its automated fraud-blocking systems from one banking institution to another. Using their historical data, they were able to introduce a system that automatically blocked transactions that they deemed fraudulent based on past transactional data and IP usage, cutting down on the amount of fraud, and increasing their legal compliance. Machine learning, such as OpenAi’s offering ChatGPT has been made possible due to the increase in availability of Big Data. Machine Learning (ML) and Large Language Models (LLMs) are trained on these large data sets and increase in efficiency and reliable output when more data and parameters are fed into them.

"Big Data has been instrumental in [ChatGPT’s] development, providing the vast and diverse datasets that fuel [its] learning capabilities and enable [it] to generate accurate and contextually relevant responses for users" (ChatGPT 4.0, 2023, Verified by Lee, 2023, See Appendix #2). Increases in operational efficiency can occur by using analytical tools to “analyze and assess production … [to] reduce outages and anticipate future demands” (Oracle, 2023). By increasing operational efficiencies, you can decrease your costs and make your business more efficient over all, potentially giving your firm a competitive advantage. Big Data can be used to drive innovation through analyzing trends and what is needed in the world, much like with product development efforts. By using data to provide something new to the marketplace, you will be able to set yourself apart from the competition and increase profitability.

Conclusion

Big Data has evolved past the traditional definition of the ‘Three Vs’ and now requires more defining characteristics in order to differentiate normal data sets from ‘Big Data.’ Through the expansion of Big Data, we have enabled firms to take advantage of the continuous amounts of data that they are intaking, and analyze it through various means to do different things such as increase customer experience, operational efficiency, or decrease the amount of successful fraud. “As demonstrated in this paper, a deeper understanding of Big Data's characteristics and potential applications can significantly impact a firm's competitive advantage and long-term success. By harnessing the power of Big Data and strategically implementing the insights derived from it, organizations can better adapt to rapidly changing market conditions, enhance customer experiences, and drive innovation in their industries, ultimately leading to increased profitability and sustainable growth” (ChatGPT 4.0, 2023, See Appendix #3).

Thank you for reading my blog post! If you found any of this information helpful or know someone who would find it useful, please share it with them. Let me know what you think in the comments below. Consider joining my mailing list to keep learning as I continue to add value with more blog posts and articles about topics such as Information Technology Management and Marketing.

Once again: This post has been slightly adapted from an APA paper to a blog post written by the author. Download the APA paper here!