Blog Archives
The Big Data Movement: Importance and Relevance today?
We are entering into a new age wherein we are interested in picking up a finer understanding of relationships between businesses and customers, organizations and employees, products and how they are being used, how different aspects of the business and the organizations connect to produce meaningful and actionable relevant information, etc. We are seeing a lot of data, and the old tools to manage, process and gather insights from the data like spreadsheets, SQL databases, etc., are not scalable to current needs. Thus, Big Data is becoming a framework to approach how to process, store and cope with the reams of data that is being collected.
According to IDC, it is imperative that organizations and IT leaders focus on the ever-increasing volume, variety and velocity of information that forms big data.
- Volume. Many factors contribute to the increase in data volume – transaction-based data stored through the years, text data constantly streaming in from social media, increasing amounts of sensor data being collected, etc. In the past, excessive data volume created a storage issue. But with today’s decreasing storage costs, other issues emerge, including how to determine relevance amidst the large volumes of data and how to create value from data that is relevant.
- Variety. Data today comes in all types of formats – from traditional databases to hierarchical data stores created by end users and OLAP systems, to text documents, email, meter-collected data, video, audio, stock ticker data and financial transactions. By some estimates, 80 percent of an organization’s data is not numeric! But it still must be included in analyses and decision making.
- Velocity. According to Gartner, velocity “means both how fast data is being produced and how fast the data must be processed to meet demand.” RFID tags and smart metering are driving an increasing need to deal with torrents of data in near-real time. Reacting quickly enough to deal with velocity is a challenge to most organizations.
SAS has added two additional dimensions:
- Variability. In addition to the increasing velocities and varieties of data, data flows can be highly inconsistent with periodic peaks. Is something big trending in the social media? Daily, seasonal and event-triggered peak data loads can be challenging to manage – especially with social media involved.
- Complexity. When you deal with huge volumes of data, it comes from multiple sources. It is quite an undertaking to link, match, cleanse and transform data across systems. However, it is necessary to connect and correlate relationships, hierarchies and multiple data linkages or your data can quickly spiral out of control. Data governance can help you determine how disparate data relates to common definitions and how to systematically integrate structured and unstructured data assets to produce high-quality information that is useful, appropriate and up-to-date.
So to reiterate, Big Data is a framework stemming from the realization that the data has gathered significant pace and that it’s growth has exceeded the capacity for an organization to handle, store and analyze the data in a manner that offers meaningful insights into the relationships between data points. I am calling this a framework, unlike other materials that call Big Data a consequent of the inability of organizations to handle mass amounts of data. I refer to Big Data as a framework because it sets the parameters around an organizations’ decision as to when and which tools must be deployed to address the data scalability issues.
Thus to put the appropriate parameters around when an organization must consider Big Data as part of their analytics roadmap in order to understand the patterns of data better, they have to answer the following ten questions:
- What are the different types of data that should be gathered?
- What are the mechanisms that have to be deployed to gather the relevant data?
- How should the data be processed, transformed and stored?
- How do we ensure that there is no single point of failure in data storage and data loss that may compromise data integrity?
- What are the models that have to be used to analyze the data?
- How are the findings of the data to be distributed to relevant parties?
- How do we assure the security of the data that will be distributed?
- What mechanisms do we create to implement feedback against the data to preserve data integrity?
- How do we morph the big data model into new forms that accounts for new patterns to reflect what is meaningful and actionable?
- How do we create a learning path for the big data model framework?
Some of the existing literature have commingled Big Data framework with analytics. In fact, the literature has gone on to make a rather assertive statement i.e. that Big Data and predictive analytics be looked upon in the same vein. Nothing could be further from the truth!
There are several tools available in the market to do predictive analytics against a set of data that may not qualify for the Big Data framework. While I was the CFO at Atari, we deployed business intelligence tools using Microstrategy, and Microstrategy had predictive modules. In my recent past, we had explored SAS and Minitab tools to do predictive analytics. In fact, even Excel can do multivariate, ANOVA and regressions analysis and best curve fit analysis. These analytical techniques have been part of the analytics arsenal for a long time. Different data sizes may need different tools to instantiate relevant predictive analysis. This is a very important point because companies that do not have Big Data ought to seriously reconsider their strategy of what tools and frameworks to use to gather insights. I have known companies that have gone the Big Data route, although all data points ( excuse my pun), even after incorporating capacity and forecasts, suggest that alternative tools are more cost-effective than implementing Big Data solutions. Big Data is not a one-size fit-all model. It is an expensive implementation. However, for the right data size which in this case would be very large data size, Big Data implementation would be extremely beneficial and cost effective in terms of the total cost of ownership.
Areas where Big Data Framework can be applied!
Some areas lend themselves to the application of the Big Data Framework. I have identified broadly four key areas:
- Marketing and Sales: Consumer behavior, marketing campaigns, sales pipelines, conversions, marketing funnels and drop-offs, distribution channels are all areas where Big Data can be applied to gather deeper insights.
- Human Resources: Employee engagement, employee hiring, employee retention, organization knowledge base, impact of cross-functional training, reviews, compensation plans are elements that Big Data can surface. After all, generally over 60% of company resources are invested in HR.
- Production and Operational Environments: Data growth, different types of data appended as the business learns about the consumer, concurrent usage patterns, traffic, web analytics are prime examples.
- Financial Planning and Business Operational Analytics: Predictive analytics around bottoms-up sales, marketing campaigns ROI, customer acquisitions costs, earned media and paid media, margins by SKU’s and distribution channels, operational expenses, portfolio evaluation, risk analysis, etc., are some of the examples in this category.
Hadoop: A Small Note!
Hadoop is becoming a more widely accepted tool in addressing Big Data Needs. It was invented by Google so they could index the structural and text information that they were collecting and present meaningful and actionable results to the users quickly. It was further developed by Yahoo that tweaked Hadoop for enterprise applications.
Hadoop runs on a large number of machines that don’t share memory or disks. The Hadoop software runs on each of these machines. Thus, if you have for example – over 10 gigabytes of data – you take that data and spread that across different machines. Hadoop tracks where all these data resides! The servers or machines are called nodes, and the common logical categories around which the data is disseminated are called clusters. Thus each server operates on its own little piece of the data, and then once the data is processed, the results are delivered to the main client as a unified whole. The method of reducing the disparate sources of information residing in various nodes and clusters into one unified whole is the process of MapReduce, an important mechanism of Hadoop. You will also hear something called Hive which is nothing but a data warehouse. This could be a structured or unstructured warehouse upon which the Hadoop works upon, processes data, enables redundancy across the clusters and offers a unified solution through the MapReduce function.
Personally, I have always been interested in Business Intelligence. I have always considered BI as a stepping stone, in the new age, to be a handy tool to truly understand a business and develop financial and operational models that are fairly close to the trending insights that the data generates. So my ear is always to the ground as I follow the developments in this area … and though I have not implemented a Big Data solution, I have always been and will continue to be interested in seeing its applications in certain contexts and against the various use cases in organizations.
Applying Gamification in the Workplace
Wikipedia defines gamification as the use of game mechanics and game design techniques in non-game contexts. It applies to non-game applications and processes, in order to encourage people to adopt them, or to influence how they are used. It makes technology use more exciting and engaging, and encourages users to engage in desired behaviors with fruitful consequences to the environment where these techniques and processes are being deployed.
Many years ago, I took a series of courses at Cal Tech in Pasadena at the School of Industrial Relations. One of the courses was applying tools to encourage teamwork and participation. Thereafter, I have attended field trips in organizations in strategic off-sites where we had to do rope walking, free fall, climbing bamboo structures strung together to retrieve flags, etc. Thus, in those days – we applied sports and board games to fuel a shared success environment. Now things have become more technology oriented, and we have thus seamlessly transitioned to some extent from those environments to consumer web based experiences. This does not suggest that the other alternatives are less rewarding; they draw upon other types of triggers but gamification through technology is more accessible and generally less expensive with less overhead in the long run.
What are the four key elements in Gamification?
Games generally tend to have four elements that are closely intertwined. Absent any of these four elements and the jury would be out on whether the application could suitably be considered gamified. Clearly, when these elements are being applied to non-gaming contexts, you will find that some of these elements are more watered down or cruder representation of game design principles or applications to actual game play environments. Regardless, all of these elements are necessary conditions that must come into play.
1.Narratives
Games have narratives. They must be able to tell a story. They must place the player or user in a context, make them aware of the context, create a temporal dimension of a past, present and future and provide a theme or a set of themes that the players pursue.
2.Game Mechanics
These constitute the provision of tools and use cases that create PvP (Player vs. Player) or PvE (Player vs. Environment) experience. Common tools like teleporting, cockpit load (number of player controls), in-game user interaction, human-computer interaction, etc. come into play. The mechanics must aptly support the narrative.
3.Aesthetics
People look for rich experiences. In MMORPG, the aesthetics are extremely rich and immersive. In gamified applications, it need not be so. Regardless, users have continued to raise the bar on aesthetics and richness of media to support their interaction. So the trend toward aesthetics will continue, albeit at a lower benchmark than would be in the extreme case of a high quality MMORPG game.
4.Rewards
Finally, games have to have a purpose. The narratives have to have a light at the end of the tunnel. There is a carrot and stick principle in game design. It is a very important component to either persuade people to behave or not behave in a certain manner. Rewards are vanity points awarded for achieving goals that are user-driven or context driven. Either way, it is and will continue to remain the key element in game design.
The myth of rewards!
In one my earlier blogs, I laid out the distinction between intrinsic and extrinsic motivation. This has bearing on the concept of rewards and recognition in the workplace. You can find the details in my blog – “Intrinsic and Extrinsic Motivation: Impact on Employee Engagement”. (https://linkedstarsblog.com/2012/10/16/intrinsic-and-extrinsic-motivation-impact-on-employee-engagement/
Designing an application with rewards to fuel engagement in the workplace is a good idea. But rewards have to follow a narrative, a storyline. For example, an application that simply awards points and badges based on transactions without a narrative cannot be considered an application that applies all of the elements of the gamification process. It only addresses one element, and in fact, for some it is the least important component. When one focuses the product design around this single component, I contend that you are not really gamifying; you are in fact drawing upon some temporary impulses that are not sustainable and enduring.
Hence, the narrative and craftsmanship is quite critical to gamifying an application and making it relevant for employees in the workplace.
One must adopt the right mix of the gaming elements to ultimately create ends such as stickiness, re-engagement, and deeper levels of interaction, fun, challenge, promoting options to cooperate and also compete, and broadcast success.
Scales:
So now we arrive at assessing the scales to benchmark each of those ends. I am being particular by not calling these tools, since tools are to support mechanics whereas scales are manifestations of an end result. For practical design and implementation purposes, here are a few scales that are common across all games, some of which are quite relevant for gamification in an employee setting. Some of the more common scales to assess or broadcast success are:
1) Leaderboards
2) Achievement levels and measures of achievements
3) Challenges between users
4) Progress Bars
5) Reward Points that have redemption value
To reiterate, for the final outcomes associated with the scales to be meaningful, the narrative is extremely important. Storyboarding the experience in various settings is the key to designing relevant gamified applications. In fact, applying the appropriate narratives concerning particular industries is a very interesting architectural initiative that can be pursued.
Thus, in the case of workplace engagement, if the nuances of the work and the industry were emulated around themes with contextual narratives, it would truly make for wonderful experiences that ignite employee engagement while furthering corporate objectives.
Linked Stars Blog 