According to Amarendra Bhushan, Editorial Director of CEO World Magazine, "To recognize and learn more about big data technologies and architecture, vendor developments in the big data and analytics industry, and numerous technical challenges. I have decided to compile a list of the Most Influential Voices in Big Data and Quantitative Analytics Arena. And what they think are their biggest challenges when implementing big data in their storage environments."
Myth 1: Big Data is big
Nope. Big is relative. The ‘Big’ is marketing lingo, an attention grabber, as if we should automatically be afraid of scary unknown big things. Big actually means ‘more difficult to access and query than we are used to’. Yes, at some point you will probably need new tools. Most people and companies are not working with Big Data, just data.
Myth 2: Big Data is a technology thing
Wrong, it is a paradigm shift in business models. And up until the new models are a common thing, Big Data will keep its adjective ‘Big’. Don’t be fooled by the traditional soft- and hardware vendors that are the first ones to step up to sell you Big Data Solutions. This doesn’t mean it’s an IT thing, don’t confuse the message with the messenger. Big data is about acting smart, and right now about changing organizations to be smart, to have a competitive advantage with a better understanding and serving of customer needs.
Myth 3: There is an enormous shortage in analytical talent and experienced analysts.
A lot of companies are having a hard time recruiting the right analytically skilled people. The quest for analysts and data scientists is a hard one, with consultancy agencies acclaiming that message in advertorials everywhere. Although there can never be enough analytical talent, and yes there is a shortage, an important part of the problem lies somewhere else.
Your company is not interesting enough
Let’s be honest, your company’s corporate website might show pictures of young and good-looking people, working on an apparently fun business problem while pointing at a computer.. however in the non-Photostock reality you probably are a boring company that sells boring products and have boring problems to solve. So how on earth can you compete with interesting start-ups and cool, tech savvy companies? Well, just create interesting problems. Create multidisciplinary teams where analytical talent is not only (mis)used as support. Allow them to create the interesting problems.
You already hired them, they are working in the wrong department
And I bet it is the IT department.. All your STEM ( Science, Technology, Engineering, Mathematics) skills are in one place. Nicely hidden away on a separate floor, or in some sort of ‘incompany quarantine’. So besides a big mentality- and organizational change, there are two things you can do. You either teach your marketing staff analytical skills, or you teach your analytical staff marketing skills.
Data Analytics still is unnecessarily complex
A data analyst loves to analyze data, not the hardship of accessing the data. Are programming skills required to create cool tools, models and applications? Or are they an absolute necessity because otherwise 90% of the time would be filled by meeting with the IT department? Data analysts work on the frontiers of data. That means the data is by definition not structural, seldom relational and hardly quickly accessible. The company providing a plug and play like sandbox solution for all company data will leap an important part of the analytical gap.
Myth 4: Big Data is social data
Social is data’s super sexy showcase. It will often start with social data, not only because it (still) is up for grabs and there is a lot of it, but also because everything with a like button on it appeals to marketers, creative companies and more than the usual suspects (yep IT, BI and CRM, I mean you guys). So don’t forget to have a focus on your own ‘big' data. Got a central cash register system, a website with a large volume of clicks and views (and it’s up to you if you decide to label it large or ‘big’) or even better, have check-ins, sensor data or some other sort of activity generated data? Even better! That’s your big data!
Myth 5: Big Data is a hype
Stop the definition debate. Who cares. Everybody agrees on the possibilities and disruptive force data can have. The era of data has already begun.
I watched at TED a good lecture on the use of analytics to fighting crime, by Anne Milgram. an american attorney. In the lecture, Anne Milgram explains that when she became the attorney general of New Jersey in 2007, she quickly discovered a few startling facts: not only did her team not really know who they were putting in jail, but they had no way of understanding if their decisions were actually making the public safer. And so began her ongoing, inspirational quest to bring data analytics and statistical analysis to the US criminal justice system.
Anne Milgram said that decided to focus on using data and analyticsto help make the most critical decisionin public safety,and that decision is the determinationof whether, when someone has been arrested,whether they pose a risk to public safetyand should be detained,or whether they don't pose a risk to public safetyand should be released.Everything that happens in criminal casescomes out of this one decision.It impacts everything.It impacts sentencing.It impacts whether someone gets drug treatment.It impacts crime and violence.
So she went out and built a phenomenal team of data scientists and researchersand statisticiansto build a universal risk assessment tool,so that every single judge in the United States of Americacan have an objective, scientific measure of risk.In the tool that they've built,what they did was they collected 1.5 million casesfrom all around the United States,from cities, from counties,from every single state in the country,the federal districts. Their goal, is that every single judgein the United States will use a data-driven risk toolwithin the next five years. She finished with a statement: "Some people call it data science.I call it moneyballing criminal justice."
TED link: Why smart statistics are the key to fighting crime