Using Big Data approaches to disaster response

One of the things I like best about technology is its use to solve problems that contribute to the improvement of society's quality of life in general. Yesterday, the U.S. National Science Foundation (NSF) and the Japan Science and Technology Agency (JST) announced joint support for 6 projects at universities in both countries that leverages Big Data and Data Analytics to transform disaster management for individuals and for society at-large. According the announce, when disaster strikes, it is critical that experts, decision makers and emergency personnel have access to real-time information in order to assess the situation and respond appropriately.

USC's spatial crowdsourcing platform, MediaQ, collects pictures and videos during disasters.
Credit: Cyrus Shahabi, USC via NSF

The researchers will look at how to capture and process data associated with disasters and improving the resilience and responsiveness of emerging computer systems and networks in the face of disasters to facilitate real-time data analytics in their aftermath. Jim Kurose, head of NSF's Computer and Information Science and Engineering Directorate said: "We're proud to collaborate with JST to address the global need for Big Data and data analysis for disaster management. Collaborative programs such as this one bring diverse perspectives and expertise to bear in mutually synergistic ways on critical problems that impact all of society. These are challenges that no single country can address in isolation."

The US-Japan Big Data and Disaster Research projects are:

Human-Centered Situation Awareness Platform for Disaster Response and Recovery
Researchers at the University of Southern California and the National Institute of Informatics in Japan will collaboratively design a computer platform that decision makers can use during disasters to analyze incoming data and coordinate responses.

Data-Driven Critical Information Exchange in Disaster-Affected Public-Private Networks 
Florida International University and University of Tokyo researchers will design context-aware and user-specific information delivery systems that could be deployed during disasters to supply accurate information to citizens.

Efficient and Scalable Collection, Analytics and Processing of Big Data for Disaster Applications
Researchers from the Missouri University of Science and Technology and Osaka University in Japan will collaborate to develop new methods to compress, transmit and query data from sensor networks.

Disaster Preparation and Response via Big Data Analysis and Robust Networking
Working together, researchers from Arizona State University and Japan's National Institute of Information will explore resilient networks, social media mining and information dissemination during disasters.

A Big Data Computational Laboratory for the Optimization of Olfactory Search Algorithms in Turbulent Environments
Researchers at Johns Hopkins University and the University of Tokyo will collaborate on the development of new olfactory search algorithms that use sensors to identify sources of pollutants or other agents released in the air or sea.

Dynamic Evolution of Smartphone-Based Emergency Communications Networks
Researchers from Temple University and the University of Aizu in Japan will collaborate to design smartphone-based ad hoc emergency networks that can evolve as a disaster unfolds.

The Sentient Enterprise

Oliver Ratzesberger gave a good lecture, entitled The Sentient Enterprise, published by Teradata in its YouTube channel. It's worth watching!

Ratzesberger presents the evolutionary journey of analytics capabilities that begins with today’s Agile Data Warehouse and culminates in the Sentient Enterprise. The Sentient Enterprise is an enterprise that can listen to data, conduct analysis and make autonomous decisions at massive scale in real-time. The Sentient Enterprise can listen to data to sense micro-trends. It can act as one organism without being impeded by information silos. It can make autonomous decisions with little or no human intervention. It is always evolving, with emergent intelligence that becomes progressively more sophisticated. Ratzesberger also presents a framework to assess your organization’s progress along this journey and “next practices” that business leaders can harness to unlock the full potential of Big Data and analytics.

Big Data in Healthcare: Hype and Hope on the Path to Personalized Medicine

I watched a video with an interesting panel on Big Data and Healthcare, recorded at the 2014 Digital Health Conference, entitled Big Data in Healthcare: Hype and Hope on the Path to Personalized Medicine, moderated by Bonnie Feldman.

As the author of “Big Data in Healthcare Hype and Hope”, Dr. Feldman has interviewed over 180 emerging tech and healthcare companies, always asking, “How can your new approach help patients?” Her research shows that data, as an enabling tool, has the power to give us critical new insights into not only what causes disease, but what comprises normal. Despite this promise, few patients have reaped the benefits of personalized medicine. The panel discuss the evolving health data ecosystem and how big data is being leveraged for research, discovery, clinical trials, genomics, and cancer care.

The Panel Members are:
- Bonnie Feldman, DDS, MBA - Chief Growth Officer, @DrBonnie360 (Moderator)
- Colin Hill - CEO, GNS Healthcare @GNSHealthcare
- Jonathan Hirsch - Founder & President, Syapse @JonathanHirsch @Syapse
- Andrew Kasarskis, PhD - Co-Director, Icahn Institute for Genomics; Multiscale Biology; Associate Professor, Genetics; Genomic Studies, Icaahn School of Medicine at Mt. Sinai @IcahnMountSinai
- William King - CEO, Zephyr Health @ZephyrHealth

10 Big Data Pros To Follow On Twitter

Last week, InformationWeek published an article with the 10 Big Data Pros To Follow On Twitter, written by Kevin Casey. I'm very honored to have been mentioned in the list.

According the article: "Twitter's kind of an ironic place to look for big data wisdom. It's an example of the ubiquitous services used by consumers and businesses alike that help generate this avalanche of data in the first place. Twitter has a valuable collection of big data knowledge -- if you know where to find it. Like other social platforms, Twitter can sometimes get noisy. Throw in a buzzword like "big data," and the noise can get downright cacophonous. So how do you find the information you want?"

He wrote a cool description on my Twitter's account: "Borba's active feed is a particularly good read if you're interested in how big data translates to bottom-line business -- in other words, making money. In addition to offering regular tweets on big data and analytics, Borba shares recommendations of other people to follow."

Dilbert on Big Data Brother

(Click on comic strip to view larger image)
Dilbert by Scott Adams-Dilbert ©2014, Universal Uclick
Published at May 11, 2014 in dilbert.com

Top Big Data Executives and Experts to Follow on Twitter

A few days ago, the CEO World Magazine published the Top Big Data Executives and Experts to Follow on Twitter. I'm very honored to have been selected in the list at #8, with so great people.

According to Amarendra Bhushan, Editorial Director of CEO World Magazine: "To recognize and learn more about big data technologies and architecture, vendor developments in the big data and analytics industry, and numerous technical challenges. I have decided to compile a list of the Most Influential Voices in Big Data and Quantitative Analytics Arena. And what they think are their biggest challenges when implementing big data in their storage environments."

5 Big Data myths debunked

 

Recently I read a great article entitled 5 Big Data myths debunked, where the author Sanne Steegstra, questions the main myths of Big Data clear and humorously. Below is the summary of the article:

Myth 1: Big Data is big
Nope. Big is relative. The ‘Big’ is marketing lingo, an attention grabber, as if we should automatically be afraid of scary unknown big things. Big actually means ‘more difficult to access and query than we are used to’. Yes, at some point you will probably need new tools. Most people and companies are not working with Big Data, just data.

Myth 2: Big Data is a technology thing
Wrong, it is a paradigm shift in business models. And up until the new models are a common thing, Big Data will keep its adjective ‘Big’. Don’t be fooled by the traditional soft- and hardware vendors that are the first ones to step up to sell you Big Data Solutions. This doesn’t mean it’s an IT thing, don’t confuse the message with the messenger. Big data is about acting smart, and right now about changing organizations to be smart, to have a competitive advantage with a better understanding and serving of customer needs.

Myth 3: There is an enormous shortage in analytical talent and experienced analysts.
A lot of companies are having a hard time recruiting the right analytically skilled people. The quest for analysts and data scientists is a hard one, with consultancy agencies acclaiming that message in advertorials everywhere. Although there can never be enough analytical talent, and yes there is a shortage, an important part of the problem lies somewhere else.

Your company is not interesting enough
Let’s be honest, your company’s corporate website might show pictures of young and good-looking people, working on an apparently fun business problem while pointing at a computer.. however in the non-Photostock reality you probably are a boring company that sells boring products and have boring problems to solve. So how on earth can you compete with interesting start-ups and cool, tech savvy companies? Well, just create interesting problems. Create multidisciplinary teams where analytical talent is not only (mis)used as support. Allow them to create the interesting problems.

You already hired them, they are working in the wrong department
And I bet it is the IT department.. All your STEM ( Science, Technology, Engineering, Mathematics) skills are in one place. Nicely hidden away on a separate floor, or in some sort of ‘incompany quarantine’. So besides a big mentality- and organizational change, there are two things you can do. You either teach your marketing staff analytical skills, or you teach your analytical staff marketing skills.

Data Analytics still is unnecessarily complex
A data analyst loves to analyze data, not the hardship of accessing the data. Are programming skills required to create cool tools, models and applications? Or are they an absolute necessity because otherwise 90% of the time would be filled by meeting with the IT department? Data analysts work on the frontiers of data. That means the data is by definition not structural, seldom relational and hardly quickly accessible. The company providing a plug and play like sandbox solution for all company data will leap an important part of the analytical gap.

Myth 4: Big Data is social data
Social is data’s super sexy showcase. It will often start with social data, not only because it (still) is up for grabs and there is a lot of it, but also because everything with a like button on it appeals to marketers, creative companies and more than the usual suspects (yep IT, BI and CRM, I mean you guys). So don’t forget to have a focus on your own ‘big' data. Got a central cash register system, a website with a large volume of clicks and views (and it’s up to you if you decide to label it large or ‘big’) or even better, have check-ins, sensor data or some other sort of activity generated data? Even better! That’s your big data!

 

Myth 5: Big Data is a hype
Stop the definition debate. Who cares. Everybody agrees on the possibilities and disruptive force data can have. The era of data has already begun.

Kenneth Cukier on Big Data: The data revolution

Kenneth Cukier, the Data Editor of The Economist, gave a good interview about his new book and how our unprecedented access to data changes how we live and think. Watch and enjoy!

Why smart statistics are the key to fighting crime

I watched at TED a good lecture on the use of analytics to fighting crime, by Anne Milgram. an american attorney. In the lecture, Anne Milgram explains that when she became the attorney general of New Jersey in 2007, she quickly discovered a few startling facts: not only did her team not really know who they were putting in jail, but they had no way of understanding if their decisions were actually making the public safer. And so began her ongoing, inspirational quest to bring data analytics and statistical analysis to the US criminal justice system.

Anne Milgram said that decided to focus on using data and analytics to help make the most critical decision in public safety, and that decision is the determination of whether, when someone has been arrested, whether they pose a risk to public safety and should be detained, or whether they don't pose a risk to public safety and should be released. Everything that happens in criminal cases comes out of this one decision. It impacts everything. It impacts sentencing. It impacts whether someone gets drug treatment. It impacts crime and violence.

So she went out and built a phenomenal team of data scientists and researchers and statisticians to build a universal risk assessment tool, so that every single judge in the United States of America can have an objective, scientific measure of risk. In the tool that they've built, what they did was they collected 1.5 million cases from all around the United States, from cities, from counties, from every single state in the country, the federal districts. Their goal, is that every single judge in the United States will use a data-driven risk tool within the next five years. She finished with a statement: "Some people call it data science. I call it moneyballing criminal justice."

TED link: Why smart statistics are the key to fighting crime

Dilbert on Big Data Analysis

(Click on comic strip to view larger image)
Dilbert by Scott Adams-Dilbert ©2014, Universal Uclick
Published at Jan 27, 2014 in dilbert.com

Data Science: Not Just For Big Data

These days, Data Science and Big Data have become synonymous phrases. But data doesn't have to be big for data science to unlock big value. Kalido hosted a webinar intitled Data Science: Not Just for Big Data, with Darren Peirce, Kalido CTO, David Smith, Data Scientist at Revolution Analytics and Gregory Piatetsky-Shapiro, Editor, KDnuggets. They discussed what changes in Data Science with Big Data, what remains the same, and suggested ways for getting best value from data regardless of the size. Watch and enjoy!

The marriage of Big Data and Data Warehousing

IBM published a good two part video series, where Big Data evangelist James Kobielus discusses the marriage of Big Data and Data Warehousing that is leading us toward the Hadoop Data Warehouse. In part 1 of this 2 part video, James defines Big Data and discusses some use cases.

In the second part of this video series, James discusses the attributes of a Big Data platform that encompasses traditional data warehousing, big data analytics at rest and in motion, the importance of MDM and governance and what you should be looking for from your Big Data platform vendor.

Converting Big Data and Analytics Insights Into Results

The IBM Institute for Business Value published today an executive report entitled Analytics: A blueprint for value - Converting big data and analytics insights into results, written by Fred Balboni, Glenn Finch, Cathy Rodenbeck Reese and Rebecca Shockley. According the study, in today’s competitive marketplace, executive leaders are racing to convert data-driven insights into meaningful results. Successful leaders are infusing analytics throughout their organizations to drive smarter decisions, enable faster actions and optimize outcomes. These are among the key findings from the 2013 IBM Institute for Business Value research study on how organizations around the globe are leveraging key capabilities to amplify their ability to create value from big data and analytics.

Through the research, they identified nine levers that enable organizations to create value from an ever-growing volume of data from a variety of sources – value that results from insights derived and actions taken at every level of the organization.

• Culture: Availability and use of data and analytics within an organization
• Data: Structure and formality of the organization’s data governance process and the security of its data
• Expertise: Development of and access to data management and analytic skills and capabilities
• Funding: Financial rigor in the analytics funding process
• Measurement: Evaluating the impact on business outcomes
• Platform: Integrated capabilities delivered by hardware and software
• Source of value: Actions and decisions that generate results
• Sponsorship: Executive support and involvement
• Trust: Organizational confidence

The research also identified three levels of value impact among the nine levers: Enable levers form the basis for big data and analytics; Drive levers are needed to realize value; and Amplify levers boost value creation.

That is an interesting and comprehensive look at organizational activities related to data and analytics. You can download the full IBM Institute for Business Value Study in (Registration required): http://ibm.co/9levers