DSAA’2015 Trends & Controversies Invited Talks

The goal of exploratory data analysis or data mining is making sense of data. We develop theory and algorithms that help us understand our data, with the goal that this helps formulating better hypotheses. The role of statisticians is to provide methods that give detailed insight in how data is structured: characterising distributions in easily understandable terms, showing the most informative patterns, associations, correlations, etc.Statisticians are part of the big data science wave but which part exactly next to data accessibility, data communication and visualization?

While in the past the computer vision community focused on tasks considered trivial for humans, e.g. distinguishing cats from dogs, it has started very recently to focus on tasks which are non-trivial even for humans, such as recognising breeds of cats or dogs. These are two examples of fine-grained recognition tasks, i.e. tasks which involve a large number of semantically related and visually similar classes. Other examples of fine-grained tasks include the recognition of plants and flora or the recognition of brands and models of cars. The visual distinctions between similar categories are often quite subtle and therefore difficult to address with general-purpose object recognition machinery. It is likely that radical re-thinking of some of the matching and learning algorithms and models that are currently used for visual recognition will be needed to approach fine-grained categorisation. I will briefly discuss recent progress in this field (including some of my work on the topic) and mention some of the challenges still ahead of us

The distinction between ‘traditional business intelligence’ vs ‘big data analytics’ is often portrayed, typically by IT departments, as a case technology evolution from relational data warehouses to data-lakes based on distributed frameworks such as Hadoop. However, this distinction is superficial at best and distracts from the real issues, which are both scientifically deeper as well as cultural.

Regular polls demonstrate that apart from the relatively few web companies, the volume of data actually being analyzed in practice hardly warrants a technology change on its own; i.e., ‘big’ data is not really that ‘long’, and distributed processing is usually not essential, especially as Moore’s law progresses.

Instead, it is the increasing availability of many more pieces of information about each entity of interest, e.g., a customer, often from diverse sources (social-media, mobility, internet-of-things) that results in a ‘big data’ issue; i.e., ‘big’ data is more about ‘wide’ rather than ‘long’ data.

For one, queries are often an inefficient and inaccurate mechanism to derive insight from high-dimensional data. Second, disparate datasets often lack a natural ‘join key’. Third, datasets may describe measures at different levels of granularity, e.g., individual vs. aggregate data, and finally, different datasets may be gathered from physically distinct populations.

Thus, once data has been ingested in ‘raw’ form in a ‘data lake’, rather than mere distributed execution of traditional ETL operations, what is really needed is a systematic mechanism to address the challenges of diverse datasets.

I shall argue Bayesian graphical models provide a useful rudder with which to navigate a data lake and fuse disparate islands of data. Last but not least, a Bayesian paradigm also promises the ability to enable causal analysis that goes beyond querying the available data or its distribution.

While the cultural gulf between SQL-touting business intelligence technologists and Bayesian modelers may appear vast, it need not be so, if the big-data frameworks of the future focus on bringing together these different world-views.

Open source technologies in the Big Data space are maturing day by day with powerful new features around scalability, performance and security that are a must for enterprise class solution deployments. Although there is a plethora of development around open source in Big Data and Analytics space, enterprises continue to face significant challenges in adopting these open source components. The challenge often lies not in the quality of platform offerings, but in its lack of sufficient tooling around packaging, development, administration and monitoring, and a lack of effective integration into organizational processes. Learn about how Infosys envisages an open source platform with the best of big data and analytics capabilities, running on commodity hardware that can address a diverse set of enterprise use cases across various industries like retail, financial, logistics, healthcare, manufacturing, etc. , thus making it significantly easier for large enterprises to adopt these open source capabilities while accelerating their Data-to-Insights-to-action journey.

In March 2015, the first-ever Data Science Bowl was won by a team from Ghent University who developed the most effective algorithm to automatically classify more than 100,000 underwater images of plankton, marking a major step forward for the marine research community, as plankton populations are key indicators of ocean health. The competition, co-sponsored by Booz Allen Hamilton and Kaggle and created with data from oceanographers at Oregon State University Hatfield Marine Science Center, challenged participants to create an algorithm that automates an ocean health assessment process, which would have taken marine researchers more than two lifetimes to manually complete. After 90 days of intense competition, in which the data science community applied its skills to effect change on a global scale, the team of deep learning specialists had won in inaugural challenge. In this talk, Jonas Degrave, a member of the winning team, will discuss the deep learning approach they used to win.