Chiranjib Bhattacharyya and Jayant Haritsa

• Report rights infringement
• published: 01 Sep 2014
• views: 1904

On May 23, 2014, we had Prof. Chiranjib Bhattacharyya and Prof. Jayant Haritsa talk about big data from the perspective of statistics and software engineering respectively. Prof. Bhattacharyya's talk, "Learning from Big Data: Using Statistics to tame the Complexity" highlighted recent research showing that in settings where the bigness of the data seems to overwhelm our computational capabilities, understanding of the statistical properties of the data may help. He described how this idea works in solving classification problems. Prof. Haritsa gave an entertaining talk about the underbelly of big data analytics: software testing. Designing algorithms and software are exciting research activities, but at the end of the day, we need to have tools that work. And there are often cases when software works for small input sets but fail dramatically when the input consists of Big Data. How do we avoid such catastrophes without requiring the testing phase itself to require as much time as the actual deployment? Prof. Haritsa presented one solution, the data-less database, that is currently in use worldwide. ------------------------------- Learning from Big Data: Using Statistics to tame the Complexity Abstract: The problem of learning Statistical models from Data can be posed as Optimization programs. These programs often become unwieldy, in the Big Data setting, as the number of variables and constraints grow with number of data-points. Distributed Optimization, requiring expensive parallel hardware, is the current state of the art remedy for such problems. However, in Statistics growth of data points is often welcomed as it yields more understanding.This then begs the question: Are there alternatives to distributed processing where statistical understanding, gleaned from large volumes of data, can be used for taming the computational complexity of optimisation programs? Following this paradigm we present two ideas for solving classification problems: the first involving resampling constraints and the second involving chance constraint programming. Time permitting we will show how these ideas can be leveraged to build large scale focussed crawlers. Speaker Bio: Chiranjib Bhattacharyya is an Associate Professor in the Department of Computer Science and Automation, Indian Institute of Science. He is interested in Robust Optimization and Machine Learning. Prior to joining the Department he was a postdoctoral fellow at UC Berkeley. He holds BE and ME degrees, both in Electrical Engineering, from Jadavpur University and the Indian Institute of Science, respectively, and completed his PhD from the Department of Computer Science and Automation, Indian Institute of Science. ------------------------------- Big Data, Small Testing? Abstract: Big Data has become the buzzword of choice in recent times, especially in the software industry. The accompanying hoopla has spawned frenetic claims foretelling the development of great and wondrous solutions to Big Data challenges. However, there is very little said about the testing of such systems, an essential pre-requisite for deployment. In this talk, we will discuss the research challenges involved in the testing process, especially from the database perspective. We will also present CODD, a graphical tool that takes a first step towards the effective testing of Big Data deployments through a new metaphor of "data-less databases". CODD is currently in use at industrial and academic institutions worldwide. Speaker Bio: Jayant Haritsa is on the faculty of the Supercomputer Education & Research Centre and the Department of Computer Science & Automation at the Indian Institute of Science, Bangalore, since 1992. He received a BTech degree from the Indian Institute of Technology (Madras), and a PhD degree from the University of Wisconsin (Madison). His research interests are in database system design, analysis and testing.