Student Media Body of IIT Kgp


Jewels Of Kgp: Parag Havaldar

Posted on Feb 22, 2016

In the series of 'Jewels of KGP', this time we bring you the one who has surely raised the flag of IIT Kharagpur to tremendous heights. He is none other than Mr. Parag Havaldar. He is honored with the Technical Achievement Award for the development of expression-based facial performance-capture technology at Sony Pictures Imageworks. He had done his B.Tech. in computer science and engineering in 1991 from IIT Kharagpur and went on to do a Ph.D. in computer vision and graphics from the University of Southern California in 1996. Currently, he is a software supervisor at Sony Pictures Imageworks, where he leads the company's proprietary efforts in the area of performance capture.

The technology developed by him and his group have been used to create stylized and realistic character animations in a variety of movies including "Alice in Wonderland", "Monster House", "Hancock" and "Spiderman".

Let’s have a look at what Parag has to say about his Journey from a Kgpian to Technical Achievement Awardee.

How did your dreams and aspirations change over the period of time? When did you find yourself interested in the field of multimedia and image processing?

I am sure you agree that dreams and aspirations change over time depending on experiences you go through and people you meet and get inspired by. As a young child, I wanted to join the medical field, but some of my experiences with math and science changed that to attending an IIT. I don’t know when I got interested specifically in the visual fields of computer graphics, computer vision, images, videos and multimedia in general. I always enjoyed geometric and spatial reasoning, and luckily my undergraduate project presented an opportunity to learn a lot about processing digital images, which you never got as an undergrad, and that led me to study computer vision at the University of Southern California. The professors I met had research interests in imaging and vision – from the low level signal processing aspects to high level cognitive brain models of human and computer vision. Being in the Los Angeles area, I was at the right place for entertainment and at the right time where digital technologies were poised to make a big influence on entertainment. So you never know what opportunities come your way, who ends up inspiring you. Having aspirations and a desire to do something is the very first step, one only hopes that you are also fortunate to be surrounded by mentors to guide you and opportunities to explore.

When did the thought of receiving the “TECHNICAL OSCAR” cross your mind? How is the feeling of being on the receiving end of it?

When a large and influential organization like the Academy of Motion Pictures gives one an award such as this, for developing a technology that has influenced moviemaking, one has no answers. I feel very humbled and grateful. At the same time, I do feel overjoyed that so many years of work was recognized at this level but I also feel tremendously responsible – with a need to advise and influence younger generations in a practical way so that they make informed decisions in their scholastic and professional endeavors. That said, I don’t think you can ever work towards such an award, or for that matter any recognition. When you are engrossed in any work, you are always caught up in that moment to solve problems. An award of any kind was never in my mind. But I also need to express that some of my other work in a different area was investigated/nominated by the Academy for a potential award in 2011, only that the Academy decided that the technology was premature and not influential enough. So you never know!

What role did IIT KGP play in your overall development as an individual? Share some of your tough phases and how did you overcome those?

IIT KGP has played a very formative role for me, it has done so for all the wonderful friends I made there and it will for all of you. We leave the comfort of a home at a very pristine age, ambitious but foolhardy, to start a special phase in our young lives – that of education at KGP. Our premature ideals and inchoate goals are bound to take a sail in such an environment by sharing the same with friends that we end up making. I grew up as a kid in various places – Pune, Ahmedabad, Ndola, Lusaka, Livingstone, Nairobi, Blantyre, Arusha, Kampala. So learning different languages and peer assimilation was always a struggle. You learn very quickly that you have to be good at something to gain respect. But in a place like KGP, it is hard to be good at something - the selective nature of the process ensures smart, well read, talented individuals. My seniors were charismatic and embodied leadership qualities that we wanted to emulate. They were always a great source of inspiration. I took part in sports, cultural events and learned to speak Bengali in a few months. Comparing notes afterward with my friends from other IITs, I can conclude that IIT KGP was very different, because it was not a city and nowhere close to another big city when I embraced it. This might have changed now, but during my times most students stayed back on campus all weekends. The forced togetherness created an ecosystem for spending more time together leading to exchange of ideas, development of ideals and friendships - that played a role in sculpting our youth and consequently laying a pathway for our future. So, conclusively and decisively I can say that KGP has its own benefits.

How was your stay at the University of Southern California as a Ph.D. student? How challenging do you find being a part time faculty there now?

USC was great and in fact, my experiences at KGP prepared me well for it. After the rigor that the computer science department at KGP subjected us to, you are ready to learn more. The research exposure at USC was a breath of fresh air. Of campus life was easier to deal with in terms of roommates, friendships, cultures etc. because you had already met and dealt with all the possible personalities while growing up in KGP. The research groups and faculty with very progressive, plus the proximity to the industry that you get in a large city like Los Angeles has an added benefit and exposure. The peers I graduated with at USC have become fairly influential in their own right, including the current chairperson of the department.

Now, as a part time faculty at USC, I enjoy the experience thoroughly because it has allowed me to give back to the community with a unique perspective of having worked in the industry. It was an uncanny experience when I began teaching at USC because I became a faculty member few years after I graduated. So I had not lost my student perspective and could relate to all students, especially international students. It was a challenge to not act like a student amongst them. But over the years, you meet so many fresh and young bright green minds, who never stop challenging your own learning. That is what I love about graduate research schools– the curiosity that young smart minds bring to the environment.

Kindly share with us how it felt to lead the crew of performance capture at “SONY Pictures Imageworks”

Looking back, it was a wonderfully unique experience. The movie industry is all about creating pixels on the screen, held to a very high standard of storytelling. Technology is always great, but the movie industry is a creative endeavor and it was always important to understand the impact that computational algorithms and automation can have on a creative client base. The clients we had, especially director Robert Zemeckis, Spielberg were well aware of the technology but in the end, it does not matter if the technology does not deliver in the creation of the final art form. I led a small but very talented group of six to ten computer visionaries who had to be all conscious of not only the advantage but also the limits of technology, under production pressures. Just because your algorithm works, does not mean a movie production of an army of artists will use it if they cannot control it to get what they want. It was more about our team understanding artist workflows and artistically controlled outputs. In the end, the greatest sense of accomplishment for me was leading engineers to think about the synergy between technology and art!

How is your current work with “Blizzard Entertainment” as R&D lead going on? What advances can we expect in this field in near future?

At Blizzard, I lead an R&D group that model creates and animates digital characters, but for the gaming industry. Blizzard Entertainment, as part of Activision-Blizzard, is a big computer gaming company with some high profile multiplayer games such as World of Warcraft, Overwatch, Starcraft, Hearthstone. With faster computing platforms, such as GPUs. with dedicated gaming hardware (PlayStation, Xbox), and new interface paradigms such as Virtual Reality, computer gaming is poised to transition from what it is right now to a more character-centric and emotionally engaging game experience. Interaction with digital characters will be a big part of this change. Also with the confluence of multi-disciplinary progress in the computer sciences such as natural language processing, audio analysis, and synthesis, cognitive computational models we are bound to see intelligent digital characters. This might lead to a transformation in entertainment from a passive "story telling" experience as we witness with movies, to a possibly active and even interactive "story defining" experience. Computer gaming technologies that advance digital characters at their helm will be at the forefront of such an evolutionary entertainment paradigm. At that time, you perhaps won't be sitting in Netaji and shouting "Tarapodo lights!"

“Alice in Wonderland”, “Spiderman”, “Hancock” and many more, all were truly a masterpiece. Which one is your personal favourite and why?

It is hard to define any one project as a favorite. The creative/artistic needs of each project are different, requiring different technological advances. My most memorable projects have been ones in which I was involved in research from the very beginning, went onto production sets and interacted with the creative side people to understand requirements of technology, or see first hand how well (or not so well) our technology worked. For instance, creating virtual actors on Beowulf with Angelina Jolie, Hancock with Will Smith was very challenging because digital doubles of actors such as Jolie and Smith need to animate and remote with immaculate accuracy on screens to sell a performance as “real” to the human eye. Automation in this regard was really really hard! Monster House was a different challenge because we had to capture child actors and transfer their actions onto digital stylized/cartoony characters creating a very nuanced animation. I also enjoyed leading a team of computer scientist and artists to help create stereo movies. The problem of creating stereoscopic (two) images captured from single (one) vantage viewpoint of the film camera (or any camera) is still not a completely solved problem even in research and academia. At SONY we have always enjoyed interacting with great visionary directors like Robert Zemeckis on Beowulf, Tim Burton on Alice in Wonderland, Sam Raimi and Marc Webb on the Spiderman stories and their projects have always needed solutions which were a privilege to address.

What suggestions would you like to give a student interested in “computer vision and graphics”?

I hope not to get too technical! Firstly, understand the difference between computer vision and computer graphics. Computer vision deals with real visual (images/video) signals acquired by cameras as input to computationally extract intelligent descriptions – metrically accurate for engineering purposes as well as analytically sound to derive models of human perception. For instance, you might reconstruct the world around you seen via photographs - to the very millimeter, or/and you could describe intelligently what your signal has sensed – who is there in the image, what are they doing, Computer graphics aims to take accurate descriptions (such as those that computer vision is trying to extract) and produce realism in rendered images – … broadly speaking both processes are the exact reverse of each other. And this entire area has greatly changed since I was inspired by that problem space as a student. It has now spawned into a variety very insightful and much needed research and industry explorations. Today, there is a lot of research on mapping the human brain, and how the human visual system works, but computational models that describe these aspects correctly are still needed. Machine learning, especially deep learning is at the confluence of solving a lot of problems today but these solutions are still very typified to solve specific problems and not unified. The field is poised to make influential progress in the coming years. Computer Graphics – a lot of progress in terms of modeling, shading, animation and rendering to make virtual worlds seem very real. But there is a problem of scale – to create content for movies, games, virtual reality still, takes a lot of human labor. There need to be algorithms, tools, and pipelines that can democratize this effort. – still largely perplexing. In short, these related fields of vision and graphics, (like all other interesting areas!) have more questions than answers. If you have intriguing and purposeful questions after initial research, I would love and welcome conversations.

Is there any possibility of you taking out some time from your packed curriculum and visiting your alma mater?

Of course, I would love to. I will try to do so next time I come to India and hope KGP gives me a chance to share my inspiration and research with the people on campus. For so many of us, the journey started there.

We would be delighted if you can share any photo or video from the time of your stay in IIT KGP.

Sure here are a few:

Parag Havaldar

A Trip to Digha Beach – organized by our friends from SN Hall at the end of the first year. Very memorable because we were asked by local authorities to leave our hotel and Digha at 3:00 am because of hajaar bamaal.

Parag Havaldar

A Dance performance that we won at Spring Fest repurposed for senior send off from LLR Hall. Left – Sanjay Verma (1989), center yours truly (1991), right Satyajeet Dodia (1990). I don’t think acid wash jeans are popular now.

Finally, any message for all the present students of IIT KGP from your side? A word of motivation can inspire them a lot.

Winning this award has certainly given a lot of attention to my work and me. Many, many thanks for all the wishes from my KGP friends, professors, my seniors, juniors and all alumni, who have reached out to me on social media. It feels great to get this respect, but I also need to put it in some perspective. This was many years of work, and perhaps fierce competition, but there are lots of people who continue to work and contribute towards more noble causes. Their work is not glamorized as done by an Academy Award and it should be! For instance, our professors, researchers, scientists, philanthropists etc are an inspiration with their dedication and quality of work. A common trait among all such inspirers is that they always stay curious and not lost hope in their desire to make a difference. So - stay curious, be passionate about your work and you will be successful professionals poised to make a difference somewhere! Finally- KGP ka tempo … (please shout out in unison – HIGH HAI!)

Share this article on:

Related Articles

Know Your Entrepreneur : Arjun Malhotra

Posted on: Dec 13, 2018

Jewels Of KGP: Mr. Shrey Goyal

Posted on: Sept 07, 2018

Disha Seema Centre School: A KGPian's classic tale of selfless service

Posted on: August 20, 2018

Jewels of KGP: Dr. Sanghamitra Bandyopadhyay

Posted on: Apr 01, 2018

Survey Analysis: A Report on B C Roy Technology Hospital

Posted on: March 27, 2018

Proposal Update: Mr. Mayank Srivastava, Vice President, TSG

Posted on: March 26, 2018

Jewels Of KGP: Mr. Siddartha Khastgir

Posted on: March 15, 2018

Need Of Basic Amenities

Posted on: March 13, 2018

Lean In, IIT Kharagpur

Posted on: March 8, 2018

Jewels of KGP: Mr. Shailendra Kumar

Posted on: Feb 24, 2018

Jewels of KGP: Dr. Harish Hande

Posted on: Nov 11, 2017

Awaaz : Do's & don'ts for Department Change

Posted on: Nov 5, 2017

National Digital Library: A gift from IITKGP

Posted on: October 15, 2017

Know Your Professor: Parama Barai

Posted on: Oct 12, 2017

Jewels of KGP : Mr. Lakshya Jain

Posted on: October 6th, 2017

Happiness Project: What Made Me

Posted on: August 31, 2017

Know Your Professor: Debalay Chakrabarti

Posted on: Aug 28, 2017

Jewels of KGP : Mr. Sikhar Patranabis

Posted on: August 08, 2017

The Synthetic Track Fiasco

Posted on: August 08, 2017

Know Your Professor: Joy Sen

Posted on: May 8, 2017

NSO H&F and its Relevance

Posted on: April 18, 2017

There's more than meets the eye

Posted on: April 15, 2017

Know Your Professor: Suman Chakraborty

Posted on: April 13, 2017

GYWS: A Big Step Towards Transformation

Posted on: April 10, 2017

Campus Politics at IITs

Posted on: April 02, 2017

Artificial Intelligence: Motivation, Implications & Application

Posted on: April 01, 2017

Will IIT KGP Ever Go Cashless?

Posted on: March 28, 2017

What If I had Turned Off the Switches?

Posted on: 23 March, 2017

An Interview with Commander V.K Jaitley

Posted on: March 8, 2017

Shedding Light on a Dark Subject!

Posted on: Mar 05, 2017

Jewels Of Kgp : Parag Havaldar

Posted on: Feb 22, 2016

NSS: Is it Relevant or Not?

Posted on: Feb 9, 2017

Jewels of Kgp : Pratish Datta

Posted on: Feb 1, 2017

Know Your Professor: Navtej Singh

Posted on: Jan 30, 2017

Know Your Professor : Somesh Kumar

Posted on: Jan 24, 2017

Know Your Professor : Asish Ganguly

Posted on: Jan 9, 2017

Return of The Fee Hike

Posted on: Dec 15, 2016

Human Rights : 'Right to Information'

Posted on: Nov 12, 2016

Learn Earn Return

Posted on: Nov 4, 2016

Science of Happiness

Posted on: Nov 1, 2016

Illumination: The Light in Kgp's Night

Posted on: Oct 29, 2016

Jewels of Kgp : Madhav Datt

Posted on: Oct 28, 2016

Foreign University Training: A Beginner's Guide

Posted on: Oct 22, 2016

CV Module: Complex Mess or Uniformly reliable?

Posted on: 19 August , 2016

Information Portal

Posted on: July 26, 2016

Election "tab" aur "ab"

Posted on: April 3, 2016

How Safe are you ?

Posted on: March 27, 2016

A Big Mess in the 'Mess'

Posted on: March 06, 2016

Placements(Phase 1) : 2015-16

Posted on: February 07, 2016

Guest Lecture: Shoojit Sircar

Posted on: January 28, 2016

Critical Analysis of QS World Rankings

Posted on: November 10,2015

Copyright © 2018 - All Rights Reserved - Awaaz, Indian Institute Of Technology, Kharagpur