Pepperdata Profiles: Five Years at the Cutting Edge of Big Data

Pepperdata Profiles: Five Years at the Cutting Edge of Big Data

As any good leader knows, a team is only as good as its people. And at Pepperdata, we have some of the very best. Our “Pepperdata Profiles” series shines a light on our talented individuals and explores employee experiences. This week, we talked to Shekhar Gupta, a veteran Pepperdata software engineer with a PhD in Artificial Intelligence and Big Data systems from the Delft University of Technology in the Netherlands.

Hi Shekhar! Before joining Pepperdata as a software engineer, what were you doing?

I was completing my PhD, actually. It was a remote PhD program from Delft University of Technology in the Netherlands, where I completed my master's. At the time, I also worked on a project in the Bay Area at the Xerox PARC Research Center, discovering ways to improve and maximize big data stacks.

What was your PhD research about?

My studies revolved around how to test Hadoop frameworks and find ways to improve the performance of Hadoop clusters and Spark workloads. Of course, my research aligned very well with Pepperdata! It was a big help when I came on board at the end of my PhD.

How did you hear about Pepperdata, and how did you end up joining the team?

I was a PhD student and working on my project at PARC when one of the interns told me that she was going to attend a meetup in Palo Alto about Hadoop performance. She encouraged me to come because the topic there was related to my PhD.

I went along, and then the Pepperdata co-founders, Sean and Chad, were presenting there. It was super aligned with the work I was doing. So I was quite interested in talking to them after that. We got chatting and had a great conversation about Hadoop, Hadoop versus Spark, and big data in general. They said I should join Pepperdata after completing my PhD! That's how this whole thing started. One great conversation and here I am.

How was the start of your time with Pepperdata?

I arrived on August 3, 2015. I joined right at the end of my PhD as a new engineer with no professional experience in computer science in general. I knew a ton about Hadoop performance optimization research, but I didn’t know anything production related. I never wrote production-grade code before. That's one thing I had to learn in the beginning: how to write code that is ready for production.

Nowadays, how do you approach your work at Pepperdata?

I try to see our products from our customers' perspectives. Because our code will be running in the data centers of customers, I have to create code that doesn't create problems in a customer environment. Some part of our software runs on the customer side, and we don't have much control once it's running.

I've been following very strict Pepperdata guidelines on how to write code, how to test it, how to create a test plan, and so on. It has to be so perfect that once the code is deployed, we don't see any problems.

It’s interesting that you mentioned following a guideline for testing everything perfectly. Achieving perfection is quite difficult in our industry. How do you strive to achieve just that?

That's something that is big here at Pepperdata. We have guidelines that go all the way down to the names of variables. How do you define a variable such that it's easy to understand what that variable is doing? It’s that granular.

We also have very, very strict code reviews. Every line of code that goes into production goes through the review. Once we do enough testing around it, then and only then does that code get deployed.

Tell us about what you value about working at Pepperdata.

There’s a lot that I value. But here’s one thing that’s great at Pepperdata: Regardless of how new you are or what your experience is, if you are trying to learn, and trying to come up with new ideas, people will listen. I love that.

Since the beginning, whenever we have problems, we vote and explore potential solutions. Everybody is allowed to share their ideas and designs. This helps us understand the thought processes that are going around. This is powerful because we deal with complex constraints such as scalability and performance, and we need different perspectives. When we have these, we are best equipped to make a system more robust, fault-tolerant, and scalable every time a problem comes in.

Even after five years as a software engineer, I’ve never had a boring day here. The software is stable and mature, but our customers constantly have requests, and we have to come up with really creative or innovative ideas to fix those problems.

What do you like about the Pepperdata company culture?

Number one, we have a great team. Really smart people from top to bottom. While things have changed in terms of upper management over five years, the style hasn't changed. The culture hasn't changed. Pepperdata has had an open culture since the beginning.

Ash Munshi, our CEO, drives this positive atmosphere. He always listens to us and answers our questions. We call him for any professional or personal issue. He always picks up his phone and talks to you. So it's really great. Team members are super collaborative, even though times are different right now. We all have been working from home, but we have been talking to each other all the time. And it's not just professional talk. I can always reach out to them.

People have this notion that working from home is quite difficult. Does such an arrangement hamper you in performing your job?

There is a notion that work from home becomes more exhausting and companies put extra pressure on you at home. That is not true here. We expect everybody to finish their things, but it is not exhausting. Nobody is getting pressured to work all the time or answer phones and messages when they're not supposed to.

If the upper management feels that somebody's exhausting themselves, they reach out and say, “okay, you need to take a break.” Everybody's learning right now about how to deal with this situation. It's great that we're getting really good support from people in the company.

And this is the norm. Three years ago, because of a family health issue, I had to be completely cut off from work for over two months. But not even a single day in that period did I feel pressured to worry about my work.

How has working at Pepperdata as a software engineer impacted you personally?

It has definitely made me a much better person in many ways. I can understand other people's concerns even when we are working with different people with different backgrounds and different cultures. I am now better able to empathize and be patient with misunderstandings. I learned this because the example was set for me. People were more patient with me, and they were more understanding with me. I saw the value of that.

Imagine someone wants advice for how to succeed as a new hire at Pepperdata. What advice would you give to him or her?

Be ready to learn new stuff. You should always have an interest in learning new things. Don't expect that somebody will give you instructions, and you do it, and that's that. Get used to quickly understanding odd and complicated problems. In the long run, it will make you very skilled. It may be scary, but you have people to help you. Always remember that you are not alone.

Be collaborative and share things easily. Collaboration is a big component at Pepperdata. There are skills you can learn, and everybody will help you with the learning process. However, your disposition has to be right. This goes for communication, too. If you have a question, even if it sounds dumb to you, the best thing to do is ask it right away. That's very important, especially when you're new.

The views expressed on this blog are those of the author and do not necessarily reflect the views of Pepperdata. Any solutions offered by the author are environment-specific and not part of the commercial solutions or support offered by Pepperdata.

Explore More

Looking for a safe, proven method to reduce waste and cost by up to 47% and maximize value for your cloud environment? Sign up now for a free cost optimization demo to learn how Pepperdata Capacity Optimizer can help you start saving immediately.