The insider's guide to the Jerome Fisher Program at the University of Pennsylvania

Beaches, Bitmojis, and Augmented Reality: My M&T Summer at Snapchat

M&T students have varied interests and use their time at Penn to explore them. One way to explore these interests is through internships. This past summer, M&T Daniel Moreno (’18) jumped at the chance to dive deeper into his interests with an internship at Snap Inc., creator of Snapchat.

Why Snap?

I applied to work at Snap Inc. because of the value they place on the power of creativity. I’ve been using Snapchat since high school and think that it’s a delightful, innovative platform. It moves away from the traditional paradigms of other social networks. Snap’s products allow you to share more authentic and spontaneous version of yourself than other platforms, which are more focused on the number of views and likes your posts gets.

I got connected with Snap recruiters while I was interning in San Francisco my sophomore summer and interviewed directly with their Camera Platform team after a recruiting event in New York City. My internship was in Venice Beach, which is a couple of minutes away from downtown Los Angeles in California.

At Snap, I became interested in understanding how they’ve been able to empower their users with creative tools such as stickers, Bitmoji (an extremely customizable animated avatar), face lenses and geofilters. I think that Snapchat and Spectacles (Snap’s sunglasses that allow you to record and share memories with your friends) are both products that bring users joy. I wanted to contribute to their development.


Augmenting Reality

Augmented reality (AR) is the process of merging of virtual objects, data, and experiences with the real world in order to create a composite “reality” for users. Good AR feels real, is interactive and can be shared with others easily. Computer vision technology and interactive engineering are essential to making this happen. The first deals with the algorithms and infrastructure to be able to detect and insert objects into the user’s field of view. The second allows users to affect the objects and respond to changes in their “augmented” world.

At Snap, I was an intern on the Camera Platform team and worked as both a computer vision and interaction engineer. I tackled a variety of augmented reality experiences that will launch in the coming months on the Snapchat app.

My main expectation this summer was to better understand how to develop and design experiences for AR. I see the adoption of AR as a reflection of a big change in the way humans and computers interact. Most of the designs and paradigms for traditional user interfaces don’t translate intuitively to 3D. Snapchat has the biggest crowd using AR on a daily basis. There is no current “right” or “wrong” way to design in AR yet, so I wanted to learn from the most creative people in the field.

I’m particularly happy about contributing to the launch of 3D Bitmoji this summer. The lenses that I worked on allow the user to place his or her Bitmoji in front of them in a variety of fun scenes. I worked on improving the simultaneous localization and mapping (SLAM) and surface tracking for those experiences, which are basically the computer vision algorithms that detect the surfaces that the Bitmoji can stand on. This technology is shared amongst other augmented reality lenses that Snapchat has launched in the past, such as the viral dancing hot dog that launched over the summer.

 

Blown Away by CA

The experience I had out in California was beyond words. From the very first day of my internship I started contributing to awesome projects using tech I was very interested in learning about and using. I worked with optical flow, image segmentation and other cool computer vision technologies to create fun experiences for Snapchat’s users.

I was absolutely blown away by the creative, welcoming and humble people working at Snapchat. I got involved in many product design conversations with my managers on AR interfaces that led to a lot of crazy prototypes and (some successful, some not) explorations.

Plus, I was able to work right on Venice Beach. As part of my work, I went to the beach and tested out the AR experiences I developed right on Venice boardwalk! That was just one of the perks to an all-around awesome experience.

 

Daniel is a computer and information science major concentrating in management and specializing in entrepreneurship and innovation. He is a member of the Class of 2018 and will be joining Snap Inc. as an Interactive Engineer after graduation. One thing Daniel loves about the M&T Program is that he will always have an on-campus home in the M&T Office. You can find out more about Daniel’s experiences at danmoreno.xyz.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Basic HTML is allowed. Your email address will not be published.

Subscribe to this comment feed via RSS

%d bloggers like this: