Iron Man, Holograms and a Hackathon at MIT

When I think back just a few years ago, almost nobody knew what I was doing in my job as Development Manager for Head Up Displays. When people asked me what head up displays (HUD) were, I started talking about Marvel’s Iron Man. Everyone knows the smart super hero in his red and gold weaponized suit, and, well, Tony Stark views the world through an immersive head up display within his helmet. It is augmenting his field of vision with important information on his enemies, suit status, a map and more.

"Today, AR, VR and XR have become quite common technologies not only in the entertainment industry, but also in manufacturing, service, healthcare, retail or even the military." René Müller

This is basically augmented reality (AR): interacting with virtual elements that are displayed in the real-world environment in real time. In contrast, virtual reality (VR) is an artificial environment that is created with software to make the user believe it’s a real environment. It takes a special headset to experience VR that blocks the user’s natural surrounding. Then again, there’s cross reality, also called X reality (XR).

It uses different hardware and software and aims to be applicable across platforms and realities. Today, AR, VR and XR have become quite common technologies not only in the entertainment industry, but also in manufacturing, service, healthcare, retail or even the military. No matter if it’s elevator service technicians literally seeing technical and expert information while repairing the machine on-site, employee onboarding at the world’s largest retailer, virtual fitting rooms and smart mirrors that interact with the customer or a scanner that projects the map of veins and valves on the patient’s body.

The use cases of virtual, cross and augmented reality are numerous, and overall pretty cool as they are leveraging the potential of future technology in various industries to a whole new level.

Prototyping and training with virtual and augmented reality

Of course, that holds true for the automotive industry as well. At Porsche, we have discovered the potential of new technologies for our training and service as well as the customer experience quite some time ago and are constantly working on improving these areas.

For example, virtual reality and our drone “Alice” are helping our after-sales employees worldwide to get to grips with complex technical concepts. Alice guides the mechanics through all the repair steps on the high-voltage battery of our Panamera 4 E-Hybrid. Under the supervision of Alice, each step can be done without any risk, because it takes place in virtual reality — a huge advantage that makes even the most complex training safe for our staff.

Another digital innovation is the Power Wall that displays a detailed 3D model of a car that may be still in development. The Power Wall allows our after-sales colleagues to test the repair of individual vehicle components on the basis of 3D data, without the need for a real prototype. This gives us the possibility to incorporate their needs into the product development process long before the first prototype parts exist.

New Customer Experiences with AR and VR

Besides training and after-sales, virtual reality is the perfect technology for making sports car lover’s hearts beat faster. For the debut of the new Porsche 911 Carrera S at LA Autoshow last year, we worked together with Slightly Mad Studios to present an exclusive virtual experience, where visitors were able to test drive the all-new model in a unique, intense VR.

But this isn’t everything yet: In collaboration with Google, we have developed an augmented reality app to virtually experience tomorrow’s technology today: the “Mission E Augmented Reality” app. It features different view modes which allow our customers to digitally discover Porsche’s first purely electric sports car, the Taycan.

Looking at all these great projects, I have to pay tribute to our great partners. At Porsche, we are convinced that you cannot and should not shape the future alone — no matter how many experts, resources, and experiences you have. Developing technologies, and cars, for the future requires collaboration, cross-functional teams and new methods.

Partnering up with WayRay to integrate virtual objects seamlessly into the driving experience

Investing in WayRay, we found a truely innovative partner bringing tomorrow’s technology directly into our cars. WayRay produces holographic augmented reality head up display solutions like Navion, the first holographic AR navigation system for cars. It projects digital data onto the windshield through a holographic optical element using a process called SLAM, simultaneous localization and mapping.

It continuously maps the environment while simultaneously keeping track of your car’s location. The technology impressed me deeply, and made me incredibly curious at the same time.

Shaping the future requires curiosity, diversity and collaboration

In fact, this is one of the things I love most about my job — working on the edge of tomorrow, meeting creative minds, experiencing future technologies and developing ways to bring them into Porsche cars. In my job, it’s extremely important to look beyond the horizon and keep being curious and inspired. Regarding this, you can imagine how excited I was when I received an invitation from WayRay to join the Reality Virtually Hackathon at MIT Media Lab in Cambridge, Massachusetts, USA from January 17th to 21st. WayRay partnered up with the hackathon to sponsor the Automotive Prize Category and invited me to be a judge.

Of course, I said yes!

The Reality Virtually Hackathon: From strangers to awesome interdisciplinary teams

The Reality Virtually Hackathon is the largest immersive hackathon and explores how VR, XR and AR technologies can be used for building new experiences. It focuses on interdisciplinary teams and diversity and is open to developers, designers, engineers, artists, coders, sound designers, storytellers, students, and imaginative AR/VR enthusiasts all around the world. Then, 400 participants are selected from approximately 1,600 applicants, half students, half pros, to create original cross reality experiences and applications. What’s really interesting in the concept of the hackathon: Each participant has to promote his idea in order to form a team.

Alice in VR land

Virtual reality (or VR for short) opens up a host of new possibilities. And this futuristic technology now makes it easier for After Sales employees around the world to get to grips with complex technical concepts.

I was so eager to meet the other 400 participants from 53 countries all over the world. We kicked off the first day with a full day of skill-building workshops on XR design. During the next days, I met so many new people, brainstormed a lot and learned so many new AR/VR skills. The diverse atmosphere was simply inspiring. On the last day, there was an expo showcasing the other teams’ projects. Later that day, the expo opened to the MIT and the Boston/Cambridge AR and VR technology community, which was pretty exciting.

And of course there were winners: In the category “Best use of True AR SDK”, which is WayRay’s software development kit for bringing AR experiences onto the windshield, the Wayriders team won! The team created an Accudrive app to make people drive more accurately by gamifying the driving experience. As a judge, the connection of a fun driving experience and more safety on the road, convinced me!

I think what I loved most about the Reality Virtually Hackathon is the cross-functional approach of it — not only regarding the participants, but also the range of topics. It’s not only a hackathon for automotive solutions, the projects were about mobility in general, accessibility, gaming, social good, health and art.

I saw so many great ideas applying AR and VR technology onto real use cases, transforming my understanding of the potential of some industries. I am beyond thankful for this experience and came back to good old Germany with a mind full of inspiration and rolled up sleeves, ready to take Porsche’s head up display technologies towards our #nextvisions.