A Quick Start Guide to Mixed Reality

0share
0
0
0

Photo by Bram Van Oost / Unsplash

Have you ever been interested in making your own virtual reality, augmented reality, or mixed reality experiences? Or maybe you need a new workflow to liven up your VR practice. Whatever stage you’re at as a VR designer, this step-by-step guide will provide you a general rundown of how to create breath-taking projects from start to finish.

Concepting

The first step in any immersive project is concepting. Concepting is simply the process of coming up with a concept or idea of what you want your audience to experience. You’ll be creating a conceptual framework that guides the rest of your creative process.

At this point, you want to figure out if you want to work with augmented reality, virtual reality, or something in between. Try sketching out a storyboard to show some different scenes and actions your audience will go through. Collecting relevant visual references like physical objects or photos can also help to guide the look and feel of the project experience.

Concepting at its core is figuring out what you want to achieve with your ideas and brainstorming how to put those ideas into action. You want to start your projects with clear themes so you and your potential collaborators execute a clear creative direction.

Modeling

After concepting comes modeling. Branching off from concepting, you want to prototype some of your storyboard scenes to scale in 3D.

Modeling involves defining the geometric structure of a 3D object, virtually painting it with a texture, then showing how light interacts with the object. Think of it as creating a sculpture made out of clay. You’re pushing and pulling parts of the clay to form it into an object, which is essentially what you’re doing when creating a 3D object. You have to figure out what you want your object to look like and what the internal structure and material of the object is.

Some common tools for modeling include Maya, Blender, 3D Studio Max, Cinema 4D, Substance Painter, and ZBrush. All these tools have steep learning curves, but once you get the hang of them, you can make impressive, high-caliber modeling and animation projects using the same software as the pros.

Animation

Once you’ve created make-shift models for your project, you’ll want to incorporate animation. The animation process is all about bringing your 3D objects to life. 3D animations differ from 2D animations since we’re not capturing motion through individual frames. Rather, we’re making a stream of data that commands how an object or character needs to move over time.

One key concept that is used to create 3D animations is rigging. Rigging refers to the process of creating a skeleton for your object so it requires less data to animate that said object from scratch. So if an object has an underlying rig, you can attach the mesh, or skin around the rig to contain the object within its own shape. You can look into some other 3D animation techniques here.

Visual Effects (VFX)

In film and theater, we can see how a scene feels through how it’s lit. Especially evident in theater, atmospheric effects come into play to create surface areas for light to interact with.

Visual effects are typically calculated in runtime. The runtime engine you use would determine what you can do, and how to do that action efficiently. Engines make extensive use of shaders to create effects. Shaders can be compared to the gels (tinted filters) placed in front of lights when lighting a film set or stage. Animation software like Unity have built-in plugins for shaders, one example being Shader Forge. Applying shaders then help to create unique, visual aesthetics to the environments and objects within your piece.

Coding

Coding within the context of the mixed reality creative process is about describing the dynamic behaviors and overall flow of your project’s experience.

Let’s say you want to spin something programmatically, you would need to write a piece of code that tells the engine to spin for a specific amount of time. Coding implies defining logic and using language to store that logic. Each animation engine has a different language and approach to coding. Since most animations are now created completely digitally, coding helps to streamline production behind 3D visuals.

Sound Design

Sound is a subtle and critical ingredient to mixed reality projects. Sound design is based on figuring out how sound helps audiences locate themselves in a scene. Because sound itself is a major component of our perceptive system, you want to pay careful attention to how sound can affect your viewers.

People often think of sound design in terms of musical soundtracks. However, it’s the ambient noises, the everyday sounds and the foley (reproduction of everyday sounds) that elevate a scene.

In mixed reality, sound design is spatialized, so the output of an engine is dynamically created by the engine in runtime. That means the sound adapts to where the virtual camera is located. The closer an object is to the camera, the louder the sound will be. Audio cues are great ways to help an audience know where to look.

One of the challenges of sound design in a 3D context is designing sound to be all-encompassing. If a sound cue is placed in one point in a virtual space, then audience members will react by paying attention to that cue.

Testing & Deploying

Testing and Deploying are the final stages of the project. The testing component is concerned with getting feedback from your audience and making sure your experience successfully executes your intentions. Deploying is about sharing your work with a larger audience.

It can be a good idea to bring people unfamiliar with your project to go through your experience. With mixed reality, it’s critical you are testing and verifying how your project will be experienced and displayed. If you’re creating something with virtual reality, then you should test it with a VR headset and not on a 2D display like a TV screen.

Consider these factors as you test your project:

What kind of space is required?
How long is the minimum and maximum duration of the experience?
Is it clear how the user will engage in the environment once they enter it? Do they have a clear objective of what actions they should take?

You can also publish your project as an app so audiences can easily have access to your creations. For example, one way mixed reality apps are integrated in a public space is through performance art pieces or various museum settings. If you decide to pursue the app route, take careful consideration of the interactive design of your app. How will the app provide the necessary instructions for users to experience your project in the way it was intended? Help your audience explore and figure out what they should be doing.

Now that you know the fundamental components of what goes into making VR/AR/MR experiences, you can start creating your own projects today!

This content comes from the course Exploring the Mixed Reality Landscape by Pacific Northwest College of Art. Sign up for the course below to learn more about concepts and techniques in mixed reality: