There are many theories that reality is a mere illusion, that it is a figment of our imagination. However, we have the potential to augment that reality and even transform it entirely into virtuality. Technological advancements have empowered us to make this science-fiction our 'reality'. Virtual Reality and Augmented Reality technology have developed rapidly in the last 5 years - we can now augment our physical reality with the help of a simple app, or step into a different world by simply wearing a pair of glasses.
This technology has the potential to transform our lives. It can be implemented in fields ranging from art, music, and performance, to sports, education, and medicine. Unfortunately, these fields have been dominated with the visual medium, while excluding the 7,297,100 people who report having a visual disability in the United States. Visual impairments are not signifiers of intelligence or brilliance. Many people, such as Richard Turner, have preferred not being associated with their visual impairments due to the biases it creates in the minds of the people they interact with. Why then are we choosing to leave out 2.3% of the population when it comes to developing mixed-reality technologies?
In order to understand how a VR+AR agency navigates this upcoming technology space, and what the current challenges and limitations are, I interviewed Cortney Harding, Head of VR/AR Creative and Strategy at Friends With Holograms. Friends with Holograms is a young VR+AR agency taking on clients ranging from CBS and Verizon to Unity.
Photo by JESHOOTS.COM on Unsplash
Can you briefly describe what your role at Friends with Holograms entails?
Friends with Holograms is my company. I’m the founder, and we’re a full-service VR and AR agency. How we generally work is - we start by meeting with our clients and we do an opportunity assessment. We figure out where VR and AR can fit in terms of what the client's broader goals are. Once we’ve identified a project we want to work on, we do the creative and strategy around the experience. We have production capabilities, so we will build the experience. From there, we can help with distribution if that is needed - helping people figure out what headset to use, or how to set up a bigger installation piece, and what equipment that might need. Augmented Reality is about figuring out where the experience should live, and who the right partners are for the experience. Augmented Reality is more of a challenge right now because it has to be viewed through an app, although that is going to change really soon. For us it’s about figuring out can it live in an app so that people have access to it. If not, who can we partner with so that the experience can live in an app that people have more access to?
Would you say then that your work is more VR or AR?
I would say it’s probably 50-50 at this point.
What drew you to the field of creating cutting-edge mixed-reality experiences?
My background is pretty diverse I guess! I started as a Music Journalist for many years. I was a writer and editor at Billboard for a long time. I was writing more and more about music and technology, and I really wanted to work in that space. I found that to be really interesting, so I started working with different music tech startups. I did Business Development and Strategy for a couple of different music tech companies, and I really enjoyed that. A couple of years ago, I saw a VR installation at a museum and I thought it was really fascinating! I walked out of there and I couldn’t stop thinking about it - it was so exciting and new - I was completely blown away. Then, while I was finishing up my contract work, I started learning, thinking and writing a lot about VR. In 2016, I met the founder of a VR production company, we were on a panel together and we had a really good conversation and I started working with them. I did their Business Development and Strategy for about a year. It was a great experience, I learned a lot, and had an amazing time. Then I decided that I wanted to do something in more of an agency model, or a production company model. It has been about a year since I started the company. It has been a really interesting journey with a lot of ups and downs. Sometimes, there are projects where people are really interested, but they don’t get the funding, or a project is supposed to happen, but the creative changes. It’ been a lot of fun, but there have been a lot of ups and downs and learning, and pivoting a little bit. It's definitely been a really cool experience, and I learned a lot. There were definitely points where I felt like, oh my god, what did I do. I think that’s how it when you’re building your own thing. It's the kind of thing where, even if it looks like you’re doing amazing on the outside, on the inside, you’re like "Oh my god, what's happening." It’s a lot of work. I'm building the company more or less by myself - although I do have a pretty amazing team of people around me. It can be pretty intense! That being said, it's really fun!
What would you say is the biggest challenge when you’re working on a project?
There’s a bunch of different challenges. If we were, say, a Social Media agency, I wouldn’t have to sell the idea of Social Media. In this day and age, everyone knows you need Social Media. With VR and AR, people are still unfamiliar with it. A lot of people still don't understand the value proposition. So I don’t only need to convince them to work with me, but sometimes, I also need to convince them to do this project in the first place. A lot of people need help understanding what exactly to do, how to think about, what the technical limitations are, and what can be done. That stuff has definitely been a challenge. Going from there, it’s easier to sell the idea of us, once I’ve sold the idea itself. There aren’t a lot of agencies out there, it’s not like we have tonnes of competition, which is great. Another challenge for me has been figuring out, who is serious about this stuff, and who isn’t - who just wants to chit chat, and who doesn’t. When I first started, I was just so excited, I would just take meetings upon meetings with people, and brainstorm with them. I realized, that these meetings weren’t going anywhere. People just thought this was neat, and wanted to chat. They didn’t really have any intention of doing anything. Part of this business is figuring out how much to charge, when to start charging, being comfortable asking for money, being really good about having a capabilities conversation. But I can’t sit and brainstorm and chit-chat for hours unless there is a commitment to doing something. It’s really sort of making sure, that people have an intention of doing something. I read this really amazing article recently, that suggests treating your business as a law firm. Lawyers charge, you know, for almost everything. We’re not there yet, but one of the challenges of doing work that you really love and enjoy, and you’re really passionate about and that is creative, is that you tend to maybe undersell yourself because the work is fun. It’s fun to chat for an hour about what can be done, but that’s not going to pay your rent, or student loans, or staff. Figuring out how to get people excited about this, get people into it, and how to build a business model - we’ve finally gotten to a pretty good point with what we charge for different things - that’s been really good. In terms of other challenges, it's like any other small company might face - where we’ve scaled a little, but we haven’t scaled a lot. I’m very conservative about not wanting to scale until we’re absolutely ready. A lot of the work might fall on me. In VR and AR, in particular, as much education as we try to do, people don’t have a great understanding of the capabilities. So for us, it’s about being really clear about what is possible and what is not.
Do clients approach you or do you approach potential clients?
Both. We’ve gotten clients a bunch of different ways. We’ve gotten clients through connections we had, through other different jobs I’ve had, and through people that I know from the industry. There’s been a lot of referrals, and word-of-mouth. I meet a lot of people at conferences or events. I speak at a lot of conferences and events and I reach out to people. I do a newsletter every few weeks which people forward and people hit me up through that. People have emailed us through the website about a project. If I had a clearer sense of that, I would be able to focus a little bit more. We get business from a lot of different places, so we’re doing a lot of things.
Do people who do reach out to you have a clear idea of what they want?
Sometimes. It really varies. I’ve had clients come to me with very clear ideas of what they want to build, and then we can really refine it. I’ve also had clients come to me with much more general ideas, and that’s fine too. A lot of people that come to me have a sense that they should be doing something in AR or VR, or they’ve seen someone else do something. A lot of brands don’t want to be the first. But as soon as someone does something, they want to be the second in line to do something. “Our competitors are XYZ, and we want to do this too.” That’s fine, but then we're figuring out what that work is. A lot of creative VR, and AR, especially, might get a lot of press, it might get a lot of buzz but doesn’t generate a lot of interest, or engagement. So, it's about figuring out, okay, your competitors did this project, how can we make it better, and make more people interested, or make this more interesting.
How do you experiment, or brainstorm for new ways to create these experiences.
I spend a lot of time talking to people and going to events. I have my group of folks who work with me, and we spend a lot of time chatting. The busier we get, I feel like we have less brainstorming time. There are some points when it’s a little quieter, and we’ve spent some time brainstorming and coming up with things. I spend a lot of time just talking to people and brainstorming ideas. We’re all trying to stay on top of what is the latest technology - what are people jump-starting to put out, or experiment with - so when I go to my clients, I can be like "Here is this new thing, and you can be one of the first to do this. Let’s do this. This hasn’t been done before!" In VR and AR, there’s just so much that just hasn’t been done before. People are constantly building, technology is constantly changing and getting better. It is possible to just do a brand new thing.
In your articles, you touch upon various important topics related to AR and VR, such as art and activism, and the ethical considerations, changing behaviors, and also about providing users with unforgettable experiences. What are you most excited about?
I think it’s hard to pinpoint one thing. Broadly, I think this is going to be a new platform for communication. I think the way we learn is going to change. The way we view information is going to change. We’ve already seen the change from text to visual interaction. That’s not to say that books will disappear. I really like to read, and I like text-based things. I don’t think that will disappear anytime soon. I think what we’re moving towards, is a world that is more inclusive of different learning and communication styles. Augmented Reality and Virtual Reality will help move that change forward. There are a lot of people that aren’t text-based learners. They don’t learn well, just by reading a book and then writing something. They want to be involved in it, they want to be a part of it. And I think VR and AR can accomplish that. That is going to be really beneficial in terms of job training, and education, that is going to be massive, and allow people to take part in something. There is all this research about VR as the next best thing to actually doing something in terms of learning. In a lot of instances, it is not practical to teach someone of the job, because either the task is very dangerous, or the task is very hard to do, or very complicated, or they need to have basic skills before they can even attempt it, or it’s like a high-stress environment. Any of these things can be replicated in VR and give people a head start. With AR, I read something recently about AR and literacy. There are still so many people who are completely or functionally illiterate in the world. Allowing them to use AR with images and interactions for learning could be really interesting and groundbreaking. AR makes consuming information, much more natural. When I’m traveling, we’d be trying to navigate somewhere, and in a city, we didn’t really know, where we don’t speak the language. We’d be always looking at our phones trying to figure stuff out. If we’d been able to have a headset on, or a much more natural AR interface guiding us around the city, that would’ve been a much better user experience.
What would inclusivity look for people who are differently abled, for example, for someone with low vision?
The low vision thing is interesting and challenging. The better interface for someone with low vision is voice. There are a lot of cool VR experiences that use voice, I’m actually working on one now. The problem with some of the headsets is that they are not great for people with low vision. I put my father in a headset, he is in his early 70s, and wears glasses, and has age-related vision issues. He could never get the headset to focus - the parameters in the headset are limited - so it wasn’t a great experience for him. As the headsets get better, that will change. For Augmented Reality, building into the headsets, or the glasses would be really great. I definitely think for people with limited vision, that is a bigger challenge. Definitely, for people with limited hearing, AR especially can be really helpful, because it’s presenting information, that you would normally have to listen to, now you can get it visually. For people with learning disabilities, or different learning styles, its great. I think for people who have physical limitations, or physical disabilities, allowing them to use VR to experience things has been fantastic. There has been a lot of stuff about VR for different medical treatments, or pain relief. So for people who have chronic pain issues, it could be something really useful. Also, going further, for people who are living in a place where they don’t speak the language - they might be tourists, or immigrants, or refugees - allowing them to use a Google translate model with AR, where they can see everything around them translated, and sort of have that experience to help acclimate. That can help them at least get around, so they’re not isolated at first. Eventually, people that are living in new places, they would want to acclimate even more. But just to get people started, its really good to have that capability as well.
Where can we see trends emerging beyond marketing, art, games and installations?
For VR, I think consumer VR is still a ways off. There is an early adopter community that is really interested in it - the oculus go, is generally great - more people are starting to buy it. But I think the bulk of growth in VR, will be in training and education. Pretty much all of our VR projects are in training and education. AR, I think is much better for consumer use, because people already have smartphones. Once AR on the web launches, which will be later this year, or early next, that is going to be huge because you don’t have the barrier entry of downloading an app. That’s something we’ve actually built for. We’ve built some really cool web-AR prototype, so they can see what exactly it would it would like, and what the functionality would look like. I feel like AR, initially will have a much bigger and wider impact, but VR is going to take a little bit longer. The other thing to keep in mind is that no one knows what this is going to look 10 years from now. The headsets look better now than they did 2 years ago. With each new iteration, they’re getting lighter and faster, and now we don’t need to have a phone for it. Everything is going to change. Everything is going to continue to evolve.
What will the technology look like from in 5 years, in terms of people with visual disabilities or impairments? How prevalent do you think VR and AR are going to be in the next 5 years?
My hope is that when head-mounted glass displays come out to the wider market, care will be taken that the glasses work for everyone. So people with visual disabilities can, in fact, use the glasses, and enjoy the glasses just as much as anyone else can. More broadly, in 5 years, we’re likely going to have really good AR glasses, and displays that people will be using just as much as people use now. There is a lot of stuff on the market right now, but it’s mostly for industrial use. The glasses are just functional - they work, but they don’t necessarily look very good. For the glasses to have any sort of great, wide option, they need to look nice, and they also need to have a robust app ecosystem. The iPhone came out in 2007 I think, but there wasn’t an app store or an app ecosystem. It’s only when the app ecosystem started to grow, that we saw the wide adoption of smartphones. My guess is that the first generation of smart or AR glasses - people like you are going to buy them, but I don’t know if it’s going to attract a wide audience out of the gate. I think it’ll take 2 or 3 generations of the technology to get people, get everyone wearing them or most people wearing them. The other thing that is going to have to happen, is a sort of behavior changing when it comes to how we interact with the technology. I remember I got my first iPhone 9 years ago, and I remember having to learn how to type on it, and now it’s second nature. But at the time, it took a long time When we first get the glasses, it’ll probably be a lot of hand gestures. There’ll be hand-gestures to do certain things. It’s going to take a minute for people to get used to doing those things. So you’re going to see this really beautiful symphony of people walking down the street, all gesturing because I think that’s how people are going to wind up interacting with these things. I think voice is a lot of how people are going to interact with these things. What’s great is that we’re already seeing more and more people interacting with voice because of Amazon Eco and Siri and all that. I think we’re moving to a voice and image-based model of communication, and away from a text-based model of communication.
How would haptics be incorporated?
I love haptics! I think haptics have a bunch of different use-cases. I think more generally, in VR than in AR. I can definitely see some sort of world where you have your glasses, watch, or wristband, or some sort of haptic device. I’ve had an Apple watch for a couple of years, and it buzzes me when I have a meeting or a text message from someone. I can definitely see that as complementary to the headsets as people move away from the phones. I think in VR, haptics are incredible - there was one thing I saw a couple of months ago, that was kind of this RC experience. You hold the controller and there were certain parts in the experience where I felt more and less stressed out, or my pulse moved quicker. I didn’t really think about. I was there with a colleague, and he was just like freaking out “Oh my god, they have a haptic heartbeat!” This very subtle thing, you kept your thumb on, that would speed up and slow down ever so gently, but it would have this interesting effect on you. So you can use haptics for really subtle things - and I thought that was incredible. There's a really cool haptic glove that I’ve seen, I saw it like a year ago. I remember putting on their headset, it was just their demo, so it was super basic. I went to pick up the rock in the demo, and I had the glove on, so I felt like I was picking up the rock in real life - that was amazing. The haptic suits are really interesting - there are full body ones, which I don’t know if people will ever own. But for training purposes, they would be really cool. The sub-paths is really interesting. A lot of the uses right now are for music, or for some cool artsy VR thing, which I like. I’m also really interested in touch in VR. I did this experience, where, there was this really cute, fun experience. There are all these horses galloping around and you’re supposed to sit with your friend and tap on them. I thought that was just a really nice, playful, enjoyable VR experience. A lot of the touch-based VR experiences I’ve seen so far, are either very sexual, which is totally fine, but it’s kind of a done thing, or then there will be either really emotional touch-based VR experiences or very scary touch-based VR experiences, which is fine, but it’s a done thing too. So it was nice to have this playful, joyful touch-based VR experience.
AR and VR are primarily focused on screens, some kind of visual interface. Complementary to that, there are haptic, or sound integrated experiences. But what would AR or VR look like if there was no visual aspect as all?
Wow! That’s interesting - I’ve never thought about that. I think AR would be just voice. AR involves so much looking at visual. AR without the visuals might be just voice activation. With VR, I think it would be very much like immersive listening, and haptics. VR without visuals would be sound and physicality. I did an experience that was visually very boring, but sonically really beautiful. I just closed my eyes and listened. There was just this beautiful audio. Visual is a huge component of both of these technologies. But I think there are ways to experience them without seeing things as well.
7,297,100, or 2.3% of this country’s population reports having a visual disability. Many of whom are completely blind. Do you know any work that is being done, that might not have any visuals at all?
Not off the top of my head. I think that is really interesting. I wish I knew - I should look into that.
How does the team at friends with holograms address accessibility when you’re designing something for a client?
We try to make sure that it is as accessible as possible, within what we can do. We’re working on something for a client right now, we make sure that it is optimized visually, we optimize it visually, so we can turn up or turn down the sound based on someone’s hearing. Some of the things we do are gaze activated, for people with physical disabilities, so that they don’t need to use a hand control or anything. That’s probably something we should think more about. We spend a lot of time on client briefs. It is interesting now that you bring this up, I should, we should as a company, and as an industry think about even more “How do we make sure this can reach the widest number of people? How do we design to make sure that it reaches the widest number of people?" We are sort of working within the constraints of client briefs, and what they want. That’s kind of the first thing we think about it. I do also think that as a company, and as a company what we need to do better, is how do we make sure that this works for everyone?
You did mention how VR might be easier to design for someone who is blind. Without the visuals, I feel like there is a thin line between AR and VR. What would AR look like for someone who is blind?
That would mainly be voice. For someone who cannot see the digital layer on top of the physical world, but you can do these really robust voice experiences, and use that as a companion piece. For VR, I think it becomes more of an immersive audio or physical experience. Maybe some physical interactions and tactile stuff. These are really visual mediums. At a certain point, that’s just what the technology is. Accessibilty is something we need to think about, and other people need to be thinking about it as well. That’s really important.
Talking to Cortney was a great learning opportunity. Virtual and Augmented Reality technology has transformed experiences in bursts - with Pokemon Go, Daydream and Oculus, and with the rise of voice assistant devices such as Google Home and Alexa, there seems to be a general interest in assistive applications of these technologies. However, there is still a long way to go to make these experiences truly holistic and accessible for individuals of varying abilities. This conversation seemed to spark something in Cortney - who feels that it is crucial that the industry think more about how to reach wider audiences, and advance together. I hope agencies such as Friends with Holograms start incorporating accessibility as part of their design process - it might just have a ripple effect and transform our every day communication!