ALL SECTIONS

Now Reading
Saving the World With Virtual Reality – Spring 2018

ALL SECTIONS

ALL SECTIONS

— May 30, 2018

Saving the World With Virtual Reality – Spring 2018

  • One of the most promising pieces of future technology spreading awareness and shaping the world in ways we never thought possible.
pocket

VIRTUAL REALITY (VR) is a promising technology that has yet to capture a sizable audience. Those clunky headsets that let users immerse themselves in fantastic worlds are still relatively costly, and that indispensable experience—aka, killer app—remains elusive. Think of VR, and you dream up fantastic flights that take you across universes faster than you can slip on a pair of Nikes. Or you can imagine brutal swordfights with medieval knaves atop castle walls. Perhaps you will find yourself a superstar of sports or music plying your craft to a captive and adoring audience, all within the space of just a few square inches in front of your eyes. VR can be all these things, but many developers also are corralling the technology for social good and altruistic causes.

“VR for Good” and “VR for Impact” are catchphrases that have emerged to identify this nascent application of the technology. Last fall in Washington, DC, what is believed to be the first “VR for Good” conference was held. At the conference, spearheaded by Robert Fine, founder of the virtual reality website VRVoice.com, a few dozen idealists converged and put their plans to fix the world on display. We report on some of their stories on the following pages.

Architects follow guidelines to ensure high-rise buildings are disability-friendly—but what’s the equivalent of a wheelchair ramp when developing a virtual environment? The folks at Disney who designed “Trials on Tatooine” and “Meet BB-8” wrestled with that question when they developed and unveiled the Star Wars-themed VR games.

Developers for “Star Wars” creator Lucasfilm Ltd., speaking to the “VR for Good” conference via video hookup, outlined how their research on “Trials on Tatooine” paved the way for the more robust accessibility features in “Meet BB-8.”

“Trials on Tatooine,” set on a desolate desert world seen in the original 1977 “Star Wars” film, was the first foray into a real-time VR experience by ILM Experience Lab (ILMxLAB), a division of Lucasfilm that builds immersive entertainment experiences including VR, AR, real-time cinema and theme park attractions. Users of “Trials on Tatooine” can repair the iconic “Star Wars” ship, the Millennium Falcon, defend it from stormtroopers and wield a lightsaber in a roomsize space.

Developers discovered the positive accessibility features and shortcomings of the VR game when they unveiled it at Star Wars Celebration, a 2016 three-day fan festival in London.

For instance, members of the game’s development team had deuteranopia or color blindness, so designers were sensitive to the issue. As a result, users with the condition were unaffected by a scene in which Han Solo asks them to press a red or green button because the designers overlaid another visual cue on each color.

The game also scored points among the hearing-impaired because developers added a feature letting users switch between headphones and a bassheavy speaker system that brings to life the vibrations of the Millennium Falcon landing. “This had an interesting accessibility consequence because we later got reactions from people using hearing aids that headphones can cause feedback,” said Ben Peck, a Lucasfilm engineer who worked on the game. “For those individuals perhaps having a speaker mix is helpful.”

Some of the “Trials on Tatooine” designs aimed at first-time VR users also served to make the game more accessible to people with disabilities, Peck noted. For instance, designers made sure any button presses were optional, meaning players who could not press buttons did not “miss out on any part of the experience,” he said.

Designers also built the game so it can be played with one hand and a single controller to capture the form of a Jedi wielding a lightsaber. “That had added benefits for people who have an arm in a sling or who are holding a cane,” Peck said, “or maybe you’re in a wheelchair, and you’re using one hand to manipulate your wheelchair to turn it.”

It was among wheelchair-bound people that “Trials on Tatooine” fell short, the developers acknowledged, but for other dynamics. Developers made sure that people at three various standing heights could reach up, grab a handle and pull it down or forward, as called for in one section of “Trials of Tatooine,” though anyone seated in a wheelchair would be unable to reach the handle.

“That could have been solved easily with some really cursory play tests in our research,” Peck said.

Similarly, anyone bound to a wheelchair would have his or her sight lines obscured. In one section of “Trials on Tatooine,” Peck demonstrated how crates and barrels in the foreground of a scene would block a seated person’s sight. That user would not see enemies in the distance or the projectiles they fire.

Before tackling their next VR project, the developers dove into research to make sure they wouldn’t miss any key accessibility issues. They turned to gameaccessibilityguidelines.com, a website covering cognitive, vision, and hearing and speech standards, in creating basic to advanced games.

They pored over an architectural resource, Residential Remodeling and Universal Design, which offered ideas as simple as placing a shelf near a front door so someone entering a building could set something down so he could more easily use the door handle. And they consulted blind architect Chris Downey, who has given TED Talks on accessibility and offers tips like placing an audible water feature, such as a fountain, near an entrance to make it a focal point.

“Chris likes to emphasize that when designing a physical space we need to engage people along a multiplicity of senses, not just sight,” Peck says. “When you do this you’re lowering the overall cognitive load that it takes to perceive that space.”

The developers also turned to the research of Alice Wong, founder of the Disability Visibility Project, and created an online survey. The survey of 79 people with disabilities put locomotion and anything requiring specific body movements as the most difficult activity to perform in a VR environment. Roughly one-third of the respondents said they used VR once a day.

“The most fruitful part of this entire journey thus far, all the research we’ve done, the people we’ve interviewed and the survey results, was taking that information and putting it into practice with a new experience called “Meet BB-8,” Hannah Gillis, a Lucasfilm project manager told the “VR for Good” attendees.

Where “Trials on Tatooine” put users in a VR environment, “Meet BB-8” was more character-focused, giving users a chance to interact with the droid introduced to moviegoers in the 2015 film “Star Wars: The Force Awakens.” In the game, BB-8 enters a ship’s quarters where you can play laser tag with him, prompt him to dance by moving your head and pet him in what Gillis called a symbolic “moment of trust.”

Based on their research, the developers designed “Meet BB-8” to include a one-handed controller and required that no buttons be pressed except to engage the laser tag tool. And the game calls for no movement on the part of the user because the droid remains in his or her field of view.

That subdued sense of movement, coupled with the game being what Gillis called a “self-paced” and “low intensity” VR environment, made for what users told the developers was a very calming and soothing game experience.

Gillis and Peck said the most satisfying feedback they get on the VR games comes from disabled users who connect meaningfully with the environments. Even “Trials on Tatooine,” with weaker accessibility features than “Meet BB-8,” has drawn accolades, they said.

Peck recounted an email from the caretaker of a guest with “multiple severe conditions” expressing how much fun the guest had playing “Trials on Tatooine.” “And this guest was performing motions that were usually reserved for physical therapy, which the guest typically found either painful or really boring,” Peck said. “They were playing ‘Trials on Tatooine’ and didn’t notice any of this was happening.

“It was an empowering message for us,” Peck continued. “As a team, we were super-happy to get this kind of response. It was a call to action to keep enabling these experiences.”

Giving Children with Cancer a Place to Go – Even for a Little While

Piotr Łój runs his own creative agency, designing graphics and promotional media for the likes of Samsung and the Polish National Orchestra, but his most rewarding— and most personal—work comes through his Virtual Dream Foundation, which he founded last year to adapt VR and other technologies to humanitarian and medical uses.

The foundation was borne of the constant, heartrending emotional ups and downs that Łój, 35, felt and witnessed in his native Poland at hospitals and clinics where his mother was treated for brain cancer and at a hospice where she died at age 51 in 2008, three years after her diagnosis.

“I know what stress there is for the patient and the families all the time,” Łój says. “Every day, every meeting, every morning going to the hospice, you don’t know if you will see this person again.”

It was especially painful for Łój to see children in the hospice. “It’s a very difficult place to be, but everybody is doing their best and trying to bring as much relief as they can,” Łój recalls. “I remember how important every second of relief was, where you can take your head and stop thinking of the stress for just a second, and then you can go back with a new energy to cope with it.”

Four years after Łój’s mother died, a friend let him try on a pair of Oculus VR goggles for a virtual rollercoaster ride. He was less taken with the thrill of the ride than the realization that the immersive nature of VR can put you in another world within seconds—perhaps even if you are afflicted with cancer.

He immediately contacted Cape of Hope, a children’s oncological clinic in Wroclaw, Poland, where he was met with suspicion by the facility head and chief oncologist. Their concern: Was he was trying to exploit the children for monetary gain? It took two months, but Łój finally gained their trust. Before long, in consultation with doctors, he drew up guidelines to roll out the idea of putting children with cancer in a virtual world that would let them escape their daily pains, if only briefly.

The priority was to ensure the children’s safety, so patients who tended to have fewer fears were chosen. Second, the use of VR had to come at the right moment during treatment, when the children were in their most battle-ready frame of mind.

“We didn’t want to use it with kids in a depressed state because it could work completely the opposite way,” Łój says. “To have them fulfill dreams through VR when they’re depressed, they can just think, ‘Oh, this is what I’m not going to experience in my life because I’m sick.’ It’s very important to choose the kids in a mode when they are focusing on fighting cancer.” Finally, Łój and the clinic had to decide on content. After another two months of brainstorming, they decided they would virtually immerse the children among a litter of golden retriever puppies. Łój found a breeder and waited four months for the puppies to arrive. “Then I recorded it: four minutes of video of the puppies playing with each other and playing with the camera, which gives you a feeling of being in the middle of the puppy pile.”

Because many children in oncology units can neither sit up nor move much, for those first sessions Łój ruled out a clunky VR headset that requires a computer hookup via cable. He opted instead for a Samsung Gear VR unit for its mobility.

The reactions of the children and their parents were exactly what Łój and the oncology staff hoped for: happiness and laughter among the children, and tears of joy from their parents, who were seeing their children laugh and smile for the first time in months. “Then I recorded it: four minutes of video of the puppies playing with each other and playing with the camera, which gives you a feeling of being in the middle of the puppy pile.”

Because many children in oncology units can neither sit up nor move much, for those first sessions Łój ruled out a clunky VR headset that requires a computer hookup via cable. He opted instead for a Samsung Gear VR unit for its mobility.

The reactions of the children and their parents were exactly what Łój and the oncology staff hoped for: happiness and laughter among the children, and tears of joy from their parents, who were seeing their children laugh and smile for the first time in months.

“You could feel the relief,” Łój says. With that success, Łój sought more content. A few producers donated videos to his foundation. It was mostly travel and relaxation footage: tropical beach scenes, dolphins swimming, the expansive fountains at the Bellagio Hotel in Las Vegas.

Łój began to expand the concept. He contacted more hospitals in Poland to provide the VR experiences to other children. He received funding for more equipment and connected with Sheba Medical Center in Tel Aviv, the largest oncology clinic in the Middle East, where, with Łój’s experience as guidance, researchers are studying the viability of VR as a medical tool.

Łój is developing his own VR game for children with cancer. Unlike the content with the frolicking puppies, however, his game prods patients to confront their cancer head-on and embrace their treatment. As the children receive chemotherapy injections, they see themselves in their VR headsets become superheroes equipped with armor and guns.

“In front of them they see parts of the body being attacked by cancer cells, and, thanks to this chemo, they are superheroes and their guns can destroy the cancer cells,” Łój says. “It’s a mental approach to fighting the illness.” Łój, who showed a beta version of the application at the “VR for Good” conference, is consulting with doctors as he develops it further and seeks investors.

Łój has also expanded his work to include a Polish man, whom he identified as Christian, 30, who became paralyzed when he fell in an awkward position at a trampoline park. Christian is immobile and breathes with the help of a respirator. Łój recorded 360 degrees of Christian’s room from the paralyzed man’s perspective. While taping, Łój moved Christian’s hand and flexed his fingers in front of the camera. In post-production, Łój cut himself out of the video so that Christian sees himself moving his hand on his own, giving him five minutes of motivational bliss.

“After a week his wife called me and told me Christian asked to make the video longer, to a half hour, because he is very motivated,” Łój says. “Right now he is leaving the hospital and going home for physical therapy, so we’ll record another video in his home so he can keep using it. It’s changing his attitude.”

Łój says one patient in particular sticks in his mind—a sick girl who became very lonely during several months in a hospital room. Łój gave her his VR video showing the fountains at the Bellagio. When the girl realized there was a virtual crowd around her, she lost interest in the fountains and spent the rest of the session immersed in the crowd.

“She was amazed that she was standing in a crowd of people and was not alone,” Łój recounts. “She was smiling at them and waving.”

In its 12 years, an organization called Honor Flight has flown more than 180,000 older veterans to visit the memorials in Washington, DC, and elsewhere that have been erected to honor their service and that of their fallen brothers in arms.

Honor Flight has 131 hubs in 45 states from which to fly the veterans, but the supply is having a hard time keeping up with demand: more than 27,000 veterans are on the non-profit’s waiting list.

And time is not on the side of the veterans. Honor Flight, citing the Department of Veterans Affairs, says some 640 WWII veterans die each day. Moreover, many of the 80- and 90-year-old veterans who are left are too infirm to travel.

Sarah Hill, a journalist and a volunteer at an Honor Flight hub in Columbia, Missouri, took notice of the frail veterans unable to make the trip. “I would get voicemails from an 80- or 90-year-old veteran who said he was on too much oxygen or his doctor said his heart couldn’t withstand the trip,” recalls Hill, who founded the central Missouri hub with other volunteers in 2007.

So, in 2014, Hill and her team shot 360-degree video of an Honor Flight tour of the World War II Memorial in Washington, DC. She stitched it together to make a seven-minute virtual experience of the monument and visiting veterans, through whose eyes the vets on the virtual tour are seeing what’s around them. Tapping her journalism background, Hill added narration to the experience, which she put under the banner Honor Everywhere.

Besides being moved by the veterans in her own community, Hill says she was inspired by her grandfather, Staff Sgt Russell Hellwig, a WWII veteran who served in the Army Air Corps. He died before he could visit the World War II Memorial, which was unveiled in April 2004. “I look at these guys, these other veterans,” Hill says, “and I can see his face.”

The World War II Memorial, flanked by the Washington Monument to the east and the Lincoln Memorial to the west, is comprised of 24 bronze panels that surround the ceremonial entrance. Fifty-six granite columns ring a pool with a fountain. A wall of 4,048 gold stars is a reminder of the 400,000 Americans who lost their lives during the war.

Since Hill launched Honor Everywhere, thousands of veterans have taken the virtual tour. She has also filmed and provided virtual tours of the Vietnam and Korean War memorials, the National Women’s Memorial at Arlington Cemetery and the USS Nimitz, filmed in the middle of the Pacific with sailors offering greetings to the virtual visiting veterans.

The virtual tours for veterans are free. Honor Everywhere ships out headset kits on loan, but it has only six kits. “Sadly we don’t have enough kits to satisfy demand,” Hill says. “We have more than 100 veterans on our waiting list now.” Honor Everywhere also visits nursing homes and assisted living facilities to bring the virtual tours of the memorials to a broader audience.

In addition, Hill is the chief executive of StoryUp, a two-year-old company that provides virtual and mixed reality experiences to help lift people’s moods and highlight positivity. It created software called Helium XR, which, when used with a VR headset and a Muse meditation headband, lets users control their VR experiences with their thoughts and emotions. It combines guided meditation with waterfalls and other serene images, Hill says, as well as stories of challenges overcome.

Helium XR is used by nurses with compassion fatigue and others whose jobs put them under acute stress, Hill says. “We’re making the user more aware that their positive thoughts have power, not only in the virtual world but the real world as well,” Hill explains. “In our experiences you’re actually seeing your own brainwave patterns in the screen, and how they ebb and flow with different emotions.”

Hill notes that proceeds from StoryUp kit sales have been helping fund the Honor Everywhere project. A more robust kit, with an Oculus Go headset, is launching in May.

Hill acknowledges that there is no replacement for personal trips to monuments, and she encourages donations to Honor Flight through her website, HonorEverywhere.com, so veterans who can travel can see the monuments for themselves.

But if you know veterans who are homebound or in assisted living facilities, instead of waiting for a loaner VR kit or a visit from Hill and her team, you can download Hill’s StoryUp app and let veterans in your community take the VR tour of one of the monuments with a cheap DIY headset such as Google Cardboard or other available VR hardware.

Hill says the VR experience with veterans is creating small miracles. Before one virtual tour, one caretaker told Hill’s team that her father would have to be helped with his headset because he could not lift his hand. “But halfway through the experience he had his arm way above his shoulders,” Hill recalls. “He was trying to reach out to some of the people he saw in the experience.”

Another WWII veteran, Dayle Garrett, was a staff sergeant with the military police who managed the prison at Camp Patrick Henry outside Newport News, Virginia. Garrett recalls visiting Washington, DC, on his 85th birthday, but that was years before the World War II Memorial was built. Now, at 98, he doesn’t leave his Missouri town.

Two years ago, Hill visited Garrett’s home and let him take a virtual tour of the World War II monument. “It was sort of an unreal experience while I was going through it,” Garrett says. “It became more normal as I went through it and I forgot about the technology. I was moved by it. It brought back a ton of memories. Standing with the other veterans gave me goosebumps.”

Seeing What Animals See: Saving Lives Through Empathy

Alkistis Mavroeidi wants to let people see what animals see. In the process, she hopes to show how animals can thrive, especially in the same environment as people. Her affinity for animals, combined with her Harvard Graduate School of Design thesis on vision and reality, led her to this work.

For her thesis, to create a scenario similar to comparing how people see on their own versus through a VR headset, Mavroeidi looked into the differences in the visual perspectives of human beings and animals. “They experience the exact same space with biological structures similar to ours but actually see a different image,” she says. “The experience is completely different for a human than it is for a mouse or a pigeon.”

So she pored over lab tests and researched the ways that different animals perceive color. She studied the types of cone cells, responsible for color perception, in animals. She found that the three types of cone cells in humans let us perceive red, green and blue, while a mantis shrimp, for instance, has 17 sets of cone cells. “If you compare that to the three we have,” Mavroeidi says, “it’s probably like seeing a rainbow all the time.” Mavroeidi, now 27, considered what’s called field of view in predators—whose eyes are set forward, giving them greater depth perception but a relatively narrow field of view—and compared that to what prey animals see; the latter have eyes set on the sides of their heads so their field of view can cover almost 360 degrees in some cases.

From that research, she created a video game enabling players to enter as one animal and switch to the visual perception of another, depending on whether they were hunting or being hunted. “That helped me experience the advantages that these animals have over their predators and prey,” she says.

But it wasn’t easy depicting, for example, an owl’s acute perception of motion. So Mavroeidi used color as an avatar of sorts to emphasize movement. Similarly, an owl uses bobbing motions to help them get more accurate depth perception, also difficult to depict in a video game because the human brain doesn’t understand depth in the same way. So Mavroeidi used dark blue for objects farther away and lighter blue for those closer.

“Virtual reality is an amazing tool for creating empathy. Going through all these visual parameters of the animals increased my respect for them so much,” Mavroeidi notes. “Even just the view of what I call the humble mouse, which is practically blind. It’s incredible to see that they have such good perception of their environment from vibrations they sense through their whiskers. It helped me understand how the image of the environment is so different than ours even though the environment is the same. It was fascinating to go through these different iterations of the same scene.” Because her graduate school work focused on architecture, Mavroeidi’s research raised the question of whether creating a means for empathy with animals could improve building designs to avoid bird collisions, particularly with glass buildings. As many as 1 billion birds are killed annually in building collisions in the United States, according to a 2014 study published by the American Ornithological Society. Birds die when they see interior green habitats or the reflection of surrounding landscapes in building facades. The numbers of birds killed were confirmed by the Smithsonian Migratory Bird Center, which also found that low-rise residential buildings, because of their vastly greater numbers, were responsible for more bird deaths than high rises. Mavroeidi, who now works for the online travel site Kayak evaluating how the site’s designs enhance customer experiences, hopes her research complements the work of environmental groups such as the US Green Building Council or others that are identifying species particularly vulnerable to collisions. “It presents a realistic image of what the visual perception is for that animal,” Mavroeidi says of her research. “That could be useful where a particular animal is endangered or where the animal is important to the environmental balance. I’m really passionate about it, and I definitely want to see this move beyond an academic application.”

 

© Copyright 2020 Discover Life Magazine. All rights reserved.