Video games are huge. They've also come a long way in the past thirty years. What began as monochromatic sprites and 8-bit music has evolved into a massive industry that pushes technology, artists, online casinos, and our experiences to the limit.
One of the most important aspects when designing a video game is immersion, especially for larger games that put their players into a rich and complex world with fascinating characters and larger-than-life villains.
These can range from being as simple as Mario, where you are simply a plumber trying to save the princess from a fire-breathing turtle to something more like The Witcher, a story of magic and legend, with inter-factional conflict, politics, and moral choices.
My personal favorite of these more complex games is Dark Souls, by Fromsoftware- the infamously hard RPG that a lot of "gamers" treat as a right of passage. However, easily one of the most popular games of the genre, and perhaps in general, is Bethesda's Skyrim. Despite being released nearly a decade ago, the game is still immensely popular and is only finding new life in the next stage of video game immersion: Virtual Reality.
The World of VR
Virtual Reality (or VR) is not a new concept. Back in the nineties, there were mono-screen VR helmets, and I know people who used them to play games like the original Doom with them. Nintendo tried to get into the field (sort of) as early as 1995, with a stereoscopic 3D console called "The Virtual Boy".
The problem was… it sucked. It was monochromatic and stood on a stand that you hand to put on a table and stare into. It was not a helmet. Plus, nobody made any games for it, and the games that were made for it… also sucked.
For a long time, Virtual Reality floundered. 3D was often seen as a gimmick, especially during the era of "3D" televisions. Nintendo released their 3DS console, which, again, was more of a gimmick than a genuine advancement in technology.
Various companies attempted to make motion controls, such as the X-Box Kinect (which was more fun to hack for various projects than to play with as intended), Playstation’s motion controllers, Razer’s Hydra Controllers, and Nintendo’s Wii Remote (Nintendo has done a LOT of experimenting with consoles, huh?).
So VR never really kicked off until the Oculus Rift, by virtue of actually working. It was bought out by Facebook, and once people saw that there was some actual promise to the technology (and more importantly, some money to be made), everyone and their uncle seems to have some kind of VR tech in the works. Google released their 15 dollar cardboard, which allows you to strap your cell phone onto your face.
The HTC Vive made its own headset, which tracked the position of the helmet using external base stations for greater accuracy. And now there are a ton of companies working on helmets and peripherals, ranging from new controllers to better motion detection, to haptic gloves, and even omnidirectional treadmills.
Speaking of which, I highly recommend looking up a channel called Thrillseekers on YouTube, which recently made a video about running a marathon in Skyrim, in VR. It's one of the coolest experiences I've ever seen, and I wish I could try it out, even if it does entail lugging my potato butt out of my comfy chair.
Face and Eye Tracking, Privacy, and What FaceBook Can Infer about YOU
A new feature that's slowly growing in popularity in the world of VR is the ability to track facial features. VRChat is one of the most popular VR experiences, where people get together in virtual lobbies and hang out. People can talk, roleplay, whatever.
If you have the tech, though, you can take this experience to the next level. Some VR headsets, however, can track facial movements and then apply your actual expression to your in-game model. In layman's terms, if you smile in real life, you will smile in the game!
Now, THAT’S really cool!
Of course, it does have its downsides. The most obvious of which is that more tech means an increase in cost. More concerning, however, is the privacy implications. With these features, you're basically giving Oculus (Read: Facebook) the ability to observe your face, possibly for hours on end, every time you play. That may not sound so bad until you realize just how much information a company like Facebook can gather on you, JUST from looking at your eyes!
In a paper titled “What Does Your Gaze Reveal About You? On the Privacy Implications of Eye Tracking”, German researchers Jacob Leon Kröger, Otto Hans-Martin Lutz, Florian Müller go over exactly just how much information can be gathered just by observing your eyeballs (and nothing else!). Here’s the abstract:
“Technologies to measure gaze direction and pupil reactivity have become efficient, cheap, and compact and are finding increasing use in many fields, including gaming, marketing, driver safety, military, and healthcare. Besides offering numerous useful applications, the rapidly expanding technology raises serious privacy concerns. Through the lens of advanced data analytics, gaze patterns can reveal much more information than a user wishes and expects to give away. Drawing from a broad range of scientific disciplines, this paper provides a structured overview of personal data that can be inferred from recorded eye activities. Our analysis of the literature shows that eye-tracking data may implicitly contain information about a user's biometric identity, gender, age, ethnicity, body weight, personality traits, drug consumption habits, emotional state, skills and abilities, fears, interests, and sexual preferences. Certain eye-tracking measures may even reveal specific cognitive processes and can be used to diagnose various physical and mental health conditions. By portraying the richness and sensitivity of gaze data, this paper provides an important basis for consumer education, privacy impact assessments, and further research into the societal implications of eye-tracking."
That is a HECK of a lot of information from just looking at one's eyes. It's also something that most people aren't likely to piece together themselves. What appears to most as a new gimmick / feature for playing a video game becomes a window into your entire character. It's for exactly this reason why, if I had the budget for Virtual Reality, I would never get an Oculus Headset.
There's just no way that they wouldn’t gather and use all that data somehow because that's FaceBook's entire business model. Do I trust that other VR companies wouldn't take advantage of it either? Not really. But knowing is half the battle, and at the very least, I know enough to invest my money in a company whose business model doesn't entirely depend on gathering all my personal data for running ads in front of my face.
The Devil in the Details
I think this is something that most people have yet to grasp about the 21st century. Everything you put online can be used to extrapolate far more information than you could ever expect. Even if you haven't already posted your entire life onto Twitter and Facebook, comments, posts, emails, purchasing history, and even what makes you pause scrolling is tracked.
You’re naive if you don’t think that FaceBook isn’t reading through all your WhatsApp messages to build their profile on you. This is a multi-billion dollar industry, and while I think nowadays most people know that their data is being tracked, most people have no idea just how much these companies know about you.
There are several machine learning algorithms that have been decried by various relevant news outlets because of their uncanny abilities to extrapolate information based on a seemingly unrelated set of input data. One of these AIs was capable of looking at skeleton X-Rays, and correctly identify the patient's race with an accuracy above 90 percent. Keep in mind, this is something a human doctor can't do, and the makers of the algorithm themselves weren't even sure what data the AIs were using to get this information.
More hilarious, however, is an AI that VOX dubbed the "algorithmic gaydar". An AI made for a Stanford University study in 2017 was supposedly capable of determining whether someone was gay or not just by looking at a photo of their face. Men could be identified as gay or not accurately over 80 percent of the time, and women could be accurately identified at a mere 75 percent of the time.
The AIs used photographs taken from an undisclosed dating site as it's dataset. I find the implications immensely funny, although VOX doesn't, claiming that this will hurt the LGBT community by blah, blah, blah. In my opinion, it’s too statistically significant to be decried out of hand, but that’s a whole other conversation.
My point is that a lot of information can be extrapolated from a seemingly irrelevant set of data. With advancements in Machine Learning, this process is only going to get easier, with more information being gathered more efficiently than ever before. The necessities of Virtual Reality are going to require gamers to strap all sorts of sensors and equipment to our bodies to increase the immersion.
Tech industry giants, like Elon Musk and Gabe Newell, have already talked about direct brain interfacing, and Elon Musk has already started a company called Neuralink to do exactly that (ostensible for medical purposes, but if the tech works, it works).
Now, I'm not saying that a trans-human Cyberpunk dystopia is on the way… but it very easily could. The world is changing faster than anyone twenty years ago could have ever imagined, and things that we take for granted today (such as the internet or the computer you're likely reading this on) were science fiction then.
Companies track millions of people, all day, every day, via computers. What’s going to happen, however, when the computers are actually inside us? Not because I worry that there are chips inside of vaccines, or something stupid like that, but because of the far greater worry that the companies will simply ask people to get their chips… and people will eagerly agree. Because it’ll be fun and cool.
And who the heck knows, maybe I’m worrying over nothing. However, there are unscrupulous people in this world, that if you give an inch to them, they will take a mile. These people work in Silicon Valley. They work in the government. And they work in industries that are more than happy to deplete their favorite resource- you.
