Thursday, September 29, 2016

Virtual Reality Using Your Computer as a Controller

I've been playing around with ThreeJS more and more now and I'm still not getting the hang of it. But I still enjoy using it even though I'm struggling through the darkness. I wanted to do something with its VR capabilities, but I also didn't want to pay an arm and a leg to get a decent VR headset. So I did the next best thing. I broke out my cardboard and played around with it. The results weren't Vive quality for sure, but for $15, it's exceptional what you can do with it. Unfortunately, it's hard to control something like this without a controller like the Vive. So I did something like Oculus did - made the computer the controller - but I did it without wires.
Let me explain how it works before showing you the code. I ran my own PeerJS server that facilitated the connection between phone and computer. All of the game's logic was on the phone, the computer was just a dumb terminal. Of course, you could send loads of data between the two devices using WebRTC giving more feedback to the screen, but I only used it to send the phone mouse and keyboard data. You can actually do pretty much anything that the Oculus can, provided you use WebRTC media to track head motion. There's a library that can sorta do this. I don't think it'll work with a VR Viewer, so you could rely on the phone's camera to track points in the environment. Either way, that part is hard.

I digress.

So, I ran the server on my computer in Node.JS. Then I wrote the protocol. When you open the controller, it'll give you a 5 digit code. When open the world on the phone, you are asked for that code. When you enter this code, a connection will be established with the controller, the controller will be notified, and you can start passing data. Right now, the protocol consists of number arrays that do the following things

[event,arg1,arg2]
[0,change in X,change in Y]
[1,change in mouse wheel]
[2,mouse button pressed,modmask]
[3,mouse button released,modmask]
[4,key code down,modmask]
[5,key code up,modmask]

The modmask is a bitmask of shift, control, and the alt keys.

Right now the only thing the world cares about is the mouse movement and the mouse wheel. A sphere moves around with your mouse. Scrolling moves the sphere closer or further away. To keep the sphere sorta tethered to the central point, I made the mouse change the angles around the center point on a spherical coordinate system and then I converted them to cartesian coordinates.

It works pretty well. Sometimes it freezes for a few milliseconds, but it catches up. You can get the code here. Running it is as simple as running the server and going to localhost:8120. Then throw your phone into the viewer and gaze upon the mediocrity. Move your mouse around and scroll. The ball magically moves without wires or expensive equipment... well... maybe the phone... and the computer... okay but it's cheaper overall, alright?

No comments:

Post a Comment