Insight VR

I’ve been asked to discuss both the details of both how headtracking works and how I used the DarwiinRemote library. I hope to accomplish both in this post.

I became aware of the concept of headtracking when I saw the Johnny Chung Lee video describing his wiiMote headtracking project, which is amazing, as are all his projects. I realized that it would solve the primary problem that was facing me in furthering the development of my laser games: it gave me a way to allow players to move in a game world while holding a laser gun with both hands.

I suppose I should have looked at Johnny’s code, but it is C# and I’m developing on a Mac so I never even bothered to download it. Besides, the genius of Johnny’s idea isn’t the complexity of its implementation but the idea of reversing the roles of the sensor bar and the wiiMote and in making glasses into the sensor bars.

I purchased a wiiMote and downloaded DarwiinRemote. In order to use it in my own project I actually needed to download the WiiRemoteFramework from the project’s SVN repository. This gives a program access to wiiMote generated events.

The next challenge was finding a Python wrapper for the WiiRemoteFramework. Some searching and inquiries on the PyObjC mailing list led me to a wrapper by Ian Johnson that allowed me to connect to the wiiMote (more on that in a moment) and get basic events.

There is only one event that I needed or wanted. That is the rawIRData event. Unfortunately it returns a structure and I was unable to properly wrap the method in order to retrieve the data from the structure. This caused me to have to add a some code to the WiiRemoteFramework.

WiiRemote.h:

- (void) rawIRDataHackX1:(int) x1 Y1:(int) y1 S1:(int)s1 X2:(int) x2 Y2:(int) y2 S2:(int)s2 X3:(int) x3 Y3:(int) y3 S3:(int)s3 X4:(int) x4 Y4:(int) y4 S4:(int)s4;

I also added this to the end of the handleIRData method in WiiRemote.m:

if ([_delegate respondsToSelector:@selector(rawIRDataHackX1:Y1:S1:X2:Y2:S2:X3:Y3:S3:X4:Y4:S4:)])
[_delegate rawIRDataHackX1:irData[0].x Y1:irData[0].y S1:irData[0].s X2:irData[1].x Y2:irData[1].y S2:irData[1].s X3:irData[2].x Y3:irData[2].y S3:irData[2].s X4:irData[3].x Y4:irData[3].y S4:irData[3].s];

This simply unrolls the structure the raw IR data is usually returned in and instead returns ints.

This proved easy to wrap in Python:

WiiRemoteDelegate = objc.informal_protocol("WiiRemoteDelegate",[
    objc.selector(None,
        selector="irPointMovedX:Y:",
        signature="v@:ff", isRequired=False),
    objc.selector(None,
        selector="rawIRData:",
        signature="v@:[4{?=iii}]", isRequired=False),
    objc.selector(None,
        selector="rawIRDataHackX1:Y1:S1:X2:Y2:S2:X3:Y3:S3:X4:Y4:S4:",
        signature="v@:iiiiiiiiiiii", isRequired=False),

and

def rawIRDataHackX1_Y1_S1_X2_Y2_S2_X3_Y3_S3_X4_Y4_S4_(self,*posargs, **kwdargs):
    for x in posargs:
        print '%8x' % (x)

allowed me to test that I was able to get the raw IR points in Python.

I then needed some glue code to connect the Python wrapper with my game. I put this in a file called ircontrol.py:

import math
DIST = 0
SLOPE = 0
X = 0
Y = 0
READ_ONLY = False
def set_points(x1, y1, x2, y2):
    global DIST
    global SLOPE
    global X
    global Y
    if (x1 < 1023 and y1 < 1023 and x2 < 1023 and y2 < 1023):
        DIST = math.sqrt((x1-x2)*(x1-x2)+(y1-y2)*(y1-y2))
        if (x2-x1==0):
            SLOPE = 100
        else:
            SLOPE = (y2-y1)/float(x2-x1)
        X = (x1+x2)/2
        Y = (y1+y2)/2
def get_vals():
    return (DIST,SLOPE,X,Y)

In this code I’m doing three very simple things. First I find distance between the two points. Then I find the slope between the two points. Finally I look at the locations of the two points and find the midpoint.

I currently don’t use the slope (tilt of the head) in my game. The distance between the points determines the distance the user is from the wiiMote. The further apart the points appear to be the closer the user is to the wiiMote. The location of the points is used to determine if the user has moved left or right and up or down.

Here is the game code that polls ircontrol.py in order to move the player in the game:

def move():
    global MY_X
    global MY_Y
    global MY_Z
    global LOOK_X
    global LOOK_Y
    global LOOK_Z
    global MY_ANGLE
    (dist,slope,x,y) = ircontrol.get_vals()
    if (dist > 0):
        if (x > 562):
            MY_ANGLE += (562 - x)/(8000.0)
        elif (x < 462):
            MY_ANGLE += (462 - x)/(8000.0)
        if (dist < 100.0):
            movement = (dist-100.0)/120.0
        elif (dist > 150.0):
            movement = (dist-150.0)/120.0
        else:
            movement = 0.0
        if (y > 600.0):
            jump.start_jump(TIME, MY_Y, False)
            print 'JUMP!'
        if (movement > 0.1):
            movement = 0.1
        if (movement < -0.1):
            movement = -0.1
        new_x = MY_X + movement*math.sin(MY_ANGLE)
        new_z = MY_Z + movement*math.cos(MY_ANGLE)
        if (get_base(new_x,new_z) <= MY_Y):
            MY_X = new_x
            MY_Z = new_z
    if (LEFT):
        MY_ANGLE += .05
    if (RIGHT):
        MY_ANGLE -= 0.05
    if (FORWARD):
        movement = 0.1
        new_x = MY_X + movement*math.sin(MY_ANGLE)
        new_z = MY_Z + movement*math.cos(MY_ANGLE)
        if (get_base(new_x,new_z) <= MY_Y):
            MY_X = new_x
            MY_Z = new_z
    if (BACK):
        movement = -0.1
        new_x = MY_X + movement*math.sin(MY_ANGLE)
        new_z = MY_Z + movement*math.cos(MY_ANGLE)
        if (get_base(new_x,new_z) <= MY_Y):
            MY_X = new_x
            MY_Z = new_z
    if (JUMP):
        jump.start_jump(TIME, MY_Y, False)
    MY_Y = jump.jump(TIME,get_base(MY_X,MY_Z),MY_Y)
    LOOK_X = MY_X + math.sin(MY_ANGLE)
    LOOK_Z = MY_Z + math.cos(MY_ANGLE)
    LOOK_Y = (384.0-y)/250.0
    if (y == 0):
        LOOK_Y = 0

I hope that is somewhat self explanatory. In any case it is very tied to the structure of my game environment and you will likely use some similar logic but different details to tie things into your own code.

I encountered two major difficulties in writing the demo. The first was wrapping the wiiMote Framework in Python in order to get the raw IR data. I’ve already given details concerning that.

The second is that the wiiMote consistently fails to connect to the demo every other time it is run. I have to open the bluetooth System Preferences panel and delete the wiiMote from the list of bluetooth devices after a failed attempt and then run it again. Until this problem is solved the software will never be of interest to anyone but patient hackers.