Gee!Design

I recently managed to connect an MPU-9150 breakout board to a Unity3D application, using a Teensy2.0 as an interface (also got it working with Teensy++ 2.0 and Arduino Uno). With this combination I can easily rotate an object in the real world and have a counterpart in a virtual world do the same in real time. I chose the MPU-9150 because its”sensor fusion” functions make this possible without much mathematics.

I quite like the result, and I want to share my experiences and code! On this page, I go into the details of the Unity program. The whole story (including Arduino and Unity code) can be found here.

Unity program details

I used Unity 3.5.1 on a PC. Here is a zipped Unity package which you can include in your own applications to test the Light Saber. (The light saber model which I used in the video is not included, since it is not for distribution – however, you can easily get an FBX  light saber model on Turbosquid for a few dollars). My Unity program is based heavily on the program included in the “Unity Arduino Serial Connection” by Tiuri de Jong. Installation and setup are detailed on the main page of this project.

In the Unity program, there are two main scripts that control the behavior of the object that we want to rotate on the screen:

1. “GuiArduinoSerialScript.cs“. This is basically the same script as written by Tiuri de Jong, it is the script that handles the serial connection to the Arduino. I made some minor changes, i.e., in order to make the COM port an attribute of the script. (I am under the impression the script could do with a bit of improvement).
2. “UseSerialQuaternions.js“. This is a script written by me, but copied for a large part from the script “UseSerialOnStage.js” by Tiuri de Jong. This script uses the serial connection to the Arduino to actually get some data, and transforms the data into usable attributes, e.g., quaternions in our case.

GuiArduinoSerialScript

The GuiArduinoSerialScript is the script that actually handles the serial connection to the Arduino. The essence of this script is in the Update function:

void Update ()
{   
    updateslower++;
    if(portopen && _SerialPort.IsOpen
                && updateslower>=updateinterval)
    {
        askforinput();
        updateslower = 0;
    }
}// Update

So the script will ask the Arduino every “updateinterval” frames for input. (The default value for updateinterval is 4). The function askforinput sends a character to the Arduino, and thereby asks the Arduino to send data. The data is read from the serial connection by

incommingdata = _SerialPort.ReadLine();

and subsequently transformed into (integer) arrays for digital inputs and for analog inputs.

UseSerialQuaternions

This script uses the serial connection to the Arduino to actually get some data, and transforms the data into usable attributes, e.g., quaternions in our case. It begins with the important definition in the Start function:

Cconnect = GetComponent("GuiArduinoSerialScript");

This Cconnect object will pass all the data from the Arduino to our script. So in the Update function, the following lines get the analog and digital inputs from the Arduino into integer arrays:

LocalAnalogArray = Cconnect.AnalogReceive();
LocalDigitalArray = Cconnect.DigitalReceive();

Since we transformed the float values for the Quaternion data into integers before sending them to Unity, we now need to transform the integers back into floats. This is also done in the Update function, in the call to ComputeActualSignals();

The transformation itself is obvious:

function ComputeActualSignals() {
    // compute actual value of the analog input signals
    // (using the first 4 values for quaternions)
    for (var i : int = 0; i < maxAnalog; i++) {
       ActualSignalsArray[i]
                = LocalAnalogArray[i] / 1000.0f;
    }
}

Now all that remains is to apply the quaternion data (the first 4 values in the ActualSignalsArray) as rotation to an object.  This is done in the function UpdateObject, which is also called as part of every Update.

function UpdateObject(){
    var from : Transform;
    var myRotat : Quaternion;   
    var speed = 0.1;        

    from = myGameObject.transform;   
    myRotat = Quaternion (ActualSignalsArray[0],
                          -ActualSignalsArray[2],
                          ActualSignalsArray[1],
                          ActualSignalsArray[3]);

    myGameObject.transform.rotation
           = Quaternion.Slerp (from.rotation,    
                               myRotat,       
                               Time.time * speed);
}

myGameObject is an attribute of the UseSerialQuaternions script, and is set to the “actual_pivot” object. You see here that the rotation as passed from the Arduino is used as a target rotation for the game object in the Unity application. The order of the quaternion parameters and sign changes is determined by trial-and-error. (I just tried all different combinations until the object on screen behaved the same as the object in real life.)

😉

Conclusion and future work

It would be nice if the sound of the light saber depends on the speed of rotation. For that purpose, I built the following code into the UpdateObject function:

// compute the angle of rotation between frames   
myAngle = Quaternion.Angle(from.rotation, myRotat);   
//Debug.Log("Angle = " + myAngle);   
ComputeMovingSound( myAngle);

However, I never got the sound quite right. This I leave up to you!