13 jul Magic Box
For some years now I am fascinated by the possibilities of the Leap Motion as a game controller. In 2018 I thought, inspired by a short story by Stanislaw Lem I once read, “how cool would it be, to have a real black box, containing a virtual world which your hands can manipulate?“
What is the big idea?
Stanislaw Lem once wrote a short story called “Seventh Sally or How Trurl’s Own Perfection Led to No Good“. It tells the story of Trurl the Constructor, who during one of his sallies met a frustrated king. The grumpy monarch has been deposed by his subjects because of his purported cruelty and left alone on a desert asteroid. Trurl wanted to cheer his up and decided to build a special toy for him: a miniature kingdom to rule.
I imagined this miniature kingdom as a black box. I thought, “what if I take a closed box, with two arm openings, like in a neonatal care unit, and a screen on top to see a virtual world inside?”
What have I done so far?
This is very much a work in progress. What I made so far, is this:
1] The initial model
As a first step, I built a first test model of the Magic Box using a standard computer monitor:
The sides were covered with wood panels painted black. A laptop (not visible) is connected to the Leap Motion and uses the monitor as an external screen.
What I learned from this model was:
- The Leap Motion did work, even behind closed panels, and even through an acrylic plate!
- The Leap Motion was able to still follow hands and fingers, even if these were inside rubber gloves!
- However, the reflections of the back of the monitor did interfere with the operation of the Leap Motion.
- Also, it was hard to look at the screen while standing (the angle at which you view the monitor)
- And, the arm movement inside the box was very limited.
2] How to pick up avatars
A central idea to the concept of the Magic Box was that you use the Leap Motion to pick up virtual characters inside the Magic Box, and place them on other spots in the virtual world. What I wanted, was that the avatars changed from their normal animated behavior to “ragdoll physics” when you pick them up using the Leap Motion.
I added the Final IK plugin (for inverse kinematics) to the character in the Unity3d scene, in particular the Full Body Biped IK script.
There is a demo scene in the plugin package called “Ragdoll Utility” which demonstrates the switch between Ragdoll physics and animation. What I had to do is combine that functionality with the basic functionality of the Leap Motion: the ability to grasp objects.
But actually, I want a combination of ragdoll physics and animation, so I tried my own scripts. Creating an avatar with Ragdoll physics is easy in Unity, it basically comes down to adding rigidBodies and (mostly capsule) colliders to the different parts of the avatar’s body.
I then added a script to switch between Ragdoll physics and animation, with the following algorithm:
- When Grasped, go to Ragdoll state,
- When no longer Grasped, use gravity to fall to the ground.
- When not grasped and grounded, go to Animation state.
It did work with the Leap Motion, but there were a number of problems left unsolved. In particular, the 3D space as the Leap Motion perceives it is somehow different from the coordinate system of the virtual world in which the characters “live”, so the Leap Motion seems to be grabbing not at the right position?
3] A new version of the box
The next step was to build a new version of the box. Another major issue to tackle was how to deal with the virtual world inside the box: In principle, the virtual world could be infinite, the box provides a view to a limited portion of the world.
I imagined the hardware controls on the outside of the box, e.g., knobs connected to rotary encoders, to scroll through the world in two dimensions. However, soon I realized that a truly infinite virtual world would pose a problem: how does the player know what part of the world to focus on?
Therefore, I created a simple script to “loop the camera” so that it stays inside a central square area, thus creating a “toroidal world” (a familiar concept from Pac-Man and other games). The world is composed of a seamless tile-able square, and surrounded by 8 copies of this square, just to ensure the landscape is continuous even if you look in the distance.
(If the camera leaves the world on the left side, it immediately enters the world from the right side). This ensures that the size of the terrain is limited, but seemingly infinite.
What is the next step?
The next step was to think about what games you would be able to play with such a Magic Box. Fortunately, at that time (end 2019) I had the help of a Game Programming intern, who was able to create a prototype of a game for the Magic Box. This will be discussed in Part 2 of this story.
Sorry, het is niet mogelijk om te reageren.