Category Archives: Echo of Heroes

Posts related to the Myo audio sword / magic game called Echo of Heroes.

Myo game update – Tutorial sequences

This week I focused on editing of my final report and adding tutorial sequences to the game.

I mentioned an issue in the previous post, I mentioned that one issue people playing the game had was finding it hard to know what to do without me giving hints, despite there being a written tutorial. 

I then thought that I could follow traditional methods and incorporate the tutorials into the game.  So, lacking voice actors, I decided to record the tutorials myself and add them into the game using a state manager (described below) that is updated after the player performs a certain action, or they can skip it by making a fist. 

This video shows the tutorial sequence up until completing the game.

It has been suggested that the game needs a few more variations in bosses and a bit of variety of attacks. This is true but I think the current game gives a good demonstration of how to play it. 

After a couple of tests with players, they seemed to understand the game better, although there is also a need to remove some actions from the game to prevent players accidentally performing an action in-game that has not been introduced yet. Perhaps there needs to be a game mechanic where you “learn” actions. This will need to be researched further. 

Tutorial state manager

The state manager was a very simple function that changed a boolean and reset a timer. The timer was used restrict progression of the tutorials until they had been fully played back to the player.

 // changes game state and resets timer
 private void ChangeState(string state)
 {
 _stateTimer = 0;
 _stateEntered = false;
 _gameState = state;
 }

 Where you see “stateEntered” is basically a boolean used as a simple “enter” functionality without having to create an entire state machine for a simple prototype. If this is true, during the update loop, it will perform “enter” functioanlity (such as play a sound once and reset any flags), and then set the “_stateEntered” bool to “true” and therefore not performing things every frame.

The update functionality is similar to the following:

 // state machine to control the game
 // mainly used for tutorial states
 switch(_gameState)
 {
 case "Setup":
 if(!_stateEntered)
 {
 PlaySound("Bad");
 _stateEntered = true;
 }
 if(thalmicMyo.armRecognized) 
 {
 PlaySound("Good");
 ChangeState("Tutorial 1");
 }
 break;
 case "Tutorial 1":
 if(!_stateEntered)
 {
 PlaySound("Tutorial 1");
 _stateEntered = true;
 }
 
 if(jointObject.GetComponent<JointOrientation>().spreadDone && _stateTimer > 16.0f)
 {
 PlaySound("Good");
 ChangeState("Tutorial 2");
 }
 break;
//... etc.

 basically, all the state manager is doing it play an audio file, wait for the length of this file (tutorial speech) and then wait for the player to do something that triggers the next one. Super simple. 

Future work

Some future work would include a “learning” system where the player cannot activate any action in the game until he or she has triggered it in a tutorial, thus activating it. This will prevent confusion, seeing as the game has no visuals to help with understanding the environment.

The distant future will include the addition of a story, more enemies, quests and more actions for the player to do. Each enemy would have different combinations of actions to beat it, and doing so will grant new abilities.

After this, I will be incorporating a GPS system to track quests as each quest will require the player to either be tracked moving 1 – 5 KM or a certain number of steps to reach the objective. This will hopefully be an interesting exercise routine app for mobile.

 

Myo game update 4

Last week I completed the first version of the Myo game prototype, now titled “Echo of Heroes”.

The game was a lot more simple than I imagined because I wanted it to be as simple as possible for players to get the hang of in a testing environment.

I even made an A4 poster for it:

Sword

 

The game was made for a research report about how a fun game could be made from using Myo and without visual feedback. The reason for this is mainly because of the ability to play a game without having to engage your visual senses, and thereofre making it a better platform for exercise routines on mobiles. The Myo uses bluetooth and can be taken outside, so the game is going to end up being a larger game like Zombies, Run!, and incorporate missions and jogging. 

The way I put it together was with 2 main pieces of technology plyus some clever audio design.

20141008231112

The screenshot shows how the games’ “colliders” are put together. These are basically what Unity3D calls “things that if other things overlap with, things happen”. These are built into Unity, and can be easily utilized. The way the game uses these is when the player swings his or her arm wearing the Myo arm band and if a pretend sword collider overlaps with the capsual in front of them, it registers a hit.This is the same with the collider above the sword (in the screenshot), but is used for “blocking”. If the user has his or hand up in the air high enough it will collide with that flat box and can trigger the player blocking.

The way I conveyed the game state was a big loud voice echoing thoguh your headphones saying “haha, this is the end of the line, you tiny little speck” to signify that there’s a big bad monster in front of you. The big boss attacks, and this is identified by a climbing “reverse sword clang” sound effect stretched out over about 3 seconds and at the end if the player is blocking, it makes a “clang” sound, otherwise a “bash” sound and a type of buzzer sound to identify that the player has been hit and this is bad. The player attacks the boss by waiting for him to swing and as soon as he hits the player’s block, the player has 1 second to react by swinging his or her sword at the boss. If it hits, you hear a “ouch” sound from the boss, and if not, you hear a bad buzzer and a clang noise, like you are hitting the boss’s shield.

The Myo’s SDK was easy to import into Unity because they have bundled with it a package that lets you easily set up the environment. 

After studying binaural sound a little bit, I ran into a Unity plugin (free for students) called 3Dception by Two Big Ears.  This also came with an easy Unity package.

One of the clever things I implemented was the audio handling. because Unity allows 1 sound emitter per transform, I had to create a bunch of these transforms and attach sound to them. I added them all to a parent called “Sounds”, and referenced this to the game’s logic handling object. I then added to the logic object a sound manager function where I would only need to call “PlaySound(“Hurt”) for example to get it to play a sound. The function would basically search for the child by the name passed into the function and then play the sound on that object. This works well and makes future code more usable.

I also used the 3Dception audio for the sword position by having it constantly play a fire loop. The swinging sound was also played when the player swings the sword (mentioned in a previous article on how I did this). The binaural natrure of the sound made it feel like the sword was where it was in game. To test this with the users, I added one of the questions on the survey that asked this. About half of the people that did the survey said that they could tell, but it is not a vwery useful result as I did not explain to them that it was an advanced feature and they could have understood the feature as a difference to mono sound for example. To properly test this, the uisers would have needed to kjnow a bit about the issues wjth sound positioning in real life and how the ear is clever enough to know where sounds are coming from. They would also need to have a “good ear” as in, know the difference between normal stereo sound positioning in most games, and the binaural feature. 

However, there was a discussion I was involved in online about the binaural technique and was made aware of even more technology. It would be a good idea to study this further and compare different solutions.

George

Some problems I ran into in regards to the game was that the player had to firstly tell the Myo which direction the boss was. This is because the Myo’s absolute position is sent to the game, and it could be on any angle. This was half incorporated into the Unity package that came with the Myo, but it needed to be made usable in the game and disabled so that a user doesn’t accidentally activate it. The way I approached this was simply ask them to spread their fingers at the beginning of the game, while holding their hand to show them where to point it. This worked well except at some points the Myo did not register the finger spread gesture, and I was unable to do it myself without having to take the Myo off their arm and re-set it on my own. 

Another issue was when users tested the game, they had to sync the Myo with their body. Traditionally, this was a painfully long process of performing gestures, but they improved it to simply doing a “setup gesture” which was waving your hand away and then rotation your arm in the same direction. This was still difficult to explain to people, even with a picture showing them what to do. 

 Overall the results showed that despite the technical issues, the game was fun, but needed some sort of tutorial sequence to get the player used to the game. 

Myo game update 3 – Echo of Heroes

This week I focused on setting up the Myo and getting it to run in Unity.

The package that comes with the Myo development kit makes this easy as it comes with an example scene element (a ball on a stick – which is literally just that.) And as bunch of code that allows the Myo to rotate this ball on a stick. It also lets you change the texture of it and cause vibration with some gestures.

My goal this week was to get the scene working with sound and get an android build working on my phone.

Android compatibility

The first test was simply using the scene and compiling it for android. After doing this and launching it on my phone, the first issue emerged that, of course, my phone did not detect the Myo, even with Bluetooth switched on. I tried another method of using a USB extension cable (micro USB into the charger port and a normal USB out) and connect the Bluetooth dongle to that. I did this as some research revealed that the Myo needs the particular dongle hardware to function at all.

In desperation,  I contacted my trusty Myo contact via email, but got a vacation notice as a reply. I then decided to go on the Myo developer forums and ask the question of how I would go about getting it to work on my android. A Myo developer replied saying that the Myo is supported on android,  but not Unity. However, it would be supported in the future.

This was unfortunate because I had not allowed time in my schedule to get the game to work on android using Java or C++, so I would have to simply make a prototype on Windows using Unity and present it with the intent on going mobile once Unity is supported with the Myo on android. Not a complete loss, but it’s still a bit of a damper on my hopes and dreams of a fun audio game. It also means I have to re-do some of my test plans as they relied on a questionnaire about a mobile specifically.

Implementing audio

The next goal was making the game play audio. This was simple as Unity has an inbuilt audio system. So it was as simple as importing a wav file to play. I used a fireball dart sound as the sword swing. To get it to play, I wanted it to somehow detect that I am swinging the sword and then play the swing sound. To do this, I first tried detecting the difference between 2 quaternions from the previous frame and the current frame. This turned out to be more complicated as the Myo seemed to not always send data to unity at a constant rate, so the resulting difference seemed to be a random number or 0. This, I assume, is because the Myo does not send data at the same rate as the game update ticks, and so comparing the 2 differences would try and compare with the same position withing 1 Myo update a the difference would be 0.

A way to fix this would be to have a function that runs “onMyoUpdate()” and do the calculations there. However this may make the game go out of sync and make calculations with the game world harder. Also when working with audio, it is a lot easier to hear tiny differences in the audio volume or pitch than to visually see something, and so having some updates being skipped may cause the audio to be “choppy”.

In the end, I simply coded around it by first getting the difference of the vector of the end of the sword (the sword doesn’t actually move, but rather changes angle only, so the sword end would accurately represent a velocity of it spinning) of the last frame and current frame, and then added that onto a “power” variable. This would also decrease each frame slightly but never go past 0. This means that even if the difference is 0, there is still a possibility that the power value is higher than 0.

Using this power variable and a cool-down of about half a second,  I got the sound to play when the power was higher than a certain level, but also change the volume of the sound by the amount of power. This effectively made the long swing sound fade out quickly if the user stops swinging suddenly. The cool-down was added so the sound wouldn’t start to play continuously as the user swings. A better option than a cool-down would be a variable that calculates if the sword is travelling in a certain direction each frame, and if it changes, would cause it to reset some flag, telling the script that the sound can be played again. However, at this point in time we can go with the cool-down option and see how the testers go with the game.

Below is a video of the result, also showing the mentioned 0s in the console coming from the Myo.

What’s next?

The next thing I will be implementing will be a sound for when the sword collides with the center of the scene which would represent an enemy.

After this, I will implement a scoring system, where the enemy will have 5 hit points, and be blocking for some of the time. The blocking will be signified by a “shield being hit” sound with echo. The enemy will also make a sound for when he is about to swing. When this happens, you will be expected to block. To block, the player must hold his or her arm vertically and make a fist gesture. When this is done correctly, you will hear the same sort of sound as the enemy (but needs to be different, to avoid the player thinking the enemy blocked instead) this would be the core of the game.

Ideas for future

As the game cannot be ported to mobile using unity, we will not be seeing a mobile app in this version. However, once mobile Myo is supported in unity, the game could expand into a jogging adventure app where an adventurer can go on quests to slay enemies and get loot. The catch is that they have to jog at least 2 KM (using GPS or a treadmill) and fight the enemy before returning home.

Research update

As described in a previous post, I’ll need to research other motion capture and gesture capture controls like the Razer Hydra, Perception Neuron and the Nintendo Wii controller.

I will then look into other mobile games and how they do audio. Games would be “Papa Sangre” and “Zombies! Run!“. Some notable features of Papa Sangre is that they muffle audio from things that are behind the player. This is not a feature in unity, although it can be implemented. One issue is that the Myo game I am proposing has the issue of assuming the player’s head is facing towards the enemy as there is no head tracking. This will need to be experimented with.

Test cases

  • Audio panning and muffling helps or causes weird experience ?
  • Cool-down on sword. Is it annoying if you swing at a weird time? Should it play based on velocity or based on change of direction?
  • How fun is the game in general?

New Myo project

Greetings. When you last saw me, I was playing with the MYO gesture control device in an old C++ project. It seemed pretty responsive and because I have the device and haven’t done much with it, I’m going to be using it in another project this year.

An audio game

For this project, I will be attempting to make a game where you can play on your mobile device with audio feedback only (maybe MYO vibrations too). Basically, no visuals.

The idea of the project is to create a prototype and write a report about the findings of creating the game and whether the MYO can be used to engage the user without visual feedback.

Measuring engagement

The hard part of this project would be to measure the engagement of the users. After some research, there are many ways that this can be done, and there are many people that study this type of thing. I believe after enough research, I can figure out a good method of achieving engagement in the game users.

Other motion capturing hardware

In the report I will be comparing different human motion capturing devices such as the Nintendo Wii, Xbox Kinekt, Razer Hydra, and an upcoming product called the Perception Neuron. However, most of these don’t have the required specificatios needed for a game like the one proposed. The Perception Neuron, for example, uses wifi to connect to a PC that does the game simulation, rather than bluetooth which is what the MYO uses.

Other audio games

There are a couiple of audio-only games that spring to mind. One in particular is Papa Sandgre and Papa Sangre II by game company “Somethin’ Else“, where you play a game without visuals to hear the direction you are facing and various enemies. Another audio game is “Zombies: Run!” and “Zombies 5k”. These both engage the audience using a story told through audio, plus getting the player to do certain activities like leg raises or sprinting from a group of zombies. In Zombies, Run!, they also include a base that you can build up by adding resources they they collected during the jog.

Schedule

As always, I want to make due dates for certain sections of this project. Below is a matrix of milestones.

Week Date Assignment item Milestone
1 6 August
2 13 August
3 20 August
4 27 August Report intro, abstract and formatting complete.
5 3 September Myo gestures and movements implemented.
6 10 September Research on gesture devices and audio games complete and drafted in report.
7 17 September Mobile version working to test with, including sound effects. 1 boss fight.
8 24 September App tested with users and findings added to report.
9 1 October Draft of report done – more testing completed if necessary. No more adding to report – polish only.
10 8 October Report due Report completed and submitted.
11 15 October  
12 22 October  
13 29 October  
14 5 November Working prototype due Working prototype complete and submitted.
15 12 November  
16 19 November Draft presentation complete – prototype polished for presentation.
17 25 November Presentation day Final presentation complete and presented.

 

Testing

Testing will involve users playing the game and seeing how it feels. They will be asked a survey with questions similar to the following:

  • How much does the Myo enhance the game experience? (-3 to +3, where 0 is no enhancement or you would have just as much fun touching the screen, -3 is that the Myo makes it worse, or 3 where the Myo definitely creates a fun experience)
  • Would this game enhance your exercise routine if it incorporated GPS and jogging to a location?
  • How likely are you to play the game out in the open where people could see you? (-3 to 3 where 0 is neutral, 3 is very likely or don’t care, and -3 is shy and would play it at home, maybe using a treadmill.
  • How effective are the audio queues at providing proper feedback about the game world. (-3 to 3) : -3 makes it worse, 0 is neutral, 3 is makes it more fun and easier so you don’t have to look at the screen. 
  • How responsive is the Myo device at getting your arm’s orientation? (-3 – 3)
  • How responsive is the Myo device at getting the correct gestures (spread hands, fist, etc.)?
  • Anything that didn’t feel quite right?
  • Any notes that would help improve the game?
  • What was the most fun thing about the game?
  • What was the least fun thing about the game?
  • Would you buy a Myo if there were more games like this one?

 I will be adding more as the game develops, but as of now, these will be a good start at getting my report done. 

Playing with a Myo

Today I got my Myo Alpha in the mail to do some sweet codez. The device is pretty cool; it basically lets you “train” it to know about 6 or so “gestures” (like clenching your fist, or swiping your hand) and then uses an accelerometer and gyroscope to send orientation and acceleration data via bluetooth. It wasn’t too hard to set up either. 

Now, I can’t share any code or anything at the moment because of licencing in the Alpha program. But I’ve been messing with C++ in my old AI / Animation assignment from last year and I managed to get the pitch and yaw to control the camera with my hand:

What I did was, when I did the “fist” gesture, I could move the camera’s pitch and yaw with my arm’s pitch and yaw.

It may not seem like much, but I’ve technically figured out and programmed in methods that can be re-used and implemented into pretty much all of the game’s controls. Fir instance, I can easily use the same code in another game to shoot with the fist gesture, or jump with a flick gesture, all I need to go is use the glass and methods to check which gesture is being used, and the orientation data cane be accessed the same way. You can even easily make it vibrate.

If I had 2 Myos, I could do even more, like move with my left hand and look around with my right.

A future project I’m thinking about is something where I can use my arm to create music and special effects, like a fireworks display, just as a cool demo of the controller.