Flash
Having fun with Face Tracking and MP3 Pitch control
There is a lot of cool open source stuff coming out this year in the Flash community. Earlier this year we saw the Flash Face Tracking and 2 months ago Andre Michelle showed how easy it was to control the pitch of a sound in Flash 10. I love all that stuff.
So I though, what can I build to have a little fun with all this open source loveliness. A webcam/mic driven pan flute of course! 
Here is the source from this experiment.
Alice in Wonderland Papervision site is live
Great day yesterday. After all the hard work my colleague Graham and I put in, the site for the new Tim Burton movie has finally gone live.
Alice in Wonderland screen shot
For this project we wanted to showcase the characters in a way that made sense for the movie. So we came up with the idea of having them fly past while you (Alice) are falling down the rabbit hole.
The site is my 3rd Papervision site and I’m starting to understand more and more about creating a good working site using this technology. It is really important to re-use your objects and keep an eye on the memory used. If you’re not careful, you can grind a site to a halt quite easily.
The tunnel texture is a looping Movieclip that I put on a cylinder. The camera is inside this cylinder. Then I have a bunch of flat planes with things like a watch, a chair, a crown etc mapped on top of them. They fly past and as soon as they go off screen, they loop back from the bottom again.
The main object of course is the picture frame. I first started off with just a box but that didn’t give the right feel. So I created this more detailed shape that has all the edges and depth of a real picture frame. Graham then created the textures and I put it all together. I love how much more depth it gives the frame.
I use the interactive moviematerial for the front (character) and back (downl0ads) sections. I then added the neat things like the FileReference style download of the icons, so we didn’t need a popup.
Have a look at the site here: http://www2.disney.co.uk/DisneyMovies/alice/
Kaleidoscope fun in Flash. Now with superfast image export!
I found this supercool Adobe AS3 library that uses ByteArray to export images superfast. The only thing the PHP does is put a header on it and serve it up. Wicked.
And what better way to test that than building a webcam Kaleidoscope. Hours of fun! Here is the source if you’re interested.
Colour tracking with the webcam in Flash
Something I wanted to do for a while. Track a colour using your webcam in Flash. I draw the webcam into Bitmapadata so I can play around with it. If you take the getColorBoundsRect straight from the raw bitmapdata it will be very hard to get something useful as you just have to many single pixels and colours to analise. So I implement a little trick that makes it better to analise. First I blur the video so it smooths the colours. Then I turn it into a 8 channel image.
After that I just layer a fresh video feed that isnt distorted on top and start drawing based on the colour and position.
Enjoy! You can grab the source here if you want.
Flash Facetracking kicks ass
(This is a repost from March 2009, my blog crashed in the meantime )
Last week I noticed a couple Flash guys jumping on the face tracking code. The legendary Mario Klingemann (aka Quasimondo) did a great example and shared his source code.
I really appreciate the fact that people are sharing this source code for others to play with. Good work!
I totaly love the idea of being able to track your face with the webcam. I had a little demo working yesterday (st Patricks day) where I drop a green hat on your head. Hours of fun .
Today on the way into work I remembered the movie The Ring where peoples faces got blurred in pictures when they were about to die. So 20 minutes of playing around with Mario’s source code later and you can see the result below.
I can’t wait to spend a bit more time on this code and hopefully will be doing some client work using this technology soon. Fingers crossed.
Using Flex with Flash
Here at Digital Outlook we have a very integrated approach to building games and sites. Designers should be able to find things easily and be able to do the things they do best like design, layout and animation. But on the other hand developers should be able to build great code on top of these games and sites. One thing we always tend to use is the timeline. It’s very difficult for a designer to go into a Flash file and change a little thing if everything is put on the stage by code.
But being a developer I want to use good tools to write my code. I’ve coded using Flash for a long time. Although it works fine to write classes in Flash, it’s not the most helpful tool. So I tried using Flex for a while and I have to say I love it. I had to iron out a couple issues to make Flex work with our integrated workflow as I’m still using Flash a lot and so are the designers. All projects should be compiled from Flash.
In case you’re struggling with the same issue, here is how I’ve set it up.
Folder setup:
I’ve got the flash source folder and the html publish folder setup like this:
01_development\02_flash
01_development\03_html
Create a folder for the classes in ‘02_flash’ and name it ‘classes’. We’ll get to this later.
Flash setup:
Create a new Flash file and save it in ‘02_flash’.
Go into your publish settings and select the Flash tab. Hit the ‘Settings’ button next to ActionScript version.
- Make sure ‘Automatically declare stage instances’ is not ticked.
- Add a class path like this: ‘../02_flash/classes’. (for some reason ‘classes’ on its own doesn’t work)
Also make sure you export to the ‘03_html’ folder but that’s the usual stuff.
Flex setup:
To create a new project do this:
- File > new > ActionScript project.
- Give it a name: ‘WallEGame’ for example.
- Click on the Browse button and browse to the ‘01_development’ folder of your project.
- Click ‘Next >’
- At the bottom set your main source folder to ‘02_flashclasses’
- Leave the Main application as it is (this should be WallEGame.as).
- Set the output folder to ‘03_html’.
- Click ‘Finish’.
Flex now creates a couple folders and your document class that it opens immediately. One last thing you need to do is go back to the Flash file you created and set the document class to ‘WallEGame’. This should sit in the classes folder but you won’t need to do ‘classes/WallEGame’ because we added that class path.
The reason we created the classes folder is because Flex otherwise copies everything from the ‘02_flash’ folder to ‘03_html’ for some reason. If it only finds scripts like in the ‘classes’ folder it’ll be fine.
Working with it:
Ok now everything is setup, let’s enjoy the powers of both worlds.
Flex creates the folder structure for you.
Create a new ‘ActionScript Class’. File > new (or right click in the source tree). Put the package in the top line: ‘com.digitaloutlook.walle.game’ for example.
Give your class a name. ‘GameManager’ for example.
If your class extends a class (MovieClip for example), hit the ‘browse’ next to the Superclass field and select MovieClip.
Click ‘finish’.
As you can see in the classes folder, Flex just created this folder structure for you. How handy is that! It also created the class file for you and created the constructor function. Cool.
Declare stage instances
Since you unticked the box in Flash to autodeclare stage instances you now have to do this in each of your classes if you have named a movieclip in Flash. The variable needs to be a public one. So if you have a MovieClip named ‘start_mc’ on the timeline you need to declare it ‘public var start_mc:MovieClip;’ in the class that goes with it. If that start_mc also has a class linked to it you should typecast it to that class but more on that later.
Autocomplete & autoimport
Flex auto completes things for you if you want. So when you start typing a variable you declared in this class just hit ctrl+space and Flex auto completes it for you.
Flex also automatically imports the classes it needs. But you have to use auto complete before it does that. So if you start typing ‘MouseEvent.MO’ let Flex auto complete it and it imports ‘flash.events.MouseEvent’ for you.
How to work with linked Bitmaps and Sounds
Normally in Flash you just set your linkage class on a sound or bitmap and you use it like ‘new backgroundimage()’. The class doesn’t exist but Flash creates it at runtime.
But Flex then gives you an error since it can’t see the class. So what I did is create that class in either a ‘bitmapassets’ folder or a ‘soundassets’ folder. So it sits in this package: ‘com.digitaloutlook.walle.game.bitmapassets’ for example.
Create a class in Flex and name it ‘BackgroundImage’ and set it’s superclass to ‘BitmapData’.
In Flash set the linkage class to ‘com.digitaloutlook.walle.game.bitmapassets.BackgroundImage’. Now in Flex you can get to this image by using ‘new BackgroundImage()’. The same goes for sounds, but they need to subclass ‘Sound’.
Make sure you type cast everything
When you create your own classes with its public functions Flex can actually recognise this and show them as you have it auto complete the things you type. But you must type cast your variables. So when you declare your variables in the top and one of them is linked to the ‘GameIntro’ class for example, you should do it like this:
Private var gameIntro:GameIntro;
This way it shows all the public functions when you type ‘gameIntro.’.
Publish your project using Flash.
25 lines of code contest
Keith Peters organised a Flash coding contest where you could do anything you want as long as it was AS3 and didn’t exceed 25 lines. Very cool.
The contest closed last night so I can now post my entry. It’s not rocket science but I think it looks cool . Have fun playing and enjoy the code if you like.
[SWF(width=800, height=800, backgroundColor=0x000000, frameRate=24)]
stage.align = StageAlign.TOP_LEFT;
stage.scaleMode = StageScaleMode.NO_SCALE;//25 lines begins here
var bmdDisplay:BitmapData = new BitmapData(800, 800, true, 0×00000000);
var drawSprite:Sprite = new Sprite();
var insideDrawSprite:Sprite = new Sprite();
var aPoints:Array = [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]];
var aPositions:Array = [[0,0,1,1],[800,0,-1,1],[800,800,-1,-1],[0,800,1,-1]];
this.addChild(new Bitmap(bmdDisplay));
drawSprite.addChild(insideDrawSprite);
this.addEventListener(Event.ENTER_FRAME, updateStage);
function updateStage(e:Event):void {
bmdDisplay.applyFilter(bmdDisplay, bmdDisplay.rect, new Point(0, 0), new ColorMatrixFilter([1, 0, 0, 0, 0, 0, 0.85, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0.92, 0]));
insideDrawSprite.graphics.clear();
insideDrawSprite.graphics.beginFill(0xFFFFFF);
for (var i:uint = 0; i<aPoints.length; i++) {
insideDrawSprite.graphics.lineTo(aPoints[i][0], aPoints[i][1]);
}
insideDrawSprite.graphics.endFill();
for(i=0;i<aPositions.length;i++){
insideDrawSprite.x = aPositions[i][0];
insideDrawSprite.y = aPositions[i][1];
insideDrawSprite.scaleX = aPositions[i][2];
insideDrawSprite.scaleY = aPositions[i][3];
bmdDisplay.draw(drawSprite);
}
aPoints = [aPoints[1], aPoints[2], aPoints[3], aPoints[4], aPoints[5], aPoints[6], aPoints[7], aPoints[8], aPoints[9], [this.mouseX, this.mouseY]];
};
Flash on the Beach 2008 – Wednesday
Keith Peters
Some interesting insights into Inverse Kinematics. I already knew most of it after a game I did for Disney’s Wall-E movie.
I’m looking forward to the new book he’s working on which will deal with the more advanced stuff you can do in AS3 in relationship to games.
Also pointed out a good article on Gamasutra about advanced character physics.
Andre Michelle
I was blown away by his session last year. I mean come on, creating sounds in Flash using code?!? So cool. I love it when he asks ‘could you put music a bit louder please?’. He does this at the start of every one of his sessions. Usually followed by a little laugh.
What more to add to this guy, he does wicked stuff with sounds in Flash. Couple interesting links:
- Popforge code.
- Musicdsp.org.
- The crazy sound tool on Hobnox.com.
Koen De Weggheleire
This started off with the presenter struggling with his name 
. I can’t blame him if you’re English. Interesting insight is that Koen seems to be dating Angelina Jolie.
Koen showed us how to use filters like the colormatrix, convolution and colortransform. I knew most of it but it was good to see it explained again. Especially the transformation matrix. Weird, just looking at my notes and the Adobe site, I think Koen swapped the tx and ty values with the u and v values.
Mario Klingemann
Ok enough with the fun stuff, this is serious business. Just looking at my notepad, I have about 5 sheets of A5 paper with scribbles from this session. Mario basically explained how to build your own 2d barcode / QR Code reader. An example is the Kaywa Reader. Let’s see if I can make sense of my scribbles.
The three main things to use are: getPixel, threshold and histogram. The threshold basically should work in a way that it makes every pixel above a threshold white and below it black.
To extract all the blobs from the image we use Floodfill. In 3 steps, floodfill, getcolorboundsrect, threshold. Then delete it from image (fill with white).
Once all data is somewhat extracted we start looking at it to find the 3 main squares. Top left, top right and bottom left.
Then comes a bit which was quite hard to pen down. Basicaly you paint a radial gradient where center of square is. Use getPixel to check distance to that square. This is called a Voronoi Diagram. To find the closest, move towards light.
Then you have to find the smaller sqaure in the bottom right. Look for black dot in white area. Then look at the dotted lines that run from each of the corners to another corner. They are called timedots. This way you can find the scale of the grid.
Use projective mappings for image warping. This is useful as no-one keeps the image at a perfect angle. Paul Heckbert is the man who came up with this algorithm.
Ok once you’ve got your list of black and white dots into flash you can do 2 things. Compare them to a set of predefined patterns and pick the closest match. Or you can build an actual QR code reader. The source is available from sourceforge.
Breathe in, breathe out…
Jonathan Harris
This was an interesting one. Excellent pick to end this conference with. It really left you thinking. His message to the Flash community was that there are a lot of great people in the Flash world. Very technically advanced stuff is coming from the Flash community. So we speak the language well. BUT, and that was a big but, there haven’t been any real masterpieces. People can speak the language really well but we need to figure out what it is that we want to say.
I might not have put it down in the complete right way but I can certainly see where he going with it.
A couple phrases to keep in mind next time I create something:
- Can it make someone gasp or cry?
- Does it truly represent our time?
- Does it say something that has never been said before?
- Personal is powerful.
- Only fools get trapped by tools.
Flash on the Beach 2008 – Tuesday
Aral Balkan
Man than guy has some energy at 9 in the morning . Interesting talk about that you should just make and share more. This is very true for me, there are too many occasions where I thought of a cool idea and started coding it. Then decided not to put it on my blog because it wasn’t finished enough.
Also interesting one was that you should just put your idea out there. So others can comment on it and also because other people now know about it you will have to complete it.
- Scratch software from MIT that allows you to program in a very visual way. Simple but effective. I like it.
Jeremy Thorp
This session was about showing and approaching very complex stuff in a simple way. Also talked about Agent Oriented Programming. Instead of creating relatively dumb objects, he gave certain agents the possibility to take decision on their own. A simple example of this is the Flocking behavior. Where all the rules are quite simple but each bird can make its own decisions on what to do.
Another example was the ages old Conway’s game of life. I think every programmer should have programmed one in his/her lifetime .
Another interesting point was the ‘fitness landscape’. Great way to show that even though you think you found the best solution, there might be a better one.
Grant Skinner
Grant Skinner showed a couple interesting points that every developer should know. The main ones for me were:
- You must learn the rules before you can break them.
- Learn about every API. Not learn to use every API. Just learn enough so you know what’s possible.
- Spiking. Just try bits of code.
- Great way to debug: hit TAB + SPACE repeatedly.
- Write handy stuff for your designers/developers to use in Flash using JSFL. Adobe Link & Actionscript.org link.
- Have a look at a decent code editor like FDT or FLex builder.
After this was the Jam Session which was ace. Loved the performance art style coding in 10 minutes with Drum n Bass 
Hoss Gifford
Although it was an interesting session I think I enjoyed his session from last year more. Still couple interesting things.
- Ecotonoha. Interesting way of displaying messages that people have send.
- If you allow people to create something on your website, don’t start them off with an empty canvas.
I think that is definitely a good point. People sometimes don’t know where to start and it would be good to kick start their creative process. - Show the process of creation, not just the end result. This creates a narrative.
One of his examples was a drawing tool that didn’t just show the end result but the exact time line how people had drawn their pictures. This allowed the user to create a story. Simple but clever. - Design the experience not the user interface.
Small difference in approach could make a huge difference in the user experience.
After all of this I went to the session by Nando Costa and after diner to the GMUNK session. Both good.
Conway’s Game of Life in Flash
Sorry, not the most original thing to build but it felt like I had to do it
I love that something that is based on only 4 simple rules can create something so cool. Have a read through this article on Wikipedia if you want to know more about it.
These are the rules:
- Any live cell with fewer than two live neighbors dies, as if by loneliness.
- Any live cell with more than three live neighbors dies, as if by overcrowding.
- Any live cell with two or three live neighbors lives, unchanged, to the next generation.
- Any dead cell with exactly three live neighbors comes to life.
Have fun playing! Here is the source if you’re interested.