“Frame accurate” playback in flash 10.1 (or not so much), recently Adobe annonced “Frame accurate stepping” in side the spiffy new release though this feature did not get as much attention as the performance features for obvious reasons though it would be nieve not to realise the types of applications that could be built with it.

I may have read into the bold statement incorrectly the first time round assuming this meant we could finally build a fancy frame accurate player (HH:MM:SS:FF – SMPTE time code) and potentially create a rough cut editor leveraging the ByteArray access in the NetStream object (http://www.bytearray.org/?p=1689) but of course my dreams have been pushed to the side while trying to figure out a way to have consistant frame accuracy using the NetStream object.

Some information needed to get started

Two get going you will need a FMS 3.5+ server with “Smart Seeking” configured to an ON state and secondly a client that supports stepping, sadly the popular OSMF player has not integrated this yet but since it is an open source product it is “easy” enough to tear out the guts and add step capabilities or of course create your own player (no small feat for one man) by setting the inBufferSeek setting to true and calling the netstream.step() methods.

Needed background information

In FMS terms it is refered to as “Smart Seeking“, not sure why they have resorted to this on the server and stepping in the client but keeps you on your toes remembering all the lingo:

Flash Media Server 3.5.3 and Flash Player 10.1 work together to support smart seeking in VOD streams and in live streams that have a buffer. Smart seeking uses back and forward buffers to seek without requesting data from the server. You can step forward and backward a specified number of frames. (Standard seeking flushes buffered data and asks the server to send new data based on the seek time.) Smart seeking reduces server load and improves seeking performance.

This next picture shows some of my sad moments testing frame accuracy using an asset with a burnt in time code:

Not so frame accurate

As you can see we are not only out by a few frames but even a second, at first I initially thought my calculations were off and double/triple checked them:

* 24 frame/sec (film, ATSC, 2k, 4k, 6k)
* 25 frame/sec (PAL (Europe, Argentina), SECAM, DVB, ATSC)
* 29.97 (30 ÷ 1.001) frame/sec (NTSC American System (US, Canada, Mexico, Colombia, etc.), ATSC, PAL-M (Brazil))
* 30 frame/sec (ATSC)

If you are not sure what framerate you currently dealing with you could always use the callback onMetadata to rerieve, frames are simply calculated on the second and therefore ” framerate / 60 seconds”.

when building a player one thing should be thought of at all times, frame accuracy is only inside the Buffer and any action forcing the buffer to clear (seek() etc…) would ultimately cause you to lose the accuracy until the player can start sampling data from the buffer (that I am assuming on some of my discoveries), this makes true playback frame accuracy a bit of a dream.

So naturally I have done what any developer would do and pull apart each and ever piece of code I can find looking for the differences in accuracy.

Some of my current findings:

1. It appears frame accuracy is only available while inside the buffer.
2. If you use the seek() method at all for scrubbing the Netstream.time (used to get the current position) is in-accurate.
3. After a seek has complete and playback accuracy is incorrect the Netstream object will correct itself over time (+-20 seconds)
4. If you set the buffer size to ‘0′ the player is in-accurate all the way through.

It seems seek() is the biggest culprit at present and this is most likely to do with the buffer being emptied completly, on initial tests it seems if you call a few step( 1 ) a few times after a seek (giving the NetStream object a chance to sample buffer data) the player will correct itself.

more to follow on my quest to build the frame accurate player hopefully removing the little voices telling me it is not possible and being given a gold star

The basic RegExp of the validator

internal function validateWin32( target:String ):Boolean
		{
			//Check name is not blank
			if( target.length < 1 ) {
 
				return false;
			}
			// Leading, trailing space, dot
			if (target.charAt( 0 ) == "." || target.charAt( 0 ) == " " ||
				target.charAt( target.length ) == "." || target.charAt( target.length ) == " ") {
				return false;
			}
			// Multiple spaces
			if ( matchPart( target, "\\s\\s+" ) ) {
 
				return false;
			}
			// Explicitly illegal Windows characters
			if ( matchPart( target, "[$*\"/\\\\\\[\\]:;|=,]" ) ) {
 
				return false;
			}
			// Control characters
			if ( matchPart( target, "[\\x00-\\x1f]" ) ) {
 
				return false;
			}
			// Reserved names. DOS compatibility means that AUX.txt
			// is just as bad as AUX.
			if ( matchPart( target, "^(?i:CON|PRN|AUX|CLOCK\$|NUL|COM\\d|LPT\\d)(?:\\..*)?$" ) ) {
 
				return false;
			}
			return true;
		}

View Source

If you are interested in performance and memory issues in AS3 I would highly recommend reading the following:

Fast Integer math:
http://lab.polygonal.de/2007/05/10/bitwise-gems-fast-integer-math/

If you constantly have to measure items for display (positions etc…) it can have a lot of benefit using ‘Bitwise Operators’ (x << 1) as they are incredibly faster than your typical native operators (x * 2)

Optimizing Mobile Content:
http://www.bytearray.org/?p=1363

This is by far the greatest white paper I have found so far about generally optimizing AS3.  Do not let the mobile part detour you -  a very good read for flash overall

Using Vectors in Actionscript:
Mike Chambers Article ->

The new Vector class is simply a typed ‘Array’ with huge performance benefits (Kick ass and chew bubblegum)

On to some more Item Renderer specific guidelines

____________________________________________________________________________________

Extract the common visual elements

It is a good starting point to extract all the common visual items from the itemrenderers. Particularly useful if you use ‘on mouseOver show’ controls as illustrated below:

As you can see the controls are only required when a user rolls over an Item and therefore these controls (play && download buttons) only need to be instantiated once == Yippie much better performance + lower memory usage.

How to create the above TileList example:

Each Item Renderer will only have the Image and associated text (in this case album/artist name) and the controls will be created and added to the parent item (List – Gumbo, TileList-Flex3 etc…), we can then listen to the ITEM_ROLL_OUT && ITEM_ROLL_OVER events and position/show our controls accordingly using the X & Y coordinates of the event.target which will be each Item in the list.

View Source

As you scroll through the above list you can see the RED square following the items in the list (I am using tweener to follow the list but this can be popped up in the correct position instantly), it is not so apparently beneficial with so little items though start to increase the numbers and watch the render time lower and physical memory usage drop off.

Using images inside an itemrenderer

The approach to take is dependant on if the images are static or dynamic/remote, I’ll cover remote images in detail though by doing this it should answer any query’s you may have about static ones.

Remote/Dynamic Images

The easiest and heavy approach is to simply use the “Image” component (mx.controls.Image), the Image is great to use outside of an itemrender though you when you look into the image class you realize it extends SWFLoader (hence why all the loading is done for you), this can be extremely expensive in memory.

Solution: Use the new BitmapImage (spark.primitives.BitmapImage) component in junction with a static loader class (Simple static class external where all loading is handled for you).

You could also make use of current ‘Bulk Loaders‘ such as http://code.google.com/p/bulk-loader/, It is almost always better to build these types of things yourself though (in terms of performance), if anybody needs some more details please just Twitter/Hola/Email/Touch-Me

Text

When building Flex 4 Applications you should use the Spark controls where possible, the spark controls are lighter and have better font embedding, if you are NOT using “Right to Left” text you should NOT be using RichText (made the mistake early on).

The spark label (spark.components.Label) field is light enough to use with large data providers

nothing more to say here….

Events

As much as we would like ‘Events’ not to be used inside item renderers we simply can not survive without them, one of the key things to remember is that if we do not REMOVE/DISPOSE the EventListeners when the itemrenderer is destroyed (removed from stage, data changed in list) the flash Garbage Collector may not be able to completely destroy it.

/**
* Dispose Item
* @public
* */
public function dispose( ):void
{
	removeEventListeners( );
	//Destroy Children && Elements
}
 
/**
* Activate Deactivate Component
* @private
* */
private function activateDeactivate( evt:Event ):void
{
	switch( evt.type )
	{
		case Event.ADDED_TO_STAGE:
			addEventListeners( );
			break;
		case Event.REMOVED_FROM_STAGE:
			dispose( );
			break;
	}
}
/**
* Add Event Listeners
* @private
* */
private function addEventListeners( ):void
{
	addEventListener( Event.REMOVED_FROM_STAGE, activateDeactivate, false, 0, true );
}
/**
* Remove Event Listeners
* @private
* */
private function removeEventListeners( ):void
{
	removeEventListener( Event.REMOVED_FROM_STAGE, activateDeactiveHandler, false );
 
       //Remove all other event listeners here
}

This example shows how you could handle adding and removing events from the renderer, when the renderer is destroyed it is removed from the stage causing the Event.REMOVED_FROM_STAGE to dispatch, we can not remove all events and make sure all children/elements are destroyed correctly (Garbage Collector == Happy).

Typical I create an interface (IDisposable) and implement this with function destroy( ):void.

It would be worth noting this applies to any custom component, it is good to clean up after yourself (remember wifey giving you a hard time about the dishes!)

Timers VS EnterFrame

I have no idea why somebody would need either of these in Itemrenderer’s, perhaps some Photoshop ninja requires an animated swirl every couple of seconds to tempt users to interact.  I would personally  fight this off as much as possible, though if  unavoidable it is good to understand the differences between these two approaches.

The facts:

1. Timers are more expensive than the EnterFrame event (more CPU intensive).
2. Timers are more accurate than the EnterFrame as the EnterFrame event relies heavily on a users CPU (a slower CPU == less times EnterFrame fired) though it is not true that the Timer is 100% accurate itself, more information can be found here http://www.bit-101.com/blog/?p=910

This is getting long and bloated which is exactly what I wanted to avoid, maybe this should be done in parts (see you in part II)

The day started like any other until I noticed this unnecessary gap between each of my vertically stacked list’s using the TileLayout, yes it is very possible to take the approach of creating my own layout (that’s why the new Flex 4 Framework is so good) though I opted for looking for a property that might remove this, disaster! disaster! nothing works so next step is to extend tile list and look for this weird occurance.

Best I could do (though not completly happy with) is override the measure() and change the explicit height of the component as it seems explicit height is forcing the constant height.

Hopefully this helps somebody out or even better can direct me to a better fix

package com.skysongs.mediaplayer.view.layouts
{
	import mx.core.ILayoutElement;
 
	import spark.components.supportClasses.GroupBase;
	import spark.layouts.TileLayout;
 
	public class MusicTileLayout extends TileLayout
	{
		/**
		 * Constructor
		 * */
		public function MusicTileLayout()
		{
			super( );
		}
 
		/**
		 * Measure
		 * @inheritDoc
		 */
		override public function measure():void
		{
			var layoutTarget:GroupBase = target;
			layoutTarget.explicitHeight = Math.ceil(rowCount * (rowHeight + verticalGap) - verticalGap);
			super.measure( );
		}
	}
}

Please hurry up Adobe, I want the final release now…

ADDED!!! in case the measure property is not called when resizing is done you can override updateDisplayList and call the measure() function.

/**
		 * Update Display List
		 * @inheritDoc
		 * */
		override public function updateDisplayList(width:Number, height:Number) : void
		{
			super.updateDisplayList( width, height );
 
			/* Force remeasure */
			measure();
		}

Again I would not recommend this as a final solution (will most likely be fixed in an upcoming version of the SDK)

ps.. Logged an issue with Adobe (http://forums.adobe.com/thread/565055)

This is also to remind myself what music I like

The Happy Artists 2010:

Nellie Mckay (Different and fun with some intresting political views)

Regina Spektor (Brilliant, absolutely )

The Ditty Bops

Amanda Palmer (Will ok maybe not always this happy but a good listen)

The Eels (blocked out many annoying brats on the tube)

Mumford & Sons (If you have never heard it, it’s a must)

Shakespears Sister (New album after yonks)

Nightmared on Wax (Different to say the least)

Look at this I am just getting started and not once have I had to mention Lady boo boo and Joe Mck-something, I think it is high time we start to share real music with the world and not this teenage inspired chart (oh to rant on)

More music posts will be coming soon…