How Core Image Color Tracking works

CIColorTracking tracks a color blob from an image and overlays another image on top of it. It does that with a Core Image filter. How ?

Image:Core Image Color Tracking.png

To overlay an image on top of blob matching a given color, we need a location : the center of the blob.

If we have a set of 3D points, eg all vertices from a model, we add positions and divide by point count :

  • add all coordinates (x, y) into one point
  • divide by point count
  • there's the center !

To work from an image, we could go about the same way : go over all pixels, find the matching ones, average them down. We can't do that in Core Image as it is limited to kernel functions : no explicit loops, no temporary variables — only a kernel taking inputs (images, colors, numbers) and outputting an image.

The trick that CIColorTracking uses to average matching points is to go over ALL pixels of the image, repeatedly halving its size until it's 1x1. (This is done with a custom filter written in ObjC). It adds pixel coordinates and divides by area :

  • compute a mask from target color : white indicates target color, black is empty
  • convert to a coordinate mask : Red and Green store pixels' position (range is 0..1), Blue stores coverage (copied from the original mask)
  • average that mask down to a 1x1 image
  • divide the average position by the average coverage, this gives the location

SAMPLE QTZ Color Tracking.qtz You'll need to compile CIColorTracking first.

I guess this method will be obsolete in Snow Leopard, where OpenCL will hopefully simplify this down to one function.

Follow me on Twitter
Planet Cocoa

2011 02 22Distance field
2010 07 202Binding through NSApp
2010 05 122Forwarding invocations
2010 02 272Core Image black fringes
2010 02 21Quickest Way to Shell
2010 02 08Who's calling ?
2009 09 2138 ways to use Blocks in Snow Leopard
2009 08 182Bracket Mess
2009 08 124Taming JavascriptCore within and without WebView
2009 04 15Debugging with Activity Monitor
2009 03 25How Core Image Color Tracking works
2009 03 1510Custom NSThemeFrame
2009 03 10Which framework is running ?
2009 03 074CoreUI can paint pretty big
2009 02 18Localization with functions
2009 01 30Did you forget to nest alloc and init?
2009 01 16JSCocoa on the iPhone
2009 01 11Mixing WebView and JavascriptCore
2009 01 09Badge overflow
2009 01 09Find your Garbage Collection leaks with Instruments

Powered by MediaWiki