Video Annotation Revisited

We have some students doing a term project using the SMI eye tracking glasses. They need to manually annotate the eye tracking data and stimuli but we have more students than SMI BeGaze software, so we tried out some of the annotation tools I’ve mentioned previously. Unfortunately, recently while demoing the RIT Code app to some students it seems that with movies using newer codecs the application is painfully slow when trying to move thru video frames. While the software works well with older codecs (e.g. mpeg-2) – It seems to be showing it’s age as it was created 10+ years ago with the older quicktime framework – Will need to look into seeing if it can be updated. In the meantime one of the students found Anvil Video Annotation Tool

Video codecs and cross platform/application compatibility can drive you nuts – I messed around way too much today to actually get Anvil to work. The problem being it has a very particular list of ‘older’ codecs it supports. I am not sure how well maintained the software is as the links to their demo movies to test Anvil out had broken links.  I have movies from an SMI tracker which are xvid codec in an .avi container which is not supported by Anvil. To get something Anvil compatible I tried a few things out and the best I could find (after trying virtual dub, mpeg streamclip, handbrake, and looking at ffmpeg (but running into some problems wirth each.

Ultimately, I found that a combo of handbrake (which can open these avis but doesn’t support old codecs) and mpeg streamclip (which can’t open the avis but supports the old formats) will work.

Make sure to install handbrake: and mpeg streamclip:

First use handbrake to open the SMI tracker avi’s and convert to a .mp4 format with video encoder chosen as h.264 and framerate chosen as same as source.

Now you should be able to open this new video in mpeg streamclip. If you choose file->export as quicktime -and then choose the compression in the dropdown box. H.261 and H.263 work, but you can also try the others listed here:

Only problem is mpeg streamclip converts to these formats really slow (seems like it takes the same length as the video, so a 10 minute video is a 10 minute wait at least on my 2014 macbook pro) so it might be good to try alternates for better speed/quality as ymmv from mine.

–Update: Jan 23 2018–

Over the fall I had some mobile tracking data I wanted to code and due to the problems mentioned above searched again for annotation tools. I found CinePlay and found it relatively easy to use and effective for making notes on a frame-by-frame basis



Video Annotation Tool

I’ve mentioned in a prior post that Jeff Pelz’s group has a handy tool (note only for Macs) that allows you do frame-by-frame manual analysis that is common in mobile eye tracking experiments. I was showing the program to some students recently and realized that the sourceforge page does not have a compiled version and some of the quirks aren’t well explained. Here is a compiled copy with source code. Note the program expects video formats that can be opened via quicktime and must end in .mov (I find handbrake helpful for video conversion), also the labels.txt file must be on your desktop to have your predefined shortcuts. Otherwise the program is like a video player where you can easily mark time codes and milliseconds into a text editor and then save to a text file

Eye Tracking Annotation Tools

I recently attended a workshop in Aarhus discussing Methods in Mobile Eye Tracking. Several good talks there and discussions, including one on annotation tools for mobile video data. While most eye tracking manufactures provide software tools with their hardware, there are also tools from academia that are open source and hardware agnostic and can provide some features lacking in the commercial software. Here is a short list of such tools available:

The first 2 are Mac specific and ELAN supports Win, Linux & Mac

Video Coder (Foulsham Lab, University of Essex)

RIT Code (Pelz Lab, Rochester Institute of Technology)

ELAN (Max Planck Institute for Psycholinguistics, The Language Archive, Nijmegen, The Netherlands)

*Update August 8 2016*

mediaBlix VideoGazer Happened to see this commerical tool from MediaBlix

*Update Jan 31, 2018*

Anvil Video Annotation – ‘ANVIL is a free video annotation tool, developed by Michael Kipp.’