Quantitative video analysis

From FourMs

This page documents some techniques that are useful if you want to carry out quantitative analysis of musical material. Please also refer to the introduction to qualitative video analysis for more information.

1 Tools
- 1.1 VideoAnalysis
  - 1.1.1 The data generated by VideoAnalysis
  - 1.1.2 Todo
2 Plotting

[edit] Tools

There are several tools that allow for carrying out quantitative video analysis, but many of these require that you develop the patches yourself:

Eyesweb is a Windows-based platform for (mainly) video analysis and synthesis. This is a development environment, so you will need to learn how to create your own patches for this to work.
Max/MSP/Jitter is a commercial Mac/Win development platform for realtime music and video applications.
Matlab is a commercial programming language where the user can write text-based scripts to run.

[edit] VideoAnalysis

We have developed a standalone application called VideoAnalysis which allows for simple analysis on a set of video files. The application runs on both Windows and OSX computers and can import all files accessible from QuickTime.

Open the application by double clicking the application (OSX) or the exe-file (Windows).
Drag and drop a folder with video files in the available space in the upper left side of the application. It is not possible to only import a single file, then you will need to place it in a folder first.
Choose whether you want to export a motiongram along with your data.
Click in the box next to "Export to data" to start the analysis.
The result of the analysis is written to the same folder as where you started the application from (sorry about that, we will change it in the near future).

VideoAnalysis main window

Preview window

[edit] The data generated by VideoAnalysis

The data is exported to textfiles, named after the video files. The text files have 19 columns, where the first column displays time, the contents of the next 9 columns are data with an applied medianfilter, and the last nine columns are unfiltered data.

Column 1: Time stamp

This is currently not measuring correct units, please disregard the time-stamp until it has been implemented correctly. Each row in the text file contains data for a single video frame, thus for now sample index is a better time-measure.

Column 2: X position

The horizontal position of the center of gravity. X = 0 means that the center of gravity is located to the left in the picture

Column 3: Y position

The vertical position of the center of gravity. Y = 0 means that the center of gravity is located in the bottom of the picture

Column 4: X velocity

The first derivative of X position. The difference between the X[n] and X[n-1], where X is the X position (in pixels) and n is the sample index. X velocity > 0 means that the center of gravity moves to the right.

Column 5: Y velocity

The first derivative of Y position. The difference between the Y[n] and Y[n-1], where Y is the Y position (in pixels) and n is the sample index. Y velocity > 0 means that the center of gravity moves upwards.

Column 6: Absolute velocity

The first derivative of position. The distance component of the polar representation of X and Y velocities. Absolute velocity = √(Xvelocity^2 + Yvelocity^2).

Column 7: Direction

This is the direction of the movement of the center of gravity from the previous frame to current frame.

Direction = 0 means a movement to the right

Direction = pi/2 means an upwards movement

Direction = -pi og pi means a movement to the left

Direction = -pi/2 means a downwards movement

Column 8: Absolute acceleration

The second derivative of position, i.e. first derivative of absolute velocity. The difference between V[n] and V[n-1] where V is absolute velocity, and n is the sample index. This implies that if the absolute acceleration is 0, then the absolute velocity is equal in two succeding frames. If absolute acceleration is greater than 0, the speed increases.

Column 9: Change in direction

This value is meant to show the angular velocity of the center of gravity. This is not properly implemented yet, as there is no compensation for the flipping between direction = pi and direction = -pi.

Column 10: Quantity of motion (QoM)

A normalized value for the total movement in the current frame. QoM = 1 means that all the pixels in the video change from one picture to another, QoM = 0 means that there is no movement in the picture since the previous frame.

Columns 11 through 19 are the unfiltered equivalents of columns 2 through 10.

[edit] Todo

Translate to English
Evaluate the Rotation and Orientation representation. Perhaps use sin(angle) instead of the angle values to avoid flipping between -pi and pi.
Implement user-selectable destination folder
Implement single-file import
Correct mac/windows differences (e.g. ctrl-click vs cmd-click)
Complete the documentation of the extracted data
Correct the text file time tag to show frame number

[edit] Plotting

[edit] MS Excel

To import the text file with data:

File -> Open: then select the file
Original data type: choose "Delimited"
Delimiters: choose "Space"
If you are using an OS with settings other than UK/US you probably use comma as decimal separator. Then you should click on the "Advanced" button and choose to use . (dot) as the decimal separator. Otherwise your data will look like text strings and not numbers.

Choose delimiter

Select "Space"

Choose advanced settings

Select . as decimial separator

There are lots of ways of working with the data in a spreadsheet, and this is beyond this short guide. Often the first thing to do is to start by visualising the data. There are two types of displays that are useful for looking at movement data: time-series and XY-plots.

Time-series line plot

XY plot of position

For more info about plotting you can refer to Excel's help file, or take a look at one of many online tutorials.

[edit] OpenOffice

Open Spreadsheet
On non-English system comma is used as decimal separator, and creates problems. To circumvent this go to Tools -> Options -> Language and choose English as "locale setting".
Open file (File -> Open)
Remember to choose file type "Text CSV"
Choose delimiter "Space"

Select English language

Select Text CSV

Choose "Space" as delimiter

Successful import

Plotting in OpenOffice is as easy as in Excel. Start by selecting the row(s) you want to plot, and then click on the Chart button (or use the menu). Then select the type of plot you want to create.