Deep learning based beat event detection in action movie franchises

N. Ejaz; U. A. Khan; M. A. Martínez-del-Amor; H. Sparenberg

doi:10.1117/12.2309629

13 April 2018 Deep learning based beat event detection in action movie franchises

N. Ejaz, U. A. Khan, M. A. Martínez-del-Amor, H. Sparenberg

Proceedings Volume 10696, Tenth International Conference on Machine Vision (ICMV 2017); 1069608 (2018) https://doi.org/10.1117/12.2309629
Event: Tenth International Conference on Machine Vision, 2017, Vienna, Austria

Abstract

Automatic understanding and interpretation of movies can be used in a variety of ways to semantically manage the massive volumes of movies data. “Action Movie Franchises” dataset is a collection of twenty Hollywood action movies from five famous franchises with ground truth annotations at shot and beat level of each movie. In this dataset, the annotations are provided for eleven semantic beat categories. In this work, we propose a deep learning based method to classify shots and beat-events on this dataset. The training dataset for each of the eleven beat categories is developed and then a Convolution Neural Network is trained. After finding the shot boundaries, key frames are extracted for each shot and then three classification labels are assigned to each key frame. The classification labels for each of the key frames in a particular shot are then used to assign a unique label to each shot. A simple sliding window based method is then used to group adjacent shots having the same label in order to find a particular beat event. The results of beat event classification are presented based on criteria of precision, recall, and F-measure. The results are compared with the existing technique and significant improvements are recorded.

Citation Download Citation

N. Ejaz, U. A. Khan, M. A. Martínez-del-Amor, and H. Sparenberg "Deep learning based beat event detection in action movie franchises", Proc. SPIE 10696, Tenth International Conference on Machine Vision (ICMV 2017), 1069608 (13 April 2018); https://doi.org/10.1117/12.2309629

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available