Probabilistic Prediction of Student Affect from Hand Gestures, A.R. Abbasi, M.N. Dailey, N.V. Afzulpurkar, T. Uno

ABSTRACT: Affective information is vital for effective human-tohuman
communication. Likewise, human-to-computer communication
could be potentiated by an “affective barometer” able to
infer human affect using a machine vision system. For instance,
during a classroom lecture, an affective barometer might provide
useful feedback that a real or virtual instructor could use to
improve pedagogical strategies. In this paper, we explore the
feasibility of using students’ unintentional hand gestures during
a classroom lecture to predict their affective state. We propose
a maximum a posteriori classifier based on a simple Bayesian
network model. We then evaluate the classifier’s ability to predict
one of four affective states from five hand gestures observed in
video recordings of a classroom lecture. Using four-fold cross
validation, we find that the model’s generalization accuracy is
100% over cases where the student reported an affective state,
and 79.4% when we include cases where the student reported
no affective state. The experiment demonstrates that there is a
strong relationship between human affect and visually observable
gestures. Future work will explore the applicability of these
results in practical applications.