View of Computer Vision Research and Challenges for the Fuzzy Set Community by J M Keller, D T Anderson and T Han


1,+James M. Keller, 2,Derek T. Anderson and 1,*Tony Han

1,2Department of Electrical and Computer Engineering

1University of Missouri, USA and 2Mississippi State University, USA,, *


We will discuss challenges in modern computer vision (CV) research and possible directions, tools and novel ideas that the fuzzy set (FS) community may contribute. We discuss each of the difficult problems or challenges in CV that is recognized by researchers in the areas of low level, mid level and high level CV. We review standard and modern CV approaches, discuss data sets currently used by the CV community, and present some FS techniques employed in each area, always with an eye towards where soft computing can make the best impact. This event is not meant to be a survey of all techniques; if yours is left out, please do not get mad. The intent is to provide an assessment, from our perspective, of the power, limitations, and potential of fuzzy algorithms, with thoughts about the challenges for those of us in the FS family to have our technologies accepted by the CV community.

Below is a tentative list of topics and organization. A subset of these topics (marked in blue) will be covered in depth. At the end of this event we will demonstrate a number of live and pre-recorded demos to illustrate topics covered in this workshop/tutorial.


  • Why study CV?

  • CV is steeped in probabilistic methods

  • Active research communities, e.g., PAMI, ICCV, CVPR, ECCV, NIPS, etc.

  • Comparison heavy (methods and data sets)

  • CV tools: OpenCV, SimpleCV and VLFeat

  • Provide open source code: educational and basis for benchmarking

  • David Marr: principles of least commitment and graceful degradation

Low Level CV

  • Sure but ...

  • Edge extraction: FIRE, curvelets/shearlets/etc.

  • Keypoint detection: David Lowe and difference of Gaussians (DoG), maximally stable extremal regions (MSERs), dual-tree complex wavelet transform (DTCWT)

  • Change detection: Gaussian mixture models (GMMs), Eigenbackgrounds, Wallflower, robust principle component analysis (PCA), etc.

  • Image enhancement: Krishnapuram and fuzzy logic

  • Image descriptors: histogram of oriented gradients (HoG), local binary pattern (LBP), pyramid and cell-structured approaches, etc.

  • Linguistic descriptions: color and texture

  • Scoring

Mid Level CV

  • Segmentation: normalized graph cut, mean shift clustering and fuzzy clustering

  • How to label data: existing methods and formats

  • How to score

  • Tracking: mean shift, template matching, Kalman filtering, particle filtering, ensemble tracking, tracking by detection, etc.

High Level CV

  • Def. of object detection, object recognition, people detection and activity recognition

  • Techniques: spatial pyramid matching based on sparse coding (ScSPM), Poselets, discriminatively trained deformable part models, a HOG-LBP human detector with partial occlusion handling, etc.

  • Feature learning: deep structure ANNs

  • Visual bag of words (BoW)

  • Explosive hazard detection, fuzzy measures (FM) and fuzzy integrals (FI)

  • Video surveillance

  • 3D techniques: sensors (stereo vision, Kinect, Lytro light field, etc.), voxel person, linguistic summarization, histogram of oriented normal vector (HONV)

  • Scene interpretation: who, what, when and where (geolocation)?

Pattern Analysis and Decision Making for CV

  • Fuzzy logic

  • Kernel classification

  • Multiple kernels (MK) and MK learning (MKL)

  • Earth movers distance (EMD)

  • Metric learning

Discussion and Conclusion

  • Where does (should) FSs belong in CV?

  • Low level: hard to press advantage

  • Medium: if case can be made for modeling the uncertainty

  • High: biggest payoff, closest to human-like operations

  • Need to demonstrate results on common (large) data sets


  • Live: Human detector demo

  • Live: Kinect object detection demo

  • Live: Mobile vision, feature point localization Android demo

  • Pre-recorded: Voxel person and linguistic summarization for Eldercare

Short Bio's:

James M. Keller holds the University of Missouri Curators Professorship in the Electrical and Computer Engineering and Computer Science Departments on the Columbia campus. He is also the R. L. Tatum Professor in the College of Engineering. His research interests center on computational intelligence with a focus on problems in computer vision, pattern recognition, and information fusion including bioinformatics, spatial reasoning, geospatial intelligence, landmine detection and technology for eldercare.  Professor Keller has coauthored over 400 technical publications. Jim is a Fellow of the IEEE, an IFSA Fellow, and past President of NAFIPS. He received the 2007 Fuzzy Systems Pioneer Award and the 2010 Meritorious Service Award from the IEEE Computational Intelligence Society. He finished a full six year term as Editor-in-Chief of the IEEE Transactions on Fuzzy Systems, followed by being the Vice President for Publications of the IEEE CIS from 2005-2008, and since then an elected CIS Adcom member.  He is the IEEE TAB Transactions Chair and a member of the IEEE Publication Review and Advisory Committee. Jim has had many conference positions and duties over the years.

Derek T. Anderson is an Assistant Professor in Electrical and Computer Engineering at Mississippi State University (MSU). He received a B.S. and M.S. in Computer Science and a PhD in Electrical and Computer Engineering. His research interests include new frontiers in data fusion for pattern analysis and decision making. This includes fuzzy measures and fuzzy integrals, clustering, sensor fusion, remote sensing and computer vision. At MSU, Derek is a member of the signal and image processing group. He received the Best Student Paper Award at FUZZ-IEEE 2008 and the Best Paper Award at FUZZ-IEEE 2012. Derek has received funding from DARPA, the National Institute of Justice (NIJ), the Leonard Wood Institute, and he has been a subcontractor on an Army Research Office (ARO) project. He has published 11 journal articles, 35 conference proceedings and he is an Associate Editor for the IEEE Trans. Fuzzy Systems. Derek has co-chair special sessions at WCCI, "Computational Intelligence for Activity Recognition from Sensed Data" (2011) and "Computational Intelligence for Security, Surveillance and Defense" (2012).

Tony X. Han is an Assistant Professor of Electrical & Computer Engineering at the University of Missouri (MU). He received his Ph.D. degree in Electrical and Computer Engineering from the University of Illinois at Urban-Champaign in 2007. Dr. Han’s specialties lie in computer vision and machine learning, with emphasis on human/object detection, large scale image retrieval, object tracking, action recognition, video analysis, and biometrics. His research team is a joint winner of the action recognition task in the worldwide grand challenge PASCAL 2010. The human detector developed in his group is ranked 2nd in the worldwide grand challenge PASCAL 2009 and 2012. His research team together with UIUC joint team also won the first place in Facial Expression Recognition and Analysis Challenge (FERA), 2011. He is recipient of CSE fellowship. He gratefully acknowledges research funding provided by the National Science Foundation, the National Geo-spatial Intelligence Agency, the Leonard Woods Institute, as well as the University of Missouri.


Technical Sponsor

Gold Sponsor

Silver Sponsor

Technical Co-sponsor

Electronics and Communication Sciences Unit, Indian Statstical Institute, Kolkata