This is the real time ball tracking class responsible for detecting balls from the buffer filled in by kinect and pushing the tracked points into the ballpoints processing instance for processing later. More...

#include <BallTrackingUtility.h>

Collaboration diagram for BallTrackingUtility:

List of all members.

Public Slots
void	setBinaryLowerThresh (double _lowerThresh)
	All the following functions are setter functions determining the imapge processing efficiency and accuracy.
void	setStructuringElementSize (int _size)
void	setBlurSize (int _size)
void	setCannyLowerThresh (double _lowerThresh)
void	setCannyHigherThresh (double _higherThresh)
void	setNumerOfDialtions (int _numberOfDialtions)
void	setContourAreaThresh (double _contourAreaThresh)
void	setDifferenceInAreaThresh (double _diffAreaThresh)
void	setMasterRigCheck (int _masterRigCheck)
void	trackBall ()
	main function called everytime update timer expires to track the ball using kinect's image buffer
Public Member Functions
	BallTrackingUtility ()
	default constructor
	BallTrackingUtility (BallPointsProcessing *_inputBallProcessingUtility)
	param constructor taking in ballpointsprocessing instance
	~BallTrackingUtility ()
	destor
Private Member Functions
void	mouseMoveEvent (QMouseEvent *_event)
	this method is called every time a mouse is moved
void	mousePressEvent (QMouseEvent *_event)
	this method is called everytime the mouse button is pressed inherited from QObject and overridden here.
void	mouseReleaseEvent (QMouseEvent *_event)
	this method is called everytime the mouse button is released inherited from QObject and overridden here.
void	timerEvent (QTimerEvent *_event)
	called when the timer is triggered
Private Attributes
cv::Mat	m_rgb
	rgb and gray buffers filled in from kinect
cv::Mat	m_gray
cv::Mat	m_nextBuffer
	individual buffers used to store temporary results while image processing
cv::Mat	m_prevBuffer
cv::Mat	m_diffBuffer
cv::Mat	m_bufferDepthRaw
cv::Mat	m_bufferDepthRaw1
float	m_depthinm [640 *480]
	depth value buffer in meters
std::vector< cv::Moments >	m_momentList
	moment list stores the opencv moments of the tracked points
cv::Mat	canny_output
	canny edges detected by opencv module is stored into this mat buffer
std::vector< std::vector < cv::Point > >	contours
	opencv findcontours stores the contours in this multi dimensional data structure
cv::Mat	hierarchy
	opencv findcontours stores the retrieved hierarachy into this buffer
double	area
	area is calculated from moments and rect area is the area of the rotated rect placed around the tracked pts
double	rectArea
std::vector< cv::Moments >	mu
	mu is populated from moments
std::vector< cv::Point2f >	mc
	mc is moment centers populated from moments
double	m_binaryLowerThresh
	various setup values set from the UI that are used in various steps of image processing algorithms. These values define how well the ball would be detected from the recieved frame
double	m_binaryMaxVal
double	m_cannyLowerThresh
double	m_cannyHigherThresh
double	m_contourAreaThresh
double	m_diffAreaThresh
int	m_ellipsesize
	various setup values set from the UI that are used in various steps of image processing algorithms. These values define how well the ball would be detected from the recieved frame
int	m_blurSize
int	m_noOfTimesToDialate
int	m_masterRigCheck
QTimer *	m_updateTimer
	timer to call the processing module in equal interval of time
BallPointsProcessing *	m_ballPointProcessing
	ballpointprocessing instance set through the constr to call the set function to push in tracked pts

Detailed Description

This is the real time ball tracking class responsible for detecting balls from the buffer filled in by kinect and pushing the tracked points into the ballpoints processing instance for processing later.

Definition at line 30 of file BallTrackingUtility.h.

Constructor & Destructor Documentation

BallTrackingUtility::BallTrackingUtility ( )

default constructor

Definition at line 29 of file BallTrackingUtility.cpp.

{

}

BallTrackingUtility::BallTrackingUtility ( BallPointsProcessing * _inputBallProcessingUtility )

param constructor taking in ballpointsprocessing instance

Parameters:

[in] _inputBallProcessingUtility instance

Definition at line 35 of file BallTrackingUtility.cpp.

References area, m_binaryLowerThresh, m_binaryMaxVal, m_blurSize, m_bufferDepthRaw, m_bufferDepthRaw1, m_cannyHigherThresh, m_cannyLowerThresh, m_contourAreaThresh, m_depthinm, m_diffAreaThresh, m_diffBuffer, m_ellipsesize, m_gray, m_masterRigCheck, m_momentList, m_nextBuffer, m_noOfTimesToDialate, m_prevBuffer, m_rgb, m_updateTimer, rectArea, and trackBall().

                                                                                         :m_ballPointProcessing(_inputBallProcessingUtility)

{

  m_rgb = cvCreateMat(480,640,CV_8UC3);

  m_gray = cvCreateMat(480,640,CV_8UC1);

  m_nextBuffer = cvCreateMat(480,640,CV_8UC1);
  m_prevBuffer = cvCreateMat(480,640,CV_8UC1);
  m_diffBuffer = cvCreateMat(480,640,CV_8UC1);

  m_bufferDepthRaw=cvCreateMat(480,640,CV_8UC1);
  m_bufferDepthRaw1=cvCreateMat(480,640,CV_8UC1);

  m_binaryLowerThresh = 100.0;
  m_binaryMaxVal = 255.0;
  m_cannyLowerThresh = 100.0;
  m_cannyHigherThresh = 300.0;
  m_contourAreaThresh = 120.0;
  m_diffAreaThresh = 30.0;
  m_ellipsesize = 2;
  m_blurSize = 5;
  m_noOfTimesToDialate = 2;

  m_masterRigCheck = 0;

  for( unsigned int i = 0 ; i < 640*480 ; ++i)
  {
      m_depthinm[i] = 0;
  }

  // all these are separate lists filled by separate calls to interpolate functions
  m_momentList.clear();

  area = rectArea = 0.0;
  m_updateTimer = new QTimer(this);

  connect(m_updateTimer, SIGNAL(timeout()), this, SLOT(trackBall()));
  m_updateTimer->start(1);

}

Here is the call graph for this function:

BallTrackingUtility::~BallTrackingUtility ( )

destor

Definition at line 212 of file BallTrackingUtility.cpp.

{

}

Member Function Documentation

void BallTrackingUtility::mouseMoveEvent ( QMouseEvent * _event ) [private]

this method is called every time a mouse is moved

Parameters:

_event the Qt Event structure

Definition at line 179 of file BallTrackingUtility.cpp.

{
    Q_UNUSED(_event);
}

void BallTrackingUtility::mousePressEvent ( QMouseEvent * _event ) [private]

this method is called everytime the mouse button is pressed inherited from QObject and overridden here.

Parameters:

_event the Qt Event structure

Definition at line 186 of file BallTrackingUtility.cpp.

{
    Q_UNUSED(_event);

}

void BallTrackingUtility::mouseReleaseEvent ( QMouseEvent * _event ) [private]

this method is called everytime the mouse button is released inherited from QObject and overridden here.

Parameters:

_event the Qt Event structure

Definition at line 193 of file BallTrackingUtility.cpp.

{
  // this event is called when the mouse button is released
  // we then set Rotate to false
  Q_UNUSED(_event);

}

void BallTrackingUtility::setBinaryLowerThresh ( double _lowerThresh ) [slot]

All the following functions are setter functions determining the imapge processing efficiency and accuracy.

Definition at line 220 of file BallTrackingUtility.cpp.

References m_binaryLowerThresh.

{
    m_binaryLowerThresh = _lowerThresh;
    //updateGL();
}

void BallTrackingUtility::setBlurSize ( int _size ) [slot]

Definition at line 232 of file BallTrackingUtility.cpp.

References m_blurSize.

{
    m_blurSize = _size;
    //updateGL();
}

void BallTrackingUtility::setCannyHigherThresh ( double _higherThresh ) [slot]

Definition at line 245 of file BallTrackingUtility.cpp.

References m_cannyHigherThresh.

{
    m_cannyHigherThresh = _higherThresh;
    //updateGL();
}

void BallTrackingUtility::setCannyLowerThresh ( double _lowerThresh ) [slot]

Definition at line 238 of file BallTrackingUtility.cpp.

References m_cannyLowerThresh.

{
    m_cannyLowerThresh = _lowerThresh;
    //updateGL();
}

void BallTrackingUtility::setContourAreaThresh ( double _contourAreaThresh ) [slot]

Definition at line 258 of file BallTrackingUtility.cpp.

References m_contourAreaThresh.

{
    m_contourAreaThresh = _contourAreaThresh;
    //updateGL();
}

void BallTrackingUtility::setDifferenceInAreaThresh ( double _diffAreaThresh ) [slot]

Definition at line 264 of file BallTrackingUtility.cpp.

References m_diffAreaThresh.

{
    m_diffAreaThresh = _diffAreaThresh;
    //updateGL();
}

void BallTrackingUtility::setMasterRigCheck ( int _masterRigCheck ) [slot]

Definition at line 270 of file BallTrackingUtility.cpp.

References m_masterRigCheck.

{
    m_masterRigCheck = _masterRigCheck;
    //updateGL();
}

void BallTrackingUtility::setNumerOfDialtions ( int _numberOfDialtions ) [slot]

Definition at line 252 of file BallTrackingUtility.cpp.

References m_noOfTimesToDialate.

{
    m_noOfTimesToDialate = _numberOfDialtions;
    //updateGL();
}

void BallTrackingUtility::setStructuringElementSize ( int _size ) [slot]

Definition at line 226 of file BallTrackingUtility.cpp.

References m_ellipsesize.

{
    m_ellipsesize = _size;
    //updateGL();
}

void BallTrackingUtility::timerEvent ( QTimerEvent * _event ) [private]

called when the timer is triggered

Definition at line 201 of file BallTrackingUtility.cpp.

References trackBall().

{
    Q_UNUSED(_event);
    // re-draw GL
    //updateGL();

    trackBall();

}

Here is the call graph for this function:

void BallTrackingUtility::trackBall ( ) [slot]

main function called everytime update timer expires to track the ball using kinect's image buffer

Detect edges using canny

Find contours

Definition at line 82 of file BallTrackingUtility.cpp.

References area, canny_output, contours, KinectInterface::getBoxObject(), KinectInterface::getDepthRaw(), KinectInterface::getTrackingFlag(), hierarchy, KinectInterface::instance(), m_ballPointProcessing, m_binaryLowerThresh, m_binaryMaxVal, m_blurSize, m_bufferDepthRaw, m_bufferDepthRaw1, m_cannyHigherThresh, m_cannyLowerThresh, m_contourAreaThresh, m_depthinm, m_diffAreaThresh, m_diffBuffer, m_ellipsesize, m_gray, m_momentList, m_nextBuffer, m_noOfTimesToDialate, m_prevBuffer, and BallPointsProcessing::setTrackedPoints().

{


    KinectInterface *kinect=KinectInterface::instance();

    cv::Rect _boundsCheck = kinect->getBoxObject();

 if(kinect->getTrackingFlag())
 {
    static int _frameCount = 1;

    float depth = 0.0;

    kinect->getDepthRaw(m_bufferDepthRaw,m_depthinm);

    cv::blur(m_bufferDepthRaw,m_bufferDepthRaw1,cv::Size(m_blurSize,m_blurSize));
    // to reduce noise, smotthing has been done

    if(_frameCount == 1)
    {
        m_bufferDepthRaw1.copyTo(m_prevBuffer);
    }

    m_bufferDepthRaw1.copyTo(m_nextBuffer);
    cv::absdiff(m_nextBuffer,m_prevBuffer,m_diffBuffer);
    // has gpu variant // bg subtraction by frame differencing ...GMM method seems good through forums if at all we need
    m_nextBuffer.copyTo(m_prevBuffer);
    m_diffBuffer.copyTo(m_gray);

    cv::threshold(m_gray,m_gray,m_binaryLowerThresh,m_binaryMaxVal,CV_THRESH_BINARY_INV);
    cv::Mat element = cv::getStructuringElement(cv::MORPH_ELLIPSE,cv::Size(m_ellipsesize,m_ellipsesize));
    cv::dilate(m_gray,m_gray,element,cv::Point(-1,-1),m_noOfTimesToDialate);
    cv::erode(m_gray,m_gray,element,cv::Point(-1,-1),m_noOfTimesToDialate);

    cv::Canny( m_gray, canny_output, m_cannyLowerThresh, m_cannyHigherThresh, 3 ); // maintiain upper::lower ratio as 2:1 or 3:1 for better results
                                                                                   // as suggested by CANNY
    cv::findContours( canny_output, contours, hierarchy, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0) ); // methods to be decided


    for( int i = 0; i < contours.size(); i++ )
    {
        /* cv::drawContours( drawing, contours, i, cv::Scalar(255,0,0),-1, 8, hierarchy, 0, cv::Point() );*/ // -- DEV PURPOSE
        area = cv::contourArea(contours[i], false);
        cv::RotatedRect r = cv::minAreaRect(contours[i]);

        if (fabs(area) < m_contourAreaThresh) //my notes: this value is more important in reducing the
            //false positives and filters out those false positives by preventing them
            // from getting processed further..provide in UI
        {
            continue;
        }
        else
        {
            // do nothing
        }

       if (fabs(fabs(area) - (r.size.width * r.size.height * 3.14 / 4)) < m_diffAreaThresh)
       {
           m_momentList.push_back(cv::moments( contours[i], false ));
           float _xMomentsTemp = static_cast<float>(cv::moments( contours[i], false ).m10/cv::moments( contours[i], false ).m00);
           float _yMomentsTemp = static_cast<float>(cv::moments( contours[i], false ).m01/cv::moments( contours[i], false ).m00);
           int depthIndex = (int)((640 - _xMomentsTemp) + (640 * _yMomentsTemp));
           depth = m_depthinm[depthIndex] * 0.001;

           if(depth != 0)
           {
            if((_xMomentsTemp >= _boundsCheck.x) && (_xMomentsTemp <= _boundsCheck.x + _boundsCheck.width) &&
                    (_yMomentsTemp >= _boundsCheck.y) && (_yMomentsTemp <= _boundsCheck.y + _boundsCheck.height))
            {


                std::cout<<"XMoments:"<<_xMomentsTemp<<"\n";
                m_ballPointProcessing->setTrackedPoints(cv::Point3f(_xMomentsTemp,_yMomentsTemp,depth));

             }
            }

          }
        }


        _frameCount++;
  }
 else
 {
     // tracking not yet started...do nothing


 }


}

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

double BallTrackingUtility::area [private]

area is calculated from moments and rect area is the area of the rotated rect placed around the tracked pts

Definition at line 74 of file BallTrackingUtility.h.

cv::Mat BallTrackingUtility::canny_output [private]

canny edges detected by opencv module is stored into this mat buffer

Definition at line 59 of file BallTrackingUtility.h.

std::vector<std::vector<cv::Point> > BallTrackingUtility::contours [private]

opencv findcontours stores the contours in this multi dimensional data structure

Definition at line 64 of file BallTrackingUtility.h.

cv::Mat BallTrackingUtility::hierarchy [private]

opencv findcontours stores the retrieved hierarachy into this buffer

Definition at line 69 of file BallTrackingUtility.h.

BallPointsProcessing* BallTrackingUtility::m_ballPointProcessing [private]

ballpointprocessing instance set through the constr to call the set function to push in tracked pts

Definition at line 106 of file BallTrackingUtility.h.

double BallTrackingUtility::m_binaryLowerThresh [private]

various setup values set from the UI that are used in various steps of image processing algorithms. These values define how well the ball would be detected from the recieved frame

Definition at line 90 of file BallTrackingUtility.h.

double BallTrackingUtility::m_binaryMaxVal [private]

Definition at line 90 of file BallTrackingUtility.h.

int BallTrackingUtility::m_blurSize [private]

Definition at line 96 of file BallTrackingUtility.h.

cv::Mat BallTrackingUtility::m_bufferDepthRaw [private]

Definition at line 44 of file BallTrackingUtility.h.

cv::Mat BallTrackingUtility::m_bufferDepthRaw1 [private]

Definition at line 44 of file BallTrackingUtility.h.

double BallTrackingUtility::m_cannyHigherThresh [private]

Definition at line 90 of file BallTrackingUtility.h.

double BallTrackingUtility::m_cannyLowerThresh [private]

Definition at line 90 of file BallTrackingUtility.h.

double BallTrackingUtility::m_contourAreaThresh [private]

Definition at line 90 of file BallTrackingUtility.h.

float BallTrackingUtility::m_depthinm[640 *480] [private]

depth value buffer in meters

Definition at line 49 of file BallTrackingUtility.h.

double BallTrackingUtility::m_diffAreaThresh [private]

Definition at line 90 of file BallTrackingUtility.h.

cv::Mat BallTrackingUtility::m_diffBuffer [private]

Definition at line 44 of file BallTrackingUtility.h.

int BallTrackingUtility::m_ellipsesize [private]

various setup values set from the UI that are used in various steps of image processing algorithms. These values define how well the ball would be detected from the recieved frame

Definition at line 96 of file BallTrackingUtility.h.

cv::Mat BallTrackingUtility::m_gray [private]

Definition at line 39 of file BallTrackingUtility.h.

int BallTrackingUtility::m_masterRigCheck [private]

Definition at line 96 of file BallTrackingUtility.h.

std::vector<cv::Moments> BallTrackingUtility::m_momentList [private]

moment list stores the opencv moments of the tracked points

Definition at line 54 of file BallTrackingUtility.h.

cv::Mat BallTrackingUtility::m_nextBuffer [private]

individual buffers used to store temporary results while image processing

Definition at line 44 of file BallTrackingUtility.h.

int BallTrackingUtility::m_noOfTimesToDialate [private]