KINECT STATS GENERATOR FOR SPORTS VISUALISATION
1.0
|
This is the real time ball tracking class responsible for detecting balls from the buffer filled in by kinect and pushing the tracked points into the ballpoints processing instance for processing later. More...
#include <BallTrackingUtility.h>
Public Slots | |
void | setBinaryLowerThresh (double _lowerThresh) |
All the following functions are setter functions determining the imapge processing efficiency and accuracy. | |
void | setStructuringElementSize (int _size) |
void | setBlurSize (int _size) |
void | setCannyLowerThresh (double _lowerThresh) |
void | setCannyHigherThresh (double _higherThresh) |
void | setNumerOfDialtions (int _numberOfDialtions) |
void | setContourAreaThresh (double _contourAreaThresh) |
void | setDifferenceInAreaThresh (double _diffAreaThresh) |
void | setMasterRigCheck (int _masterRigCheck) |
void | trackBall () |
main function called everytime update timer expires to track the ball using kinect's image buffer | |
Public Member Functions | |
BallTrackingUtility () | |
default constructor | |
BallTrackingUtility (BallPointsProcessing *_inputBallProcessingUtility) | |
param constructor taking in ballpointsprocessing instance | |
~BallTrackingUtility () | |
destor | |
Private Member Functions | |
void | mouseMoveEvent (QMouseEvent *_event) |
this method is called every time a mouse is moved | |
void | mousePressEvent (QMouseEvent *_event) |
this method is called everytime the mouse button is pressed inherited from QObject and overridden here. | |
void | mouseReleaseEvent (QMouseEvent *_event) |
this method is called everytime the mouse button is released inherited from QObject and overridden here. | |
void | timerEvent (QTimerEvent *_event) |
called when the timer is triggered | |
Private Attributes | |
cv::Mat | m_rgb |
rgb and gray buffers filled in from kinect | |
cv::Mat | m_gray |
cv::Mat | m_nextBuffer |
individual buffers used to store temporary results while image processing | |
cv::Mat | m_prevBuffer |
cv::Mat | m_diffBuffer |
cv::Mat | m_bufferDepthRaw |
cv::Mat | m_bufferDepthRaw1 |
float | m_depthinm [640 *480] |
depth value buffer in meters | |
std::vector< cv::Moments > | m_momentList |
moment list stores the opencv moments of the tracked points | |
cv::Mat | canny_output |
canny edges detected by opencv module is stored into this mat buffer | |
std::vector< std::vector < cv::Point > > | contours |
opencv findcontours stores the contours in this multi dimensional data structure | |
cv::Mat | hierarchy |
opencv findcontours stores the retrieved hierarachy into this buffer | |
double | area |
area is calculated from moments and rect area is the area of the rotated rect placed around the tracked pts | |
double | rectArea |
std::vector< cv::Moments > | mu |
mu is populated from moments | |
std::vector< cv::Point2f > | mc |
mc is moment centers populated from moments | |
double | m_binaryLowerThresh |
various setup values set from the UI that are used in various steps of image processing algorithms. These values define how well the ball would be detected from the recieved frame | |
double | m_binaryMaxVal |
double | m_cannyLowerThresh |
double | m_cannyHigherThresh |
double | m_contourAreaThresh |
double | m_diffAreaThresh |
int | m_ellipsesize |
various setup values set from the UI that are used in various steps of image processing algorithms. These values define how well the ball would be detected from the recieved frame | |
int | m_blurSize |
int | m_noOfTimesToDialate |
int | m_masterRigCheck |
QTimer * | m_updateTimer |
timer to call the processing module in equal interval of time | |
BallPointsProcessing * | m_ballPointProcessing |
ballpointprocessing instance set through the constr to call the set function to push in tracked pts |
This is the real time ball tracking class responsible for detecting balls from the buffer filled in by kinect and pushing the tracked points into the ballpoints processing instance for processing later.
Definition at line 30 of file BallTrackingUtility.h.
BallTrackingUtility::BallTrackingUtility | ( | BallPointsProcessing * | _inputBallProcessingUtility | ) |
param constructor taking in ballpointsprocessing instance
[in] | _inputBallProcessingUtility | instance |
Definition at line 35 of file BallTrackingUtility.cpp.
References area, m_binaryLowerThresh, m_binaryMaxVal, m_blurSize, m_bufferDepthRaw, m_bufferDepthRaw1, m_cannyHigherThresh, m_cannyLowerThresh, m_contourAreaThresh, m_depthinm, m_diffAreaThresh, m_diffBuffer, m_ellipsesize, m_gray, m_masterRigCheck, m_momentList, m_nextBuffer, m_noOfTimesToDialate, m_prevBuffer, m_rgb, m_updateTimer, rectArea, and trackBall().
:m_ballPointProcessing(_inputBallProcessingUtility) { m_rgb = cvCreateMat(480,640,CV_8UC3); m_gray = cvCreateMat(480,640,CV_8UC1); m_nextBuffer = cvCreateMat(480,640,CV_8UC1); m_prevBuffer = cvCreateMat(480,640,CV_8UC1); m_diffBuffer = cvCreateMat(480,640,CV_8UC1); m_bufferDepthRaw=cvCreateMat(480,640,CV_8UC1); m_bufferDepthRaw1=cvCreateMat(480,640,CV_8UC1); m_binaryLowerThresh = 100.0; m_binaryMaxVal = 255.0; m_cannyLowerThresh = 100.0; m_cannyHigherThresh = 300.0; m_contourAreaThresh = 120.0; m_diffAreaThresh = 30.0; m_ellipsesize = 2; m_blurSize = 5; m_noOfTimesToDialate = 2; m_masterRigCheck = 0; for( unsigned int i = 0 ; i < 640*480 ; ++i) { m_depthinm[i] = 0; } // all these are separate lists filled by separate calls to interpolate functions m_momentList.clear(); area = rectArea = 0.0; m_updateTimer = new QTimer(this); connect(m_updateTimer, SIGNAL(timeout()), this, SLOT(trackBall())); m_updateTimer->start(1); }
void BallTrackingUtility::mouseMoveEvent | ( | QMouseEvent * | _event | ) | [private] |
this method is called every time a mouse is moved
_event | the Qt Event structure |
Definition at line 179 of file BallTrackingUtility.cpp.
{ Q_UNUSED(_event); }
void BallTrackingUtility::mousePressEvent | ( | QMouseEvent * | _event | ) | [private] |
this method is called everytime the mouse button is pressed inherited from QObject and overridden here.
_event | the Qt Event structure |
Definition at line 186 of file BallTrackingUtility.cpp.
{ Q_UNUSED(_event); }
void BallTrackingUtility::mouseReleaseEvent | ( | QMouseEvent * | _event | ) | [private] |
this method is called everytime the mouse button is released inherited from QObject and overridden here.
_event | the Qt Event structure |
Definition at line 193 of file BallTrackingUtility.cpp.
{ // this event is called when the mouse button is released // we then set Rotate to false Q_UNUSED(_event); }
void BallTrackingUtility::setBinaryLowerThresh | ( | double | _lowerThresh | ) | [slot] |
All the following functions are setter functions determining the imapge processing efficiency and accuracy.
Definition at line 220 of file BallTrackingUtility.cpp.
References m_binaryLowerThresh.
{ m_binaryLowerThresh = _lowerThresh; //updateGL(); }
void BallTrackingUtility::setBlurSize | ( | int | _size | ) | [slot] |
Definition at line 232 of file BallTrackingUtility.cpp.
References m_blurSize.
{ m_blurSize = _size; //updateGL(); }
void BallTrackingUtility::setCannyHigherThresh | ( | double | _higherThresh | ) | [slot] |
Definition at line 245 of file BallTrackingUtility.cpp.
References m_cannyHigherThresh.
{ m_cannyHigherThresh = _higherThresh; //updateGL(); }
void BallTrackingUtility::setCannyLowerThresh | ( | double | _lowerThresh | ) | [slot] |
Definition at line 238 of file BallTrackingUtility.cpp.
References m_cannyLowerThresh.
{ m_cannyLowerThresh = _lowerThresh; //updateGL(); }
void BallTrackingUtility::setContourAreaThresh | ( | double | _contourAreaThresh | ) | [slot] |
Definition at line 258 of file BallTrackingUtility.cpp.
References m_contourAreaThresh.
{ m_contourAreaThresh = _contourAreaThresh; //updateGL(); }
void BallTrackingUtility::setDifferenceInAreaThresh | ( | double | _diffAreaThresh | ) | [slot] |
Definition at line 264 of file BallTrackingUtility.cpp.
References m_diffAreaThresh.
{ m_diffAreaThresh = _diffAreaThresh; //updateGL(); }
void BallTrackingUtility::setMasterRigCheck | ( | int | _masterRigCheck | ) | [slot] |
Definition at line 270 of file BallTrackingUtility.cpp.
References m_masterRigCheck.
{ m_masterRigCheck = _masterRigCheck; //updateGL(); }
void BallTrackingUtility::setNumerOfDialtions | ( | int | _numberOfDialtions | ) | [slot] |
Definition at line 252 of file BallTrackingUtility.cpp.
References m_noOfTimesToDialate.
{ m_noOfTimesToDialate = _numberOfDialtions; //updateGL(); }
void BallTrackingUtility::setStructuringElementSize | ( | int | _size | ) | [slot] |
Definition at line 226 of file BallTrackingUtility.cpp.
References m_ellipsesize.
{ m_ellipsesize = _size; //updateGL(); }
void BallTrackingUtility::timerEvent | ( | QTimerEvent * | _event | ) | [private] |
called when the timer is triggered
Definition at line 201 of file BallTrackingUtility.cpp.
References trackBall().
{ Q_UNUSED(_event); // re-draw GL //updateGL(); trackBall(); }
void BallTrackingUtility::trackBall | ( | ) | [slot] |
main function called everytime update timer expires to track the ball using kinect's image buffer
Detect edges using canny
Find contours
Definition at line 82 of file BallTrackingUtility.cpp.
References area, canny_output, contours, KinectInterface::getBoxObject(), KinectInterface::getDepthRaw(), KinectInterface::getTrackingFlag(), hierarchy, KinectInterface::instance(), m_ballPointProcessing, m_binaryLowerThresh, m_binaryMaxVal, m_blurSize, m_bufferDepthRaw, m_bufferDepthRaw1, m_cannyHigherThresh, m_cannyLowerThresh, m_contourAreaThresh, m_depthinm, m_diffAreaThresh, m_diffBuffer, m_ellipsesize, m_gray, m_momentList, m_nextBuffer, m_noOfTimesToDialate, m_prevBuffer, and BallPointsProcessing::setTrackedPoints().
{ KinectInterface *kinect=KinectInterface::instance(); cv::Rect _boundsCheck = kinect->getBoxObject(); if(kinect->getTrackingFlag()) { static int _frameCount = 1; float depth = 0.0; kinect->getDepthRaw(m_bufferDepthRaw,m_depthinm); cv::blur(m_bufferDepthRaw,m_bufferDepthRaw1,cv::Size(m_blurSize,m_blurSize)); // to reduce noise, smotthing has been done if(_frameCount == 1) { m_bufferDepthRaw1.copyTo(m_prevBuffer); } m_bufferDepthRaw1.copyTo(m_nextBuffer); cv::absdiff(m_nextBuffer,m_prevBuffer,m_diffBuffer); // has gpu variant // bg subtraction by frame differencing ...GMM method seems good through forums if at all we need m_nextBuffer.copyTo(m_prevBuffer); m_diffBuffer.copyTo(m_gray); cv::threshold(m_gray,m_gray,m_binaryLowerThresh,m_binaryMaxVal,CV_THRESH_BINARY_INV); cv::Mat element = cv::getStructuringElement(cv::MORPH_ELLIPSE,cv::Size(m_ellipsesize,m_ellipsesize)); cv::dilate(m_gray,m_gray,element,cv::Point(-1,-1),m_noOfTimesToDialate); cv::erode(m_gray,m_gray,element,cv::Point(-1,-1),m_noOfTimesToDialate); cv::Canny( m_gray, canny_output, m_cannyLowerThresh, m_cannyHigherThresh, 3 ); // maintiain upper::lower ratio as 2:1 or 3:1 for better results // as suggested by CANNY cv::findContours( canny_output, contours, hierarchy, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0) ); // methods to be decided for( int i = 0; i < contours.size(); i++ ) { /* cv::drawContours( drawing, contours, i, cv::Scalar(255,0,0),-1, 8, hierarchy, 0, cv::Point() );*/ // -- DEV PURPOSE area = cv::contourArea(contours[i], false); cv::RotatedRect r = cv::minAreaRect(contours[i]); if (fabs(area) < m_contourAreaThresh) //my notes: this value is more important in reducing the //false positives and filters out those false positives by preventing them // from getting processed further..provide in UI { continue; } else { // do nothing } if (fabs(fabs(area) - (r.size.width * r.size.height * 3.14 / 4)) < m_diffAreaThresh) { m_momentList.push_back(cv::moments( contours[i], false )); float _xMomentsTemp = static_cast<float>(cv::moments( contours[i], false ).m10/cv::moments( contours[i], false ).m00); float _yMomentsTemp = static_cast<float>(cv::moments( contours[i], false ).m01/cv::moments( contours[i], false ).m00); int depthIndex = (int)((640 - _xMomentsTemp) + (640 * _yMomentsTemp)); depth = m_depthinm[depthIndex] * 0.001; if(depth != 0) { if((_xMomentsTemp >= _boundsCheck.x) && (_xMomentsTemp <= _boundsCheck.x + _boundsCheck.width) && (_yMomentsTemp >= _boundsCheck.y) && (_yMomentsTemp <= _boundsCheck.y + _boundsCheck.height)) { std::cout<<"XMoments:"<<_xMomentsTemp<<"\n"; m_ballPointProcessing->setTrackedPoints(cv::Point3f(_xMomentsTemp,_yMomentsTemp,depth)); } } } } _frameCount++; } else { // tracking not yet started...do nothing } }
double BallTrackingUtility::area [private] |
area is calculated from moments and rect area is the area of the rotated rect placed around the tracked pts
Definition at line 74 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::canny_output [private] |
canny edges detected by opencv module is stored into this mat buffer
Definition at line 59 of file BallTrackingUtility.h.
std::vector<std::vector<cv::Point> > BallTrackingUtility::contours [private] |
opencv findcontours stores the contours in this multi dimensional data structure
Definition at line 64 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::hierarchy [private] |
opencv findcontours stores the retrieved hierarachy into this buffer
Definition at line 69 of file BallTrackingUtility.h.
ballpointprocessing instance set through the constr to call the set function to push in tracked pts
Definition at line 106 of file BallTrackingUtility.h.
double BallTrackingUtility::m_binaryLowerThresh [private] |
various setup values set from the UI that are used in various steps of image processing algorithms. These values define how well the ball would be detected from the recieved frame
Definition at line 90 of file BallTrackingUtility.h.
double BallTrackingUtility::m_binaryMaxVal [private] |
Definition at line 90 of file BallTrackingUtility.h.
int BallTrackingUtility::m_blurSize [private] |
Definition at line 96 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::m_bufferDepthRaw [private] |
Definition at line 44 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::m_bufferDepthRaw1 [private] |
Definition at line 44 of file BallTrackingUtility.h.
double BallTrackingUtility::m_cannyHigherThresh [private] |
Definition at line 90 of file BallTrackingUtility.h.
double BallTrackingUtility::m_cannyLowerThresh [private] |
Definition at line 90 of file BallTrackingUtility.h.
double BallTrackingUtility::m_contourAreaThresh [private] |
Definition at line 90 of file BallTrackingUtility.h.
float BallTrackingUtility::m_depthinm[640 *480] [private] |
depth value buffer in meters
Definition at line 49 of file BallTrackingUtility.h.
double BallTrackingUtility::m_diffAreaThresh [private] |
Definition at line 90 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::m_diffBuffer [private] |
Definition at line 44 of file BallTrackingUtility.h.
int BallTrackingUtility::m_ellipsesize [private] |
various setup values set from the UI that are used in various steps of image processing algorithms. These values define how well the ball would be detected from the recieved frame
Definition at line 96 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::m_gray [private] |
Definition at line 39 of file BallTrackingUtility.h.
int BallTrackingUtility::m_masterRigCheck [private] |
Definition at line 96 of file BallTrackingUtility.h.
std::vector<cv::Moments> BallTrackingUtility::m_momentList [private] |
moment list stores the opencv moments of the tracked points
Definition at line 54 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::m_nextBuffer [private] |
individual buffers used to store temporary results while image processing
Definition at line 44 of file BallTrackingUtility.h.
int BallTrackingUtility::m_noOfTimesToDialate [private] |
Definition at line 96 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::m_prevBuffer [private] |
Definition at line 44 of file BallTrackingUtility.h.
cv::Mat BallTrackingUtility::m_rgb [private] |
rgb and gray buffers filled in from kinect
Definition at line 39 of file BallTrackingUtility.h.
QTimer* BallTrackingUtility::m_updateTimer [private] |
timer to call the processing module in equal interval of time
Definition at line 101 of file BallTrackingUtility.h.
std::vector<cv::Point2f> BallTrackingUtility::mc [private] |
mc is moment centers populated from moments
Definition at line 84 of file BallTrackingUtility.h.
std::vector<cv::Moments> BallTrackingUtility::mu [private] |
mu is populated from moments
Definition at line 79 of file BallTrackingUtility.h.
double BallTrackingUtility::rectArea [private] |
Definition at line 74 of file BallTrackingUtility.h.