Ball track visualization as green circles just for visualization of real time tracking This is enabled when the user is in setup window. Differnce between trackinguitility and this is this class uses paintgl to render the tracked points to the UI whereas tracking utility just pushes the data into processing utility. More...

#include <BallTrackingVisualization.h>

Collaboration diagram for BallTrackingVisualization:

List of all members.

Public Slots
void	setBinaryLowerThresh (double _lowerThresh)
	All the following functions are setter functions determining the imapge processing efficiency and accuracy.
void	setStructuringElementSize (int _size)
void	setBlurSize (int _size)
void	setCannyLowerThresh (double _lowerThresh)
void	setCannyHigherThresh (double _higherThresh)
void	setNumerOfDialtions (int _numberOfDialtions)
void	setContourAreaThresh (double _contourAreaThresh)
void	setDifferenceInAreaThresh (double _diffAreaThresh)
void	setMasterRigCheck (int _masterRigCheck)
Public Member Functions
	BallTrackingVisualization (QWidget *_parent)
	Constructor for BallTrackingVisualization.
	~BallTrackingVisualization ()
	Dtor for BallTrackingVisualization.
Protected Member Functions
void	initializeGL ()
	The following methods must be implimented in the sub class this is called when the window is created.
void	resizeGL (const int _w, const int _h)
	this is called whenever the window is re-sized
void	paintGL ()
	this is the main gl drawing routine which is called whenever the window needs to
Private Member Functions
void	mouseMoveEvent (QMouseEvent *_event)
	this method is called every time a mouse is moved
void	mousePressEvent (QMouseEvent *_event)
	this method is called everytime the mouse button is pressed inherited from QObject and overridden here.
void	mouseReleaseEvent (QMouseEvent *_event)
	this method is called everytime the mouse button is released inherited from QObject and overridden here.
void	timerEvent (QTimerEvent *_event)
	called when the timer is triggered
Private Attributes
cv::Mat	m_rgb
	rgb and gray buffers filled in from kinect
cv::Mat	m_gray
cv::Mat	m_nextBuffer
	individual buffers used to store temporary results while image processing
cv::Mat	m_prevBuffer
cv::Mat	m_diffBuffer
cv::Mat	m_bufferDepthRaw
cv::Mat	m_bufferDepthRaw1
float	m_depthinm [640 *480]
	depth in meters
ScreenQuad *	m_screen
	for quad pasted rendering we have an instance
std::vector< cv::Moments >	m_momentList
	same as in balltrackingutility
cv::Mat	canny_output
	same as in balltrackingutility
std::vector< std::vector < cv::Point > >	contours
	same as in balltrackingutility
cv::Mat	hierarchy
	same as in balltrackingutility
double	area
	same as in balltrackingutility
double	rectArea
std::vector< cv::Moments >	mu
	same as in balltrackingutility
std::vector< cv::Point2f >	mc
	same as in balltrackingutility
double	m_binaryLowerThresh
	same as in balltrackingutility
double	m_binaryMaxVal
double	m_cannyLowerThresh
double	m_cannyHigherThresh
double	m_contourAreaThresh
	same as in balltrackingutility
double	m_diffAreaThresh
int	m_ellipsesize
	same as in balltrackingutility
int	m_blurSize
int	m_noOfTimesToDialate
int	m_masterRigCheck

Detailed Description

Ball track visualization as green circles just for visualization of real time tracking This is enabled when the user is in setup window. Differnce between trackinguitility and this is this class uses paintgl to render the tracked points to the UI whereas tracking utility just pushes the data into processing utility.

Definition at line 34 of file BallTrackingVisualization.h.

Constructor & Destructor Documentation

BallTrackingVisualization::BallTrackingVisualization ( QWidget * _parent )

Constructor for BallTrackingVisualization.

Parameters:

[in] _parent the parent window to create the GL context in

Definition at line 19 of file BallTrackingVisualization.cpp.

References area, m_binaryLowerThresh, m_binaryMaxVal, m_blurSize, m_bufferDepthRaw, m_bufferDepthRaw1, m_cannyHigherThresh, m_cannyLowerThresh, m_contourAreaThresh, m_depthinm, m_diffAreaThresh, m_diffBuffer, m_ellipsesize, m_gray, m_masterRigCheck, m_momentList, m_nextBuffer, m_noOfTimesToDialate, m_prevBuffer, m_rgb, and rectArea.

                                                                     : QGLWidget( new CreateCoreGLContext(QGLFormat::defaultFormat()), _parent )

{

  // re-size the widget to that of the parent (in this case the GLFrame passed in on construction)
  this->resize(_parent->size());
  startTimer(20);
  m_rgb = cvCreateMat(480,640,CV_8UC3);

  m_gray = cvCreateMat(480,640,CV_8UC1);

  m_nextBuffer = cvCreateMat(480,640,CV_8UC1);
  m_prevBuffer = cvCreateMat(480,640,CV_8UC1);
  m_diffBuffer = cvCreateMat(480,640,CV_8UC1);

  m_bufferDepthRaw=cvCreateMat(480,640,CV_8UC1);
  m_bufferDepthRaw1=cvCreateMat(480,640,CV_8UC1);

  m_binaryLowerThresh = 100.0;
  m_binaryMaxVal = 255.0;
  m_cannyLowerThresh = 100.0;
  m_cannyHigherThresh = 200.0;
  m_contourAreaThresh = 100.0;
  m_diffAreaThresh = 10.0;
  m_ellipsesize = 2;
  m_blurSize = 5;
  m_noOfTimesToDialate = 2;

  m_masterRigCheck = 0;

  for( unsigned int i = 0 ; i < 640*480 ; ++i)
  {
      m_depthinm[i] = 0;
  }


  m_momentList.clear();

  area = rectArea = 0.0;
}

BallTrackingVisualization::~BallTrackingVisualization ( )

Dtor for BallTrackingVisualization.

Definition at line 272 of file BallTrackingVisualization.cpp.

References m_screen.

{
    if(m_screen)
    {
        delete m_screen;
    }
}

Member Function Documentation

void BallTrackingVisualization::initializeGL ( ) [protected]

The following methods must be implimented in the sub class this is called when the window is created.

Definition at line 65 of file BallTrackingVisualization.cpp.

References m_screen.

{

  glEnable(GL_DEPTH_TEST);


    //
  ngl::NGLInit *Init = ngl::NGLInit::instance();
  Init->initGlew();

  // now we have valid GL can create shaders etc
  ngl::ShaderLib *shader = ngl::ShaderLib::instance();
  shader->createShaderProgram("TextureTrack");

  shader->attachShader("TextureVertex",ngl::VERTEX);
  shader->attachShader("TextureFragment",ngl::FRAGMENT);
  shader->loadShaderSource("TextureVertex","shaders/TextureVert.glsl");
  shader->loadShaderSource("TextureFragment","shaders/TextureFrag.glsl");

  shader->compileShader("TextureVertex");
  shader->compileShader("TextureFragment");
  shader->attachShaderToProgram("TextureTrack","TextureVertex");
  shader->attachShaderToProgram("TextureTrack","TextureFragment");

  shader->linkProgramObject("TextureTrack");
  (*shader)["TextureTrack"]->use();
  shader->setShaderParam1i("tex",0);

  m_screen = new ScreenQuad(640,480,"TextureTrack");


}

void BallTrackingVisualization::mouseMoveEvent ( QMouseEvent * _event ) [private]

this method is called every time a mouse is moved

Parameters:

_event the Qt Event structure

Definition at line 241 of file BallTrackingVisualization.cpp.

{
    Q_UNUSED(_event);
}

void BallTrackingVisualization::mousePressEvent ( QMouseEvent * _event ) [private]

this method is called everytime the mouse button is pressed inherited from QObject and overridden here.

Parameters:

_event the Qt Event structure

Definition at line 248 of file BallTrackingVisualization.cpp.

{
    Q_UNUSED(_event);

}

void BallTrackingVisualization::mouseReleaseEvent ( QMouseEvent * _event ) [private]

this method is called everytime the mouse button is released inherited from QObject and overridden here.

Parameters:

_event the Qt Event structure

Definition at line 255 of file BallTrackingVisualization.cpp.

{
  // this event is called when the mouse button is released
  // we then set Rotate to false
  Q_UNUSED(_event);

}

void BallTrackingVisualization::paintGL ( ) [protected]

this is the main gl drawing routine which is called whenever the window needs to

Definition at line 113 of file BallTrackingVisualization.cpp.

References area, canny_output, contours, ScreenQuad::draw(), KinectInterface::getBoxObject(), KinectInterface::getDepthRaw(), KinectInterface::getTrackingFlag(), hierarchy, KinectInterface::instance(), m_binaryLowerThresh, m_binaryMaxVal, m_blurSize, m_bufferDepthRaw, m_bufferDepthRaw1, m_cannyHigherThresh, m_cannyLowerThresh, m_contourAreaThresh, m_depthinm, m_diffAreaThresh, m_diffBuffer, m_ellipsesize, m_gray, m_momentList, m_nextBuffer, m_noOfTimesToDialate, m_prevBuffer, and m_screen.

{
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);


    KinectInterface *kinect=KinectInterface::instance();

    cv::Rect _boundsCheck = kinect->getBoxObject();

    if(kinect->getTrackingFlag())
    {


        static int _frameCount = 1;
       float depth = 0.0;

        kinect->getDepthRaw(m_bufferDepthRaw,m_depthinm);

        // commented for gpu trial

        cv::blur(m_bufferDepthRaw,m_bufferDepthRaw1,cv::Size(m_blurSize,m_blurSize)); // to reduce noise, smotthing has been done
        /*  cv::gpu::blur(m_bufferDepthRawGpu,m_bufferDepthRawGpu1,cv::Size(m_blurSize,m_blurSize));*/

        if(_frameCount == 1)
        {
            m_bufferDepthRaw1.copyTo(m_prevBuffer);
        }

        m_bufferDepthRaw1.copyTo(m_nextBuffer);

        // commented for gpu trial

        cv::absdiff(m_nextBuffer,m_prevBuffer,m_diffBuffer); // has gpu variant // bg subtraction by frame differencing ...GMM method seems good through forums if at all we need
        /*cv::gpu::absdiff(m_nextBufferGpu,m_prevBufferGpu,m_diffBufferGpu);*/

        m_nextBuffer.copyTo(m_prevBuffer);

        // commented for gpu trial

        m_diffBuffer.copyTo(m_gray);
        /*m_diffBuffer.copyTo(m_grayGpu);*/

        // commented for gpu tiral
        cv::threshold(m_gray,m_gray,m_binaryLowerThresh,m_binaryMaxVal,CV_THRESH_BINARY_INV); // nned to decide the thresh factors and the thresh method
        cv::Mat element = cv::getStructuringElement(cv::MORPH_ELLIPSE,cv::Size(m_ellipsesize,m_ellipsesize)); // need to decide this size
        cv::dilate(m_gray,m_gray,element,cv::Point(-1,-1),m_noOfTimesToDialate); // try swapping this with below and try increasing iterations
        cv::erode(m_gray,m_gray,element,cv::Point(-1,-1),m_noOfTimesToDialate);

        /*cv::gpu::threshold(m_grayGpu,m_grayGpu,m_binaryLowerThresh,m_binaryMaxVal,CV_THRESH_BINARY_INV);
        cv::Mat element = cv::getStructuringElement(cv::MORPH_ELLIPSE,cv::Size(m_ellipsesize,m_ellipsesize)); // need to decide this size
        cv::gpu::dilate(m_grayGpu,m_grayGpu,element,cv::Point(-1,-1),m_noOfTimesToDialate); // try swapping this with below and try increasing iterations
        cv::gpu::erode(m_grayGpu,m_grayGpu,element,cv::Point(-1,-1),m_noOfTimesToDialate);*/

        // commented for gpu trial

        // Detect edges using canny
        cv::Canny( m_gray, canny_output, m_cannyLowerThresh, m_cannyHigherThresh, 3 ); // maintiain upper::lower ratio as 2:1 or 3:1 for better results
                                                                                     // as suggested by CANNY
        /*  cv::gpu::Canny(m_grayGpu, canny_outputGpu, m_cannyLowerThresh, m_cannyHigherThresh, 3);*/
        // commented for gpu trial

        // Find contours
        cv::findContours( canny_output, contours, hierarchy, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0) ); // methods to be decided
        /*cv::findContours( canny_outputGpu, contours, hierarchy, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0) ); // methods to be decided*/


        cv::Mat drawing = cv::Mat::zeros( canny_output.size(), CV_8UC3 );


        for( int i = 0; i < contours.size(); i++ )
        {

            /* cv::drawContours( drawing, contours, i, cv::Scalar(255,0,0),-1, 8, hierarchy, 0, cv::Point() );*/ // -- DEV PURPOSE

            area = cv::contourArea(contours[i], false);

            cv::RotatedRect r = cv::minAreaRect(contours[i]);



            if (fabs(area) < m_contourAreaThresh) //my notes: this value is more important in reducing the false positives and filters out those false positives by preventing them
                                    // from getting processed further..provide in UI
            {
                continue;
            }
            else
            {
                // do nothing
            }

            if (fabs(fabs(area) - (r.size.width * r.size.height * 3.14 / 4)) < m_diffAreaThresh)
            {

                m_momentList.push_back(cv::moments( contours[i], false ));

                float _xMomentsTemp = static_cast<float>(cv::moments( contours[i], false ).m10/cv::moments( contours[i], false ).m00);
                float _yMomentsTemp = static_cast<float>(cv::moments( contours[i], false ).m01/cv::moments( contours[i], false ).m00);

                int depthIndex = (int)((640 - _xMomentsTemp) + (640 * _yMomentsTemp));
                depth = m_depthinm[depthIndex] * 0.001;

                if(depth != 0)
                {
                   if((_xMomentsTemp >= _boundsCheck.x) && (_xMomentsTemp <= _boundsCheck.x + _boundsCheck.width)
                           && (_yMomentsTemp >= _boundsCheck.y) && (_yMomentsTemp <= _boundsCheck.y + _boundsCheck.height))
                   {
                       cv::ellipse(drawing,r,cv::Scalar(0,255,0),-1,8);
                       std::cout<<"Rect center x:"<<r.center.x<<"\n";
                   }
                 }

             }

          }

       _frameCount++;

       m_screen->draw(&drawing);

     }
     else
     {
        // tracking not yet started...do nothing
     }

}

Here is the call graph for this function:

void BallTrackingVisualization::resizeGL	(	const int	_w,
		const int	_h
	)		`[protected]`

this is called whenever the window is re-sized

Parameters:

[in]	_w	the width of the resized window
[in]	_h	the height of the resized window

Definition at line 101 of file BallTrackingVisualization.cpp.

{
  glViewport(0,0,_w,_h);

}

void BallTrackingVisualization::setBinaryLowerThresh ( double _lowerThresh ) [slot]

All the following functions are setter functions determining the imapge processing efficiency and accuracy.

Definition at line 282 of file BallTrackingVisualization.cpp.

References m_binaryLowerThresh.

{
    m_binaryLowerThresh = _lowerThresh;
    updateGL();
}

void BallTrackingVisualization::setBlurSize ( int _size ) [slot]

Definition at line 294 of file BallTrackingVisualization.cpp.

References m_blurSize.

{
    m_blurSize = _size;
    updateGL();
}

void BallTrackingVisualization::setCannyHigherThresh ( double _higherThresh ) [slot]

Definition at line 307 of file BallTrackingVisualization.cpp.

References m_cannyHigherThresh.

{
    m_cannyHigherThresh = _higherThresh;
    updateGL();
}

void BallTrackingVisualization::setCannyLowerThresh ( double _lowerThresh ) [slot]

Definition at line 300 of file BallTrackingVisualization.cpp.

References m_cannyLowerThresh.

{
    m_cannyLowerThresh = _lowerThresh;
    updateGL();
}

void BallTrackingVisualization::setContourAreaThresh ( double _contourAreaThresh ) [slot]

Definition at line 320 of file BallTrackingVisualization.cpp.

References m_contourAreaThresh.

{
    m_contourAreaThresh = _contourAreaThresh;
    updateGL();
}

void BallTrackingVisualization::setDifferenceInAreaThresh ( double _diffAreaThresh ) [slot]

Definition at line 326 of file BallTrackingVisualization.cpp.

References m_diffAreaThresh.

{
    m_diffAreaThresh = _diffAreaThresh;
    updateGL();
}

void BallTrackingVisualization::setMasterRigCheck ( int _masterRigCheck ) [slot]

Definition at line 332 of file BallTrackingVisualization.cpp.

References m_masterRigCheck.

{
    m_masterRigCheck = _masterRigCheck;
    updateGL();
}

void BallTrackingVisualization::setNumerOfDialtions ( int _numberOfDialtions ) [slot]

Definition at line 314 of file BallTrackingVisualization.cpp.

References m_noOfTimesToDialate.

{
    m_noOfTimesToDialate = _numberOfDialtions;
    updateGL();
}

void BallTrackingVisualization::setStructuringElementSize ( int _size ) [slot]

Definition at line 288 of file BallTrackingVisualization.cpp.

References m_ellipsesize.

{
    m_ellipsesize = _size;
    updateGL();
}

void BallTrackingVisualization::timerEvent ( QTimerEvent * _event ) [private]

called when the timer is triggered

Definition at line 263 of file BallTrackingVisualization.cpp.

{
    Q_UNUSED(_event);
    // re-draw GL
    updateGL();

}

Member Data Documentation

double BallTrackingVisualization::area [private]

same as in balltrackingutility

Definition at line 93 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::canny_output [private]

same as in balltrackingutility

Definition at line 78 of file BallTrackingVisualization.h.

std::vector<std::vector<cv::Point> > BallTrackingVisualization::contours [private]

same as in balltrackingutility

Definition at line 83 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::hierarchy [private]

same as in balltrackingutility

Definition at line 88 of file BallTrackingVisualization.h.

double BallTrackingVisualization::m_binaryLowerThresh [private]

same as in balltrackingutility

Definition at line 108 of file BallTrackingVisualization.h.

double BallTrackingVisualization::m_binaryMaxVal [private]

Definition at line 108 of file BallTrackingVisualization.h.

int BallTrackingVisualization::m_blurSize [private]

Definition at line 118 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::m_bufferDepthRaw [private]

Definition at line 57 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::m_bufferDepthRaw1 [private]

Definition at line 57 of file BallTrackingVisualization.h.

double BallTrackingVisualization::m_cannyHigherThresh [private]

Definition at line 108 of file BallTrackingVisualization.h.

double BallTrackingVisualization::m_cannyLowerThresh [private]

Definition at line 108 of file BallTrackingVisualization.h.

double BallTrackingVisualization::m_contourAreaThresh [private]

same as in balltrackingutility

Definition at line 113 of file BallTrackingVisualization.h.

float BallTrackingVisualization::m_depthinm[640 *480] [private]

depth in meters

Definition at line 63 of file BallTrackingVisualization.h.

double BallTrackingVisualization::m_diffAreaThresh [private]

Definition at line 113 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::m_diffBuffer [private]

Definition at line 57 of file BallTrackingVisualization.h.

int BallTrackingVisualization::m_ellipsesize [private]

same as in balltrackingutility

Definition at line 118 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::m_gray [private]

Definition at line 52 of file BallTrackingVisualization.h.

int BallTrackingVisualization::m_masterRigCheck [private]

Definition at line 118 of file BallTrackingVisualization.h.

std::vector<cv::Moments> BallTrackingVisualization::m_momentList [private]

same as in balltrackingutility

Definition at line 73 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::m_nextBuffer [private]

individual buffers used to store temporary results while image processing

Definition at line 57 of file BallTrackingVisualization.h.

int BallTrackingVisualization::m_noOfTimesToDialate [private]

Definition at line 118 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::m_prevBuffer [private]

Definition at line 57 of file BallTrackingVisualization.h.

cv::Mat BallTrackingVisualization::m_rgb [private]

rgb and gray buffers filled in from kinect

Definition at line 52 of file BallTrackingVisualization.h.

ScreenQuad* BallTrackingVisualization::m_screen [private]

for quad pasted rendering we have an instance

Definition at line 68 of file BallTrackingVisualization.h.

std::vector<cv::Point2f> BallTrackingVisualization::mc [private]

same as in balltrackingutility

Definition at line 103 of file BallTrackingVisualization.h.

std::vector<cv::Moments> BallTrackingVisualization::mu [private]

same as in balltrackingutility

Definition at line 98 of file BallTrackingVisualization.h.

double BallTrackingVisualization::rectArea [private]

Definition at line 93 of file BallTrackingVisualization.h.

The documentation for this class was generated from the following files:

Public Slots

Public Member Functions

Protected Member Functions

Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation