# EE106A Lab 6.pdf - EE106A: Lab 6 - Computer Vision ∗ Fall...

• 12

Course Hero uses AI to attempt to automatically extract content from documents to surface to you and others so you can study better, e.g., in search results, to enrich docs, and more. This preview shows page 1 - 3 out of 12 pages.

EE106A: Lab 6 - Computer Vision*Fall 2019GoalsBy the end of this lab you should be able to:Explain the concept behind pointclouds and what they representImplement some basic techniques in image segmentation and explain your resultsExplain the concept of pinhole cameras and describe how they workCombine information from different sensors to isolate an object of interest. (Build a point cloud segmentation pipeline)Relevant Tutorials and Documentation:OpenCV Python TutorialContents1Introduction22Starter Code23Intel RealSense24Point-Clouds24.1An approach to point-cloud segmentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35Image Segmentation35.1Color and Grayscale Images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45.2Thresholding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45.3Edge Detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55.4Clustering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66Projective Geometry and The Camera Matrix76.1Homogenous Co-ordinates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76.2The Pinhole Camera. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86.3The Intrinsic Camera Matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86.4Projecting the Point-cloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97Putting it all together107.1Time Synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .107.2Queue Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10*Developed by Amay Saxena and Grant Wang, Fall 2019.1
1IntroductionThis lab will introduce you to various techniques used in processing data from advanced sensors like the Intel RealSense. Inparticular, you will learn about image segmentation, point cloud processing, and how we can combine various data modalitiesto leverage the strengths and weaknesses of each data form.You will write your ownpoint-cloud segmentation pipeline. This means that by the end of this lab, you will be able toisolate an object of interest in a point-cloud, by combining information you get from RGB image sensors.2Starter CodeCreate a new workspace calledlab6. Then, unzip the contents of thelab6_resourcesfolder into thesrcdirectory ofthis workspace.The starter code includes the following packages:1.realsense-ros: This is a directory that contains several ROS packages to work with the Intel Realsense sensor,which we will be using for this lab.2.ros_numpy: A very useful ROS package that can be used to convert between ROS message datatypes and numpydatatypes. Check out the documentation at.3.segmentation: This is the package that contains all the skeleton code. All files that you will need to edit are in thispackage.After creating your workspace and copying over the contents oflab6_resourcesintolab6/src, usecatkin_maketo build your workspace. Then, go intosegmentation/srcand make all python files executable usingchmod +x*.py3Intel RealSenseIn this lab, you will be using an Intel RealSense camera, which is a powerful sensor equipped with an RGB camera, a depthcamera, an Inertial Measurement Unit, and an IR sensor which is capable of producing a rich colored point-cloud. We willconcern ourselves with the RGB camera and the point-cloud.The drivers for the RealSense have already been installed for you. The starter code includes a packagerealsense-rosthat will allow you to begin publishing information from the RealSense camera onto various topics. Ensure that you have the

Upload your study docs or become a

Course Hero member to access this document

Upload your study docs or become a

Course Hero member to access this document

End of preview. Want to read all 12 pages?

Upload your study docs or become a

Course Hero member to access this document

Term
Spring
Professor
MingWu