Vision System for a Robotic Gantry
In this video I briefly describe how we developed vision recognition routines for the robotic gantry. Sorry if I am talking fast, it was hard to condense my research into 8 minutes or less.
Toggle between list and paragraph view.
- [00:00:01.320]My presentation will be about a gantry
vision system for building detectors for
- [00:00:05.460]the Large Hadron Collider in Geneva, Switzerland.
- [00:00:08.100]My instructor is Frank golf
and my name is Hayden Swanson.
- [00:00:12.900]So first things first, uh,
- [00:00:15.250]we got to detect high energy particles
and the LHC actually collides billions
- [00:00:19.410]particles every second.
- [00:00:20.730]This is to look for new particles by studying
the debris that come up from these particles
- [00:00:25.620]and we're building new detectors
for the CMS experiment at the LHC.
- [00:00:30.120]And we needed to build thousands of
detector modules with high precision and
- [00:00:34.680]accuracy. And here's a picture
of a, one of these new,
- [00:00:39.210]um, detector modules that are being made,
- [00:00:43.740]which having higher energy and more
sensitive detectors allow for deeper
- [00:00:48.270]exploration of the universe.
- [00:00:51.240]So here's me using the gantry
and this slide is about
- [00:00:56.100]automating detector construction using
a robotic gantry with the vision system.
- [00:01:01.050]And the right picture is the robotic
gantry. As you can see the arrow points to,
- [00:01:05.970]and the gantry camera. It uses fiducials
that it gets from images,
- [00:01:12.060]using its camera and they
are used for as reference
- [00:01:17.550]which allow the gantry to have accurate
spatial awareness when assembling the
- [00:01:22.350]So now the question is how do
we use this camera to extract
- [00:01:27.090]information from images and relay
that info meaningfully to the gantry?
- [00:01:31.740]So it can build this spatial
awareness to build these modules
- [00:01:36.600]with high accuracy and
precision. So goal is,
- [00:01:41.220]uh, to locate fiducial markers
using the gantry vision hardware.
- [00:01:45.690]And how does it do this? Um,
- [00:01:49.020]with vision recognition routines to identify regions
of interest in images from the
- [00:01:53.250]camera it's written in C++ using OpenCV library,
- [00:01:56.700]which is an image processing
library. And it's wrapped in LabVIEW.
- [00:01:59.730]LabVIEW tells the gantry what to do so,
- [00:02:02.850]It speaks with it directly.
then its read to C++,
- [00:02:05.880]and the call library
function node on the right,
- [00:02:08.430]it just takes the DLL
from the C++.
- [00:02:12.480]So here's a picture of a wire bond pad,
- [00:02:14.190]which could be used as a fiducial marker.
- [00:02:16.920]And it's those xFOVs and yFOVS and how
- [00:02:21.750]big it is.
- [00:02:24.300]So what's the general
procedure for detecting, uh,
- [00:02:29.010]getting information out
of a picture. First,
- [00:02:31.980]you want to prep the image
for a contour algorithm.
- [00:02:35.730]So for wire bond pad, we
use a thresholding then a
Gaussian blur and then erode,
- [00:02:40.590]erode, and dilate,
- [00:02:43.080]then reapply a Findcontours
algorithm after it prepped it.
- [00:02:48.570]And then we simplify it using
approxPolyDP and then we uh,
- [00:02:53.580]put a box around it,
- [00:02:54.720]using the bounding rectangle from OpenCV,
- [00:02:58.800]and we have everything we need. So
we just filter out unwanted contours,
- [00:03:04.900]step one, you apply your threshold.
- [00:03:07.210]And all this does is says any pixel less
than one 125 intensity make it zero.
- [00:03:11.920]Otherwise make it white to
help separate foregrounds,
- [00:03:14.110]but it creates a lot of jagged
edges. So then we apply a blur,
- [00:03:18.460]a blur. This is what the blur looks like,
- [00:03:21.490]and this is what it looks like if you
do it before the jagged edges, you know,
- [00:03:26.200]and then it tries to put a contour
and just doesn't work as well.
- [00:03:29.110]So we use the order on
the right then we apply,
- [00:03:32.350]our erode and dilates,
- [00:03:35.230]and it puts the bounding box
around it. Its rotated with two erodes,
- [00:03:40.000]one dilate the erode and dilates
get rid of the noise in the back.
- [00:03:44.650]So you can, which helps
the bounding rectangle.
- [00:03:48.670]And it makes for better images, um,
- [00:03:55.480]that worked out well. Oh,
- [00:04:00.040]I was just, sorry. Um,
- [00:04:03.730]you might think the erode and dilates hurt
the images, as you can see from before,
- [00:04:06.970]but you can do five of them. It's fine.
- [00:04:09.490]Here's the LabVIEW interface for
it. It takes a file path.
- [00:04:14.290]The percent size tells you how big you
want your picture on the computer screen.
- [00:04:18.100]And then you plug in what your, the
width and height think it's going to be.
- [00:04:22.750]And then the tolerance around
that height that you need.
- [00:04:25.510]And then it uses the field
of view from the camera,
- [00:04:28.960]and then it gives you the
height and width and, uh,
- [00:04:34.840]and real units, millimeters,
and then the center X and Y,
- [00:04:40.810]and the center X and Y in millimeters.
- [00:04:43.930]And then it gives you the angle it's
been rotated at. Here are the results:
- [00:04:48.490]manufacturers say the wire bond pads
have a width of 0.47 millimeters,
- [00:04:51.490]and a height of 0.3 millimeters.
program from the width to be 0.45
- [00:04:55.840]millimeters and height
of 0.27 millimeters.
- [00:04:58.570]And those are the percent errors,
- [00:05:00.130]but we gotta remember that
the manufacturer has, um,
- [00:05:05.980]like its own error and manufacturing
and the error from this,
- [00:05:10.390]the measuring the field of
view. So I went to MS Paint,
- [00:05:14.020]found it to be point to .27 and .45 millimeters.
- [00:05:19.120]Um, those are the averages at the bottom.
- [00:05:22.180]So now here's another fiducial that
is more of this crosshair shape that
- [00:05:27.370]we're gonna probably be using mostly
instead of the wire bond pads.
- [00:05:31.600]But for this it is pretty simple, um,
- [00:05:35.860]more simple than the wire bond pads.
- [00:05:37.570]It just applies a blur only because
hough circles not very robust to noise,
- [00:05:42.250]and actually using any other, um,
- [00:05:47.470]any other image processing
made it worse. I found.
- [00:05:50.800]So I just used the Gaussian
blur, and then the Hough circles,
- [00:05:53.980]which is a really good algorithm, um,
returns a vector containing the center, and X Y,
- [00:05:58.910]which is all you
need to define a circle.
- [00:06:02.210]And then I drew an error
circle from the center,
- [00:06:04.280]which tells you how much you need to be
within. So you can see on the screen,
- [00:06:09.440]uh, we need to be within there.
- [00:06:11.750]Here's the interface has min
and max radius to tell you,
- [00:06:16.520]like, if you want your circles
to be within this amount,
- [00:06:19.220]and then it has the circle
- [00:06:21.680]one tells you how much of a circle is
it actually a circle and other one is to
- [00:06:26.180]deal with edge detection. So it's
like how much of an edge is an edge,
- [00:06:30.950]kind of, if you want a rough idea,
here's it in use the Param_1,
- [00:06:34.940]which is the edge detection,
one, the original,
- [00:06:37.430]which is just using the basic
values. It's off a little bit,
- [00:06:40.850]but if you just change
Param_1 it's spot on
- [00:06:46.160]you need to prep the image using
techniques like thresholding blur erode
- [00:06:50.690]dilate, and then you've got
to apply your algorithm.
- [00:06:53.690]You really only want to apply these
image processing techniques as needed.
- [00:06:57.800]You know, there's no correct
formula for doing it.
- [00:07:00.800]It's whatever you need to get the
information from your pictures,
- [00:07:04.520]and that works best with
your contour algorithm.
- [00:07:07.610]And then you can use the bounding
rectangle hough circles or whatever,
- [00:07:11.090]or you can use the approxPolyDP
if you have the right Epsilon value,
- [00:07:13.970]but I didn't really get into that because
we only detected circles and rectangles.
- [00:07:16.820]And then you just filter the objects,
which we only use, like length,
- [00:07:20.420]pretty much. Improvements are to add more
parameters, and to get a better algorithm
- [00:07:25.340]to handle less crisp pictures,
- [00:07:28.850]and then looking forwards to fix
bugs and adding features as needed.
- [00:07:33.470]And then it can also detect corners.
Log in to post comments