Vision System for a Robotic Gantry

Hayden Swanson Author

03/29/2021 Added

158 Plays

Description

In this video I briefly describe how we developed vision recognition routines for the robotic gantry. Sorry if I am talking fast, it was hard to condense my research into 8 minutes or less.

Searchable Transcript

Search:

[00:00:00.840]Hello.
[00:00:01.320]My presentation will be about a gantry vision system for building detectors for
[00:00:05.460]the Large Hadron Collider in Geneva, Switzerland.
[00:00:08.100]My instructor is Frank golf and my name is Hayden Swanson.
[00:00:12.900]So first things first, uh,
[00:00:15.250]we got to detect high energy particles and the LHC actually collides billions
[00:00:19.410]particles every second.
[00:00:20.730]This is to look for new particles by studying the debris that come up from these particles
[00:00:25.620]and we're building new detectors for the CMS experiment at the LHC.
[00:00:30.120]And we needed to build thousands of detector modules with high precision and
[00:00:34.680]accuracy. And here's a picture of a, one of these new,
[00:00:39.210]um, detector modules that are being made,
[00:00:43.740]which having higher energy and more sensitive detectors allow for deeper
[00:00:48.270]exploration of the universe.
[00:00:51.240]So here's me using the gantry and this slide is about
[00:00:56.100]automating detector construction using a robotic gantry with the vision system.
[00:01:01.050]And the right picture is the robotic gantry. As you can see the arrow points to,
[00:01:05.970]and the gantry camera. It uses fiducials that it gets from images,
[00:01:10.830]um,
[00:01:12.060]using its camera and they are used for as reference
[00:01:16.860]points,
[00:01:17.550]which allow the gantry to have accurate spatial awareness when assembling the
[00:01:21.930]modules.
[00:01:22.350]So now the question is how do we use this camera to extract
[00:01:27.090]information from images and relay that info meaningfully to the gantry?
[00:01:31.740]So it can build this spatial awareness to build these modules
[00:01:36.600]with high accuracy and precision. So goal is,
[00:01:41.220]uh, to locate fiducial markers using the gantry vision hardware.
[00:01:45.690]And how does it do this? Um,
[00:01:49.020]with vision recognition routines to identify regions of interest in images from the
[00:01:53.250]camera it's written in C++ using OpenCV library,
[00:01:56.700]which is an image processing library. And it's wrapped in LabVIEW.
[00:01:59.730]LabVIEW tells the gantry what to do so,
[00:02:02.850]It speaks with it directly. then its read to C++,
[00:02:05.880]and the call library function node on the right,
[00:02:08.430]it just takes the DLL from the C++.
[00:02:12.480]So here's a picture of a wire bond pad,
[00:02:14.190]which could be used as a fiducial marker.
[00:02:16.920]And it's those xFOVs and yFOVS and how
[00:02:21.750]big it is.
[00:02:24.300]So what's the general procedure for detecting, uh,
[00:02:29.010]getting information out of a picture. First,
[00:02:31.980]you want to prep the image for a contour algorithm.
[00:02:35.730]So for wire bond pad, we use a thresholding then a Gaussian blur and then erode,
[00:02:40.590]erode, and dilate,
[00:02:43.080]then reapply a Findcontours algorithm after it prepped it.
[00:02:48.570]And then we simplify it using approxPolyDP and then we uh,
[00:02:53.580]put a box around it,
[00:02:54.720]using the bounding rectangle from OpenCV,
[00:02:58.800]and we have everything we need. So we just filter out unwanted contours,
[00:03:04.900]step one, you apply your threshold.
[00:03:07.210]And all this does is says any pixel less than one 125 intensity make it zero.
[00:03:11.920]Otherwise make it white to help separate foregrounds,
[00:03:14.110]but it creates a lot of jagged edges. So then we apply a blur,
[00:03:18.460]a blur. This is what the blur looks like,
[00:03:21.490]and this is what it looks like if you do it before the jagged edges, you know,
[00:03:26.200]and then it tries to put a contour and just doesn't work as well.
[00:03:29.110]So we use the order on the right then we apply,
[00:03:32.350]our erode and dilates,
[00:03:35.230]and it puts the bounding box around it. Its rotated with two erodes,
[00:03:40.000]one dilate the erode and dilates get rid of the noise in the back.
[00:03:44.650]So you can, which helps the bounding rectangle.
[00:03:48.670]And it makes for better images, um,
[00:03:55.480]that worked out well. Oh,
[00:04:00.040]I was just, sorry. Um,
[00:04:03.730]you might think the erode and dilates hurt the images, as you can see from before,
[00:04:06.970]but you can do five of them. It's fine.
[00:04:09.490]Here's the LabVIEW interface for it. It takes a file path.
[00:04:14.290]The percent size tells you how big you want your picture on the computer screen.
[00:04:18.100]And then you plug in what your, the width and height think it's going to be.
[00:04:22.750]And then the tolerance around that height that you need.
[00:04:25.510]And then it uses the field of view from the camera,
[00:04:28.960]and then it gives you the height and width and, uh,
[00:04:34.840]and real units, millimeters, and then the center X and Y,
[00:04:40.810]and the center X and Y in millimeters.
[00:04:43.930]And then it gives you the angle it's been rotated at. Here are the results:
[00:04:48.490]manufacturers say the wire bond pads have a width of 0.47 millimeters,
[00:04:51.490]and a height of 0.3 millimeters. program from the width to be 0.45
[00:04:55.840]millimeters and height of 0.27 millimeters.
[00:04:58.570]And those are the percent errors,
[00:05:00.130]but we gotta remember that the manufacturer has, um,
[00:05:05.980]like its own error and manufacturing and the error from this,
[00:05:10.390]the measuring the field of view. So I went to MS Paint,
[00:05:14.020]found it to be point to .27 and .45 millimeters.
[00:05:19.120]Um, those are the averages at the bottom.
[00:05:22.180]So now here's another fiducial that is more of this crosshair shape that
[00:05:27.370]we're gonna probably be using mostly instead of the wire bond pads.
[00:05:31.600]But for this it is pretty simple, um,
[00:05:35.860]more simple than the wire bond pads.
[00:05:37.570]It just applies a blur only because hough circles not very robust to noise,
[00:05:42.250]and actually using any other, um,
[00:05:47.470]any other image processing made it worse. I found.
[00:05:50.800]So I just used the Gaussian blur, and then the Hough circles,
[00:05:53.980]which is a really good algorithm, um, returns a vector containing the center, and X Y,
[00:05:58.910]which is all you need to define a circle.
[00:06:02.210]And then I drew an error circle from the center,
[00:06:04.280]which tells you how much you need to be within. So you can see on the screen,
[00:06:09.440]uh, we need to be within there.
[00:06:11.750]Here's the interface has min and max radius to tell you,
[00:06:16.520]like, if you want your circles to be within this amount,
[00:06:19.220]and then it has the circle detection sensitivities,
[00:06:21.680]one tells you how much of a circle is it actually a circle and other one is to
[00:06:26.180]deal with edge detection. So it's like how much of an edge is an edge,
[00:06:30.950]kind of, if you want a rough idea, here's it in use the Param_1,
[00:06:34.940]which is the edge detection, one, the original,
[00:06:37.430]which is just using the basic values. It's off a little bit,
[00:06:40.850]but if you just change Param_1 it's spot on
[00:06:45.170]conclusion,
[00:06:46.160]you need to prep the image using techniques like thresholding blur erode
[00:06:50.690]dilate, and then you've got to apply your algorithm.
[00:06:53.690]You really only want to apply these image processing techniques as needed.
[00:06:57.800]You know, there's no correct formula for doing it.
[00:07:00.800]It's whatever you need to get the information from your pictures,
[00:07:04.520]and that works best with your contour algorithm.
[00:07:07.610]And then you can use the bounding rectangle hough circles or whatever,
[00:07:11.090]or you can use the approxPolyDP if you have the right Epsilon value,
[00:07:13.970]but I didn't really get into that because we only detected circles and rectangles.
[00:07:16.820]And then you just filter the objects, which we only use, like length,
[00:07:20.420]pretty much. Improvements are to add more parameters, and to get a better algorithm
[00:07:25.340]to handle less crisp pictures,
[00:07:28.850]and then looking forwards to fix bugs and adding features as needed.
[00:07:33.470]And then it can also detect corners.

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Vision System for a Robotic Gantry

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments