Robotic Tools for Climate-Smart Agriculture
SANTOSH PITLA Associate Professor of Advanced Machinery Systems, Biological Systems Engineering, University of Nebraska–Lincoln
Author
06/07/2023
Added
33
Plays
Description
Transitioning to climate-smart agriculture requires a paradigm shift in how we manage crop inputs and natural resources. Robotics, automation and AI-enabled autonomous systems offer promise for improving productivity of field operations. This talk focuses on the development of multi-robot ground and aerial robotic systems and their potential use cases for deploying climate-smart agricultural practices.
Searchable Transcript
Toggle between list and paragraph view.
- [00:00:00.810]The following presentation is part
- [00:00:02.700]of the Agronomy and Horticulture seminar series
- [00:00:05.850]at the University of Nebraska Lincoln.
- [00:00:08.340]Welcome, everyone, thank you for attending
- [00:00:10.980]both online and in person, the Department of Agronomy
- [00:00:14.130]and Horticulture Seminar series.
- [00:00:17.010]This is our second to last week,
- [00:00:18.510]and so continue to be excited about our talks,
- [00:00:22.500]and today is no exception.
- [00:00:24.990]Along with Dr. Guillermo Balboa and the entire
- [00:00:30.042]seminar committee it's my pleasure to introduce
- [00:00:35.354]this afternoon's speaker, Dr. Santosh Pitla
- [00:00:36.570]from just across East Campus here,
- [00:00:39.960]not even all the way across campus,
- [00:00:41.460]Biological Systems Engineering.
- [00:00:43.500]We've been fortunate to have several great talks
- [00:00:45.720]from BSC folks this past several semesters.
- [00:00:51.870]Today, Dr. Pitla is going to talk to us
- [00:00:55.830]about perhaps the MAARS lab and some of their work
- [00:01:00.720]in robotics, in particular in the area
- [00:01:04.410]of climate smart agriculture,
- [00:01:06.210]which we just continue to hear more and more about.
- [00:01:09.750]So I'll note, as we complete the seminar, folks online,
- [00:01:15.450]please feel free to pose your questions in the Q&A.
- [00:01:19.320]We'll then read them and answer them live here.
- [00:01:23.010]And per usual, for those of you in the room,
- [00:01:25.200]I will pass this mic around to you
- [00:01:26.723]so that the folks online can hear.
- [00:01:29.730]All right, and with that, I'll turn it over to you.
- [00:01:31.770]Thank you.
- [00:01:33.510]All right.
- [00:01:37.590]Everybody hear me okay? Online?
- [00:01:39.930]Yeah? All right.
- [00:01:40.980]Thanks for the kind introduction, Daniel. Appreciate it.
- [00:01:43.380]So, and thank you all for coming.
- [00:01:46.980]And today, like Daniel said, I'll be talking about
- [00:01:51.090]robotic tools for conservation agriculture practices.
- [00:01:55.410]So I am in the BSC department.
- [00:01:58.230]I have both a teaching and research appointment.
- [00:02:01.470]So with that, I want to kind of show
- [00:02:05.790]what I'm gonna talk about here.
- [00:02:07.650]The slide works, the slide advances.
- [00:02:12.090]Oh, I have to click first here. There you go.
- [00:02:16.650]Alright, so this is the overview of my presentation.
- [00:02:19.350]So, Roll of Robotics in Conservation Agriculture Practices,
- [00:02:23.250]and then the concept of Swarm robotics,
- [00:02:27.690]and then some of the Nebraska robotic test-beds
- [00:02:30.810]we are working on right now.
- [00:02:32.160]So you can see the list there.
- [00:02:33.930]And then, you know, next generation
- [00:02:36.360]workforce for smart farms.
- [00:02:38.130]So I wanted to touch a little bit upon,
- [00:02:40.530]today in agriculture, we are talking about AI a lot,
- [00:02:43.920]robotics, then agriculture, the fundamental
- [00:02:46.950]agriculture sciences.
- [00:02:47.970]How are we going to train the next generation workforce?
- [00:02:50.820]We're going to work at this intersection, right?
- [00:02:52.740]So I'm going talk a little bit about that.
- [00:02:56.610]So, FAO says, conservation agriculture,
- [00:02:59.310]there's a very standard definition.
- [00:03:00.750]So how do you do minimum soil disturbance?
- [00:03:04.050]How do you maintain a permanent soil cover?
- [00:03:06.840]And then how do you diversify plant species?
- [00:03:11.220]This is a very difficult thing to do at scale, right?
- [00:03:13.890]So how do we do this at scale?
- [00:03:16.560]And one of the things is, if we can do this with robotics,
- [00:03:22.080]we can boost, you know, water and nutrient efficiency.
- [00:03:25.698]So the way I see it is the robotics and automation,
- [00:03:28.950]I think they're not just, you know, ways to eliminate,
- [00:03:33.720]drudgery or hard work in the farms,
- [00:03:35.850]but also they can contribute to climate smart practices.
- [00:03:38.430]So today I'm going to talk about some of the things
- [00:03:43.770]that we are doing in my MAARS lab.
- [00:03:45.630]Again, MAARS stands for Machine Automation
- [00:03:47.610]and Agricultural Robotics Lab.
- [00:03:49.470]So if you look at precision agriculture today,
- [00:03:51.720]we still talk about dollar per acre.
- [00:03:55.410]With the help of robotics, can we go,
- [00:03:59.220]per square foot management or per plant management
- [00:04:01.890]instead of, you know, per acre?
- [00:04:05.100]So we talk about pounds per acre, gallons per acre.
- [00:04:08.220]When we are talking about application rate,
- [00:04:10.470]can we talk about, micro doses per plant
- [00:04:14.190]or per square foot using robotics?
- [00:04:16.854]You know, this could be possible with robotics.
- [00:04:19.031]So with that, you know, weeds is one of the big things.
- [00:04:23.433]Can we, especially herbicide resistant weeds,
- [00:04:26.850]can we use robotics to take care of weeds?
- [00:04:29.580]Again, chemical savings, intercropping systems,
- [00:04:33.000]again, biodiversity is important,
- [00:04:35.490]but how do you do multiple crops in the same field?
- [00:04:38.307]You have thousands of acres, right?
- [00:04:40.830]So those are some of the things that could be possible
- [00:04:43.980]if you have a good robotic systems.
- [00:04:46.166]And then, improving fertilizer application efficiency,
- [00:04:52.448]so robotics could be a good tool
- [00:04:54.480]to do these kind of climate smart practices.
- [00:04:59.490]Tell you the story of Charlie, the corn plant.
- [00:05:02.160]So today, with the amount of data we have
- [00:05:06.210]from drones and ground machines, we know everything
- [00:05:08.730]about a corn plant, at least, you know,
- [00:05:10.770]what are the needs of the corn plant?
- [00:05:12.570]So more water, more fertilizer.
- [00:05:14.820]You know, there are weeds, and we are talking about IOT.
- [00:05:20.280]Everybody heard about IOT, Internet of Things,
- [00:05:22.830]that's up and coming, right?
- [00:05:24.300]So in the future, you might be getting a call,
- [00:05:26.250]text or a feed from a corn plant,
- [00:05:28.710]so that, you know, we,
- [00:05:30.780]there is so much data and big data today
- [00:05:33.090]that we know a lot about corn plant and its needs.
- [00:05:36.300]So assuming that we have all this data,
- [00:05:39.407]how do we take care of this corn plant, right?
- [00:05:41.700]But you know, Charlie's not alone, right?
- [00:05:43.740]So there are 30,000 plants an acre on average, you know,
- [00:05:47.280]6 million plants in a 200 acre field.
- [00:05:49.994]So there's a reason why we are in the realm
- [00:05:52.830]of digital agriculture, right?
- [00:05:54.900]So there's a lot zeroes and ones,
- [00:05:56.490]there's a lot of matrixes of data,
- [00:05:58.800]all need to be solved
- [00:05:59.910]so that we can take care of the corn plant right?
- [00:06:02.820]So, well, let's send a robot like this,
- [00:06:07.710]even though this robot looks pretty cartoonish,
- [00:06:10.170]there's a lot of intelligence here.
- [00:06:14.400]It has a camera, it can see the obstacles,
- [00:06:17.130]it is connected, it has a decision support,
- [00:06:20.839]it has different types of arms, right?
- [00:06:23.400]So one for pulling the weeds
- [00:06:24.810]and one for spraying, spot spraying.
- [00:06:27.750]Even today, we don't have a robot like this.
- [00:06:29.940]You know, it's pretty hard to design a robot like this,
- [00:06:33.420]especially when we know this is
- [00:06:36.690]the current state of the art.
- [00:06:37.920]We use bigger equipment, right?
- [00:06:40.260]So to actually take care of the farm,
- [00:06:43.620]because you want to cover a lot of ground
- [00:06:45.870]in the shortest time window.
- [00:06:47.550]So that's the reason why we are here,
- [00:06:49.860]where we are using tractors up to 600 horse today, right?
- [00:06:53.520]So how many of you know Nebraska tractor test lab?
- [00:06:58.410]Alright, for some of you don't know
- [00:06:59.700]Nebraska tractor test lab is here on East Campus,
- [00:07:03.000]so that's the only one in the western hemisphere.
- [00:07:06.420]So hundred years of tractor testing happened
- [00:07:09.330]here on East Campus, so if you don't know, please, visit us?
- [00:07:13.440]And actually with this July, we are celebrating
- [00:07:15.420]a hundred years of tractor testing.
- [00:07:19.548]So this is one of the tractors we recently tested
- [00:07:21.750]on the track.
- [00:07:22.583]Again the point I'm trying to make here is,
- [00:07:26.190]it's 600 horse, 60,000 pounds of weight.
- [00:07:32.160]So we know compaction is a problem, right?
- [00:07:36.390]So, and then another important thing here
- [00:07:40.950]is this tractor is made for 40 years,
- [00:07:43.470]four zero on an average, let's say you're using this tractor
- [00:07:46.650]probably 500 hours a year.
- [00:07:49.440]This will be on the farm for 40 years.
- [00:07:51.757]What can you say about software
- [00:07:53.940]and technology on that machine?
- [00:07:55.860]Will it be valid for 40 years?
- [00:07:58.140]It's not going to be, right?
- [00:07:59.820]So this will become technically obsolete pretty fast, right?
- [00:08:03.060]So in my, in our view, like seven to 10 years
- [00:08:06.360]is a good number, at which point replacing
- [00:08:10.440]the entire technology with the new technology
- [00:08:12.210]might be better.
- [00:08:13.043]So same thing with wider planters, right?
- [00:08:15.750]So if the fields are perfectly square, great,
- [00:08:20.700]but if you have a field like this,
- [00:08:23.580]which is highly irregular, wider implements
- [00:08:26.100]are not always good, right?
- [00:08:27.150]So you are either under spraying or overspraying,
- [00:08:29.970]especially if you're using a sprayer.
- [00:08:32.010]And if you have a really wide planter,
- [00:08:35.310]guess what you're doing most of the time:
- [00:08:37.740]either you're backing up or turning
- [00:08:39.510]instead of actually planting.
- [00:08:40.680]So I think it's very important that,
- [00:08:45.150]what are the field attributes, not just about, you know,
- [00:08:48.840]how big is the field, but it's also important,
- [00:08:50.730]what are your field attributes?
- [00:08:52.140]And you need to match your implement set
- [00:08:54.450]or automation or equipment set to your field.
- [00:08:57.870]So you know a little bit about trends,
- [00:09:00.900]you know where we are going.
- [00:09:02.820]So what you see here are two really good examples
- [00:09:06.300]of full autonomy.
- [00:09:08.250]So the first one is on the left side
- [00:09:10.530]of the screen is auto cart.
- [00:09:13.350]Both of them are actually bought by Case New Holland now.
- [00:09:15.990]So both these technologies are owned by Case New Holland.
- [00:09:19.200]On the right side, you can see
- [00:09:21.300]that's called a DOT platform.
- [00:09:22.740]Actually it was a Canadian company and then bought by Raven.
- [00:09:26.310]And Raven got bought by Case New Holland.
- [00:09:28.710]So the idea is the major OEMs have autonomous programs,
- [00:09:35.610]you know, so everyone is ready
- [00:09:37.230]to launch autonomous tractors.
- [00:09:40.655]So you can see the one on the right side
- [00:09:43.560]doesn't look like a tractor anymore.
- [00:09:46.560]So in my view, in the future, you know,
- [00:09:49.110]the last a hundred years, we know how
- [00:09:50.940]the tractor looks like, but the next 25 years,
- [00:09:53.370]the tractors could be completely different.
- [00:09:54.990]They might not look like tractors.
- [00:09:56.970]And this was actually by both the director
- [00:09:59.267]of the tractor test lab and also a John Deere engineer
- [00:10:03.780]who worked there for 40 years.
- [00:10:05.430]So I agree with that.
- [00:10:07.440]In 20 to 25 years, tractor might not look like a tractor.
- [00:10:12.728]So we use drones.
- [00:10:15.180]So now we are looking at drones for spraying also,
- [00:10:18.420]I think primarily we collect a lot of images
- [00:10:22.290]about the crops for, you know, decision making
- [00:10:27.180]or identifying problems.
- [00:10:28.590]But we are also looking into spraying.
- [00:10:30.324]But in the future, we could be doing
- [00:10:32.760]more different kind of operations.
- [00:10:35.910]Again, a little bit of history.
- [00:10:37.440]So 1920, we tested the first tractor
- [00:10:42.000]at the tractor test lab.
- [00:10:46.170]So today, you know, fast forward a hundred years,
- [00:10:50.010]the tractor is a highly sophisticated machine,
- [00:10:53.280]lot of software, a lot of hardware.
- [00:10:55.410]We are talking about full autonomy
- [00:10:57.030]where we are talking about artificial intelligence, right?
- [00:10:59.460]So how does the next generation equipment look like,
- [00:11:02.790]let's say in 2040.
- [00:11:04.160]So it could look like this, you know,
- [00:11:06.210]so there could be small robotic vehicles
- [00:11:09.840]that are going in between the crop rows.
- [00:11:11.580]There could be a slightly bigger one.
- [00:11:14.070]So these are some of the test beds
- [00:11:15.390]that I have in the MAARS lab.
- [00:11:17.400]And the one on the right side,
- [00:11:18.570]we call it as a flex straw robot.
- [00:11:20.733]It's a 60 horsepower machine.
- [00:11:22.670]So I think that to us, that's a good size machine
- [00:11:26.160]that is very valuable for Midwestern row crop.
- [00:11:30.600]So the 60 horsepower machine, you know,
- [00:11:33.480]instead of having just one, you could have multiple, right?
- [00:11:37.260]So there are some companies that are
- [00:11:38.970]actually providing that farming as a service.
- [00:11:41.550]So you see on this left side here, that's a Kubota
- [00:11:44.727]and the right side is out of Australia.
- [00:11:47.640]So the paradigm shift is happening
- [00:11:51.000]where instead of using one big machine,
- [00:11:53.277]can we use multiple smaller machines?
- [00:11:56.250]If I said this, let's say 20 years ago, you'll be like,
- [00:11:59.470]if you have 10 machines, you need 10 operators, right?
- [00:12:02.603]So it doesn't make sense, but today not the case.
- [00:12:05.580]So one person can manage 10 or 20 machines.
- [00:12:09.000]If your control architecture, the Swarm architecture,
- [00:12:12.120]is good, you know, so again, this is
- [00:12:15.270]a paradigm shift that's happening today.
- [00:12:19.590]So again, with that idea, can we replace
- [00:12:23.130]one big machine with multiple smaller machines?
- [00:12:26.520]And again, these are another ones that I've seen at the top,
- [00:12:31.320]you can see there's a DOT platform.
- [00:12:33.360]And then Sabanto the orange machine.
- [00:12:36.030]And the really small one is an Earth Sense.
- [00:12:38.370]I really don't know about that if it's going to work
- [00:12:40.590]in a real field condition, but at least it could be
- [00:12:43.710]a good field scouting tool, you know, so the smaller ones,
- [00:12:48.617]so the idea is can we replace one 600 horse tractor
- [00:12:53.700]with multiple tractors, in this case,
- [00:12:57.600]let's say 10, 60 horsepower machines so that, you know,
- [00:13:01.290]bigger the field, you just have more number of machines.
- [00:13:06.510]So kind of a field scale neutral solution.
- [00:13:09.180]If you have a smaller field use, let's say two,
- [00:13:11.700]if you have a larger field, use 20 of this.
- [00:13:14.040]So the technology is going to be the same,
- [00:13:16.230]but you scale the number of machines up and down.
- [00:13:19.170]Okay?
- [00:13:20.086]So with that, we are looking at the coordination strategies.
- [00:13:25.290]Your planting is going to be different
- [00:13:27.030]from baling and picking operation.
- [00:13:28.590]For example, on the planting side,
- [00:13:30.870]they all are doing the same operation, right?
- [00:13:33.030]These three robots on the left, they're all planting.
- [00:13:35.767]So we call that as homogeneous operation
- [00:13:38.640]because there's just planting, whereas baling and picking
- [00:13:42.390]is heterogeneous because there's a baler that bales,
- [00:13:45.930]there's a picker.
- [00:13:46.763]So they're both, both those robots
- [00:13:48.480]are doing different operations.
- [00:13:50.310]And the harvesting is another big example
- [00:13:52.950]where you need communication all the time.
- [00:13:55.320]And probably the most complex, because once
- [00:13:58.290]a grain cart one is full, it needs to go.
- [00:14:01.920]And then grain cart two need to sync up with the combine
- [00:14:04.800]so that it can transfer the grain.
- [00:14:06.600]So we have to be thinking about all these
- [00:14:09.330]and program all of this into the control architecture.
- [00:14:13.260]So when we are talking about multiple robots,
- [00:14:16.158]and homogeneity and heterogeneity is also important.
- [00:14:21.540]So with that, I'll just go over
- [00:14:25.920]some robotic test beds we have.
- [00:14:28.980]So this is a smaller robot that we launch
- [00:14:33.930]in between the crop rows, you know,
- [00:14:36.120]primarily you can get corn population, you know,
- [00:14:38.940]so what's the population count?
- [00:14:40.620]And also you can put different types of sensors
- [00:14:43.170]so you can sense the microclimate underneath the canopy.
- [00:14:47.310]So drones are great, you can fly over and get a lot of data,
- [00:14:50.100]but what's happening under the canopy, right?
- [00:14:52.407]So you can use this smaller machines or robots.
- [00:14:57.750]And again, under the canopy, again, as AG engineering,
- [00:15:00.960]our challenge is under the crop canopy, there is no GPS.
- [00:15:04.003]So how does a robot navigate?
- [00:15:05.958]So we're using laser sensors to navigate between the crop.
- [00:15:09.900]So...
- [00:15:12.517]And then, so this is our
- [00:15:14.820]close to 60 horsepower robotic machine.
- [00:15:18.570]So it can run 14 hours because
- [00:15:21.780]as long as there's gas in the tank, it can work.
- [00:15:24.270]So I know there's a lot of talk about electrification,
- [00:15:26.430]but I'll show you some slides
- [00:15:27.690]why electrification might not be a great idea,
- [00:15:31.620]especially for Midwestern agriculture.
- [00:15:34.939]So this machine here, so this is a four-wheel steer,
- [00:15:39.300]four wheel drive.
- [00:15:40.133]So you see four wheel drive on a truck, right?
- [00:15:42.240]So trucks are four wheel drive.
- [00:15:43.440]But with this we can also steer all four wheel.
- [00:15:47.070]So what I'm am I talking about?
- [00:15:48.157]So if you can see, you can turn all the wheels 90 degrees
- [00:15:52.680]and you can go to the next rows if you want.
- [00:15:56.640]So the idea is one of the big inefficiency
- [00:16:00.870]in the fields is turning time.
- [00:16:03.270]So whenever you're planting, you know, or spraying,
- [00:16:05.673]you have to turn around to get to the next row.
- [00:16:09.090]So that's the biggest inefficiency in a field.
- [00:16:12.180]So how do we improve the efficiency?
- [00:16:14.190]By innovating steering mechanism.
- [00:16:16.560]So that's what we are trying to show here.
- [00:16:19.920]And then since it's autonomous vehicle, right?
- [00:16:22.950]So we need to test for obstacle detection avoidance.
- [00:16:27.000]So that's something we work on in the lab.
- [00:16:29.910]And then, you know, one of the first application we did
- [00:16:34.170]is the phenotyping, right?
- [00:16:35.820]So how many of you know spidercam facility, right?
- [00:16:39.570]So yeah, so we have a one acre spider cam facility,
- [00:16:42.810]but the big thing with this is it's only one acre, right?
- [00:16:47.160]So the camera can go in only one acre.
- [00:16:49.680]So we have the same sensors on this robot.
- [00:16:52.230]So if you want to phenotype your field on each campus,
- [00:16:57.090]we can use this machine to phenotype.
- [00:16:59.520]So we are using the RTB camera, the same type of sensors
- [00:17:05.010]that we all we have on the spidercam.
- [00:17:09.720]So you can see here when we took this robot
- [00:17:14.280]to the spidercam, so on the right side
- [00:17:19.320]is when we were actually collecting the data,
- [00:17:21.540]and you can see the spidercam in the background.
- [00:17:23.850]So we have very good correlation between the data
- [00:17:26.070]we collected using a robot and the spider cam.
- [00:17:28.980]So we were actually phenotyping the soybean field,
- [00:17:32.087]so there were different varieties.
- [00:17:33.990]You can see a little bit of data here,
- [00:17:35.970]days after planting and the average plot height.
- [00:17:39.570]So you can kind of see the differentiate between
- [00:17:43.320]the varieties of soybean here using
- [00:17:46.320]the ultrasonic height sensor that we had there.
- [00:17:51.600]And then we can get the green pixel fraction.
- [00:17:54.726]And again, the idea here is if you're doing
- [00:17:58.380]a lot of field trials, why not use robots
- [00:18:01.440]to accelerate your investigations, right?
- [00:18:03.840]So, that's kind of the idea is we could automate
- [00:18:08.310]some of these processes so that we can progress
- [00:18:11.160]and advance investigations in the field. Yeah.
- [00:18:16.320]And then we submitted a grant to the USFDA,
- [00:18:20.460]we'll see if it works or not.
- [00:18:23.580]But the idea is can we use this robot to do multiple things?
- [00:18:27.180]Can we use this robot to do cover crop inter-seeding?
- [00:18:30.180]Can we use it to do cover crop drill?
- [00:18:32.940]Because we know that's one of the important
- [00:18:34.786]climate smart agriculture practice
- [00:18:37.126]and farmers run out of time after they harvest.
- [00:18:40.530]You have a very short time window,
- [00:18:42.390]you're strained for labor.
- [00:18:43.834]And how do you do cover cropping at scale?
- [00:18:48.300]So today, the common technique is you broadcast
- [00:18:51.060]a cover crop seed, but if you're talking
- [00:18:53.070]about no-till agriculture, there's a lot of biomass, right?
- [00:18:56.670]If you're simply spreading, you know,
- [00:18:58.140]the germination might not be ideal, you know, or perfect,
- [00:19:01.530]so why not send a robot, right?
- [00:19:04.200]That can actually drill into
- [00:19:05.970]the no-till field after your harvest.
- [00:19:09.990]So, and then we can add soil sensing to this robot, right?
- [00:19:15.300]Phenotyping and air sampling.
- [00:19:17.100]So that we can characterize, you know, soil health,
- [00:19:20.430]plant health and disease medication.
- [00:19:22.080]So we can have this system,
- [00:19:26.010]we can activate this flex-ro robot with, you know,
- [00:19:29.117]let's say a cover crop drill.
- [00:19:30.720]And then if it follows a combine, you know, and drills,
- [00:19:35.820]you know, because the time windows are so short,
- [00:19:39.600]so you can use this robotic system.
- [00:19:41.255]And again, I keep thinking about,
- [00:19:45.621]if I have to offer a robotic system to the farmer today,
- [00:19:49.320]what would be they interested in?
- [00:19:52.020]Planting is a very high risk operation.
- [00:19:53.910]You don't want to, right?
- [00:19:55.080]So planting, the trust is not there yet, right?
- [00:19:58.307]So, what else can we do?
- [00:19:59.640]Maybe, you know, a cover crop drill is a perfect operation
- [00:20:02.640]because they'll be okay to try out this technology.
- [00:20:06.600]So that's something I'm looking into, you know,
- [00:20:08.550]to work with farmers.
- [00:20:09.840]And then soil sampling is a huge, huge thing
- [00:20:12.570]that even today it's a very manual
- [00:20:15.660]and laborious process, right?
- [00:20:17.550]So can we use this robot, you know, and can we pick,
- [00:20:21.360]you know, also in situ sends the leaves, you know,
- [00:20:25.680]on a corn plant, like for example,
- [00:20:27.750]this is in our greenhouse, so we use this robotic arm
- [00:20:31.830]to know what is the chlorophyll content
- [00:20:33.540]and leaf temperature.
- [00:20:35.010]So we can fly drones, right?
- [00:20:36.540]And get data, but we can ground through them
- [00:20:39.270]with these actual robotic devices instead of a person
- [00:20:41.760]going and ground trooping it.
- [00:20:43.530]Maybe we can send robots to do that, you know,
- [00:20:45.750]so, and then air sample collection,
- [00:20:49.770]this is one of our ideas that went in the proposal.
- [00:20:52.140]So we'll see, you know, so we can put this air sampling
- [00:20:54.960]device on the equipment.
- [00:20:56.670]We can collect images to look at the disease mitigation.
- [00:21:00.690]This is an example from wheat spike counting.
- [00:21:04.440]So using machine vision, we can count
- [00:21:06.420]how many wheat spikes there are.
- [00:21:08.460]So, same technique can be applied
- [00:21:10.200]to look at disease pressure, yeah.
- [00:21:17.250]So, with the...
- [00:21:19.980]This year, next week is gonna be pretty exciting for us
- [00:21:23.220]because we are trying to plant with this robot
- [00:21:26.430]for the first time ever robotically.
- [00:21:28.260]So we are going to do corn planting next week.
- [00:21:31.410]I think we are ready to go if the weather permits.
- [00:21:35.130]So I mean if you know Rogers Memorial Farm?
- [00:21:40.050]Yeah, so we are going to plant there.
- [00:21:42.180]So if it is successful, then it'll be the first five acre
- [00:21:46.830]of robotically planted corn in our research fields.
- [00:21:51.960]I don't want to claim Nebraska because I've not done
- [00:21:54.420]my research yet, but at least in our research fields,
- [00:21:57.720]this will be the first time we'll be planting some fully
- [00:22:00.180]robotically or autonomously.
- [00:22:02.010]So that's how many, how many rows?
- [00:22:03.930]Two rows, Two rows.
- [00:22:04.763]Yeah, just two rows.
- [00:22:05.910]We can actually put four rows on this equipment,
- [00:22:08.070]but we are starting with the two rows
- [00:22:09.900]because we are trying to do an innovative mechanism here.
- [00:22:13.260]I can talk about it later, but yeah,
- [00:22:15.660]so we are going with only two rows here, yeah.
- [00:22:19.800]So we also have something called Flex-Ro Mini,
- [00:22:22.860]with the goal of automating the operations
- [00:22:26.700]in a vegetable farm too.
- [00:22:28.020]So I'm working with Sam Wortman
- [00:22:31.530]and we are sketching a plan to see
- [00:22:34.620]if we can plant lettuce robotically in August.
- [00:22:39.030]So if it all works, we'll be able to do that.
- [00:22:42.060]And also we can use this machine as a dump wagon.
- [00:22:44.580]So you harvest produce, put it on this machine
- [00:22:47.550]and this is going to go drop the produce, you know,
- [00:22:50.490]at the edge of the field and come back, you know,
- [00:22:52.740]so be like a robotic helper, yeah.
- [00:22:56.516]All done by seed, right?
- [00:22:58.230]Yeah, seed lettuce by seed, yeah.
- [00:23:03.210]So now we're talking about these robots, correct?
- [00:23:07.860]But these robots are smaller compared to
- [00:23:09.930]your 12 row 16 row planters, right?
- [00:23:13.650]The tanks are obviously smaller.
- [00:23:16.320]So how are you going to refill the seed?
- [00:23:19.170]We don't want these ground robots to go back and forth.
- [00:23:23.460]Why not use drones to refuel the seed, right?
- [00:23:27.060]So that's kind of the idea here.
- [00:23:29.400]So can we use that drone to drop off the seed
- [00:23:33.810]on this ground machine so that this ground robot
- [00:23:37.080]is always moving, right?
- [00:23:39.270]So that, you know, the drones will go, take off,
- [00:23:43.080]land on the machine, drop off the seed and go back.
- [00:23:46.050]So we are utilizing the full potential
- [00:23:48.815]of both type of ground robots, both ground robots
- [00:23:53.220]and then aerial robots.
- [00:23:54.630]Aerial robots can cover long distances in a short time,
- [00:23:57.720]but then they can be there only for 20, 25 minutes.
- [00:24:00.660]Ground machines don't have the problem.
- [00:24:02.790]So we're using the best of both worlds, you know,
- [00:24:05.070]so the big limitation is drones can carry
- [00:24:09.390]only up to 20 pounds or so right now,
- [00:24:13.230]but then we want to prove out the concept,
- [00:24:15.510]the coordination strategy.
- [00:24:17.730]We want to do a scale model and prove it
- [00:24:20.370]so that when drones get bigger, you know,
- [00:24:22.350]so we can use the same technique.
- [00:24:25.830]So here's an example, nothing special here
- [00:24:29.070]we are flying a drone,
- [00:24:33.480]hard to see too.
- [00:24:34.950]So...
- [00:24:39.510]Yeah, there's a little drone,
- [00:24:40.830]you see it's landing there. (chuckles)
- [00:24:45.090]The special thing there is,
- [00:24:47.040]it's took off autonomously and landed autonomous.
- [00:24:51.450]So the typical DJI drones that we use in agriculture, right?
- [00:24:55.980]So the software is closed,
- [00:24:57.930]so we cannot make any modifications to the software.
- [00:25:00.750]So what we are trying to do is developing these drones
- [00:25:03.570]from the scratch so that we can program it.
- [00:25:05.940]So we have, we program the flight controller
- [00:25:08.880]so that we can take off and land.
- [00:25:10.830]If you have to increase the capacity,
- [00:25:12.330]we select different propellers, you know,
- [00:25:15.030]so we need that type of customization
- [00:25:17.490]or else we cannot put our algorithms on those drones.
- [00:25:20.760]So that's kind of, we are in early stages of doing that.
- [00:25:24.150]And here is another example.
- [00:25:27.447]So there are two drones.
- [00:25:29.820]The one in the front, close to the door is the leader.
- [00:25:33.720]If that leader moves to the right,
- [00:25:36.300]the follower moves to the right too.
- [00:25:38.250]So we are trying to do coordination of drones,
- [00:25:41.430]not just coordination of ground robot and aerial robot,
- [00:25:44.610]but aerial robots also.
- [00:25:47.100]So all of this becomes very important because again,
- [00:25:49.320]going back to that example of ground robot planting,
- [00:25:54.330]maybe you need two or three drones to service one machine,
- [00:25:57.150]you know, and vice versa.
- [00:25:58.380]So we are talking about
- [00:26:00.000]multi-robot systems and coordination.
- [00:26:02.670]So that's the reason why we are
- [00:26:05.040]looking into these algorithms so that we can program
- [00:26:08.970]the drones like we want to.
- [00:26:15.197]So, let's see.
- [00:26:19.290]So a lot of talk about driverless cars.
- [00:26:23.940]I don't know why we need driverless cars
- [00:26:25.920]because I don't wanna buy a Ferrari
- [00:26:27.420]and let it drive itself.
- [00:26:28.920]I want to drive it,
- [00:26:30.420]but (chuckles) there is real need on the farm though.
- [00:26:33.840]So, because we cannot find skilled labor
- [00:26:37.170]or there's no labor, you know, so we need, if we need that,
- [00:26:44.040]it's important that the robot not just,
- [00:26:48.420]like detecting the obstacle and stopping
- [00:26:50.280]like you saw before, but it also need
- [00:26:52.260]to positively identify what that obstacle is.
- [00:26:55.020]Is it a bale? Is it a cow?
- [00:26:57.360]Is it a center pivot?
- [00:26:58.680]They're very hard to detect, you know,
- [00:27:00.990]so we are training these objects that are typically,
- [00:27:04.680]that typically happen on a farm.
- [00:27:06.450]So that if my flexor robot is going
- [00:27:09.480]and it sees another equipment, it says, oh, it's a friendly,
- [00:27:12.330]I need to go close and work with it, not run away from it,
- [00:27:15.540]you know, so we are training these models
- [00:27:18.300]so that we know what are the actual objects
- [00:27:21.240]that the robot is seeing, okay?
- [00:27:23.670]So for this, we need a lot of data sets,
- [00:27:26.160]and I was surprised we, even though we are the headquarters
- [00:27:30.810]of the center pivot manufacturers,
- [00:27:32.640]there's no public database where we can get pictures
- [00:27:35.580]of center pivot, you know, so this summer
- [00:27:39.210]we want to do some data collection
- [00:27:41.160]of all these obstacles from our farms
- [00:27:43.440]so that we can train our machine learning models, okay?
- [00:27:47.880]So center pivots are very hard, so you need to identify,
- [00:27:51.330]because sometimes the model thinks it's a tractor
- [00:27:54.030]because it has wheels, you know,
- [00:27:55.620]sometimes because of the structure, it's so different.
- [00:28:01.200]And then, so this is a project called "Cloud -E"
- [00:28:07.230]that we are looking at, assuming that we solve
- [00:28:12.720]the problem of bad connectivity on the farms, right?
- [00:28:17.820]We can, if all the robots can be connected
- [00:28:21.000]through the Cloud, then we can actually do cloud computing.
- [00:28:25.110]So today, if you want to deploy AI, or if you want to do
- [00:28:28.410]a lot of computation on individual machine,
- [00:28:31.320]the problem is you need expensive hardware on each robot.
- [00:28:35.310]So we say that, okay, we need 10 robots,
- [00:28:38.580]but you have expensive hardware on each robot
- [00:28:42.000]to do this artificial intelligence,
- [00:28:43.830]then it's not going to work out because
- [00:28:46.260]that's not going to be, that's gonna be very expensive.
- [00:28:49.110]So if we solve the problem of this connectivity,
- [00:28:52.230]then we can do computation,
- [00:28:54.000]a lot of computation on the Cloud.
- [00:28:56.370]So that's where this is, this is an architecture
- [00:29:00.390]of how we can take data from the drones,
- [00:29:03.500]how we can take data from the tractors and the robots,
- [00:29:07.530]send the data to the Cloud so that instead of doing
- [00:29:10.980]heavy computation on individual machine, you know,
- [00:29:13.590]the Cloud is doing that computing for you
- [00:29:15.480]and sending the result, you know,
- [00:29:17.909]so at ENREEC we have good connectivity.
- [00:29:20.460]Now there is internet throughout the ENREEC field,
- [00:29:23.550]Eastern Nebraska Research and Extension Center, right?
- [00:29:25.890]So, we'll try some of this out, you know,
- [00:29:29.850]especially with nitrogen management,
- [00:29:32.220]I know many of your work in that area.
- [00:29:34.860]So the nitrogen, can we send the data from the drones
- [00:29:39.024]and high clearance sprayers of the crop status
- [00:29:42.930]to the Cloud so that the Cloud is going
- [00:29:45.360]to continuously compute, you know, okay,
- [00:29:48.030]this is the nitrogen status and here's the prescription,
- [00:29:51.060]you know, and the robot can go do that.
- [00:29:53.915]So you're continuously monitoring the crop status
- [00:29:58.200]and whenever there is a need for it to go do application,
- [00:30:01.980]it can go to that.
- [00:30:03.300]So...
- [00:30:06.480]And then human machine interface and gesture control.
- [00:30:10.260]So, today, okay, you have a remote,
- [00:30:14.793]you can take the robot to the field edge, launch it,
- [00:30:18.720]and you give it a path using the GPS,
- [00:30:21.150]it'll follow the path and do the operations.
- [00:30:23.070]But how can we seamlessly integrate humans, right,
- [00:30:27.960]into the mix, you know, into the control loop?
- [00:30:31.770]So what we are trying to do is we are converting
- [00:30:34.320]standard hand signs into robot signals.
- [00:30:37.980]So in this case, if he, this is a forward sign we program,
- [00:30:42.000]because we are going to have cameras anyways on the robot,
- [00:30:44.460]so might as well use that, right?
- [00:30:46.200]So use a hand gesture to move the robot
- [00:30:49.110]instead of having a separate remote.
- [00:30:51.870]So you can see, so when he is doing
- [00:30:55.860]the hand gestures, right?
- [00:30:57.360]So it moves or stops based on that.
- [00:31:00.480]So we have programed more than 10 hand signs now.
- [00:31:03.660]So we'll test it in this summer, you know,
- [00:31:05.640]in the field to see how it goes.
- [00:31:10.020]So the next thing is the, you know, we talk about robotics,
- [00:31:15.000]climate smart agriculture, anytime we say,
- [00:31:17.550]and then we say diesel engine, people will be like,
- [00:31:19.447]"Oh, it's not electric, why is it not electric?"
- [00:31:22.770]You know, the reason is the power density is not there yet
- [00:31:28.440]for batteries, especially if you're talking
- [00:31:31.050]about Midwestern farming, right?
- [00:31:32.970]Because planting season, you'll be there out in the field
- [00:31:38.430]15 to 16 hours a day, hours a day, right?
- [00:31:41.400]So, and then you have, you need a lot of power
- [00:31:44.550]and there is no charging time, right?
- [00:31:46.230]So it's not like you can take your tractor
- [00:31:48.840]and charge overnight, so there's no overnight,
- [00:31:51.060]you know, so we cannot wait for that.
- [00:31:53.373]So what we did is we collected data from this equipment,
- [00:31:59.217]something called canvas data, and also we instrumented
- [00:32:03.000]the drawbar, PTO and hydraulics to figure out,
- [00:32:06.270]you know, so how much power are these implements
- [00:32:09.450]taking from the tractor?
- [00:32:12.480]So you can see here on the y-axis is kilowatt-hour,
- [00:32:17.310]that's the energy.
- [00:32:18.870]So these are the two different tractors.
- [00:32:20.820]And again, Case B is a much bigger tractor.
- [00:32:24.780]You can see it's a-48 row planter.
- [00:32:29.760]This is a 16-row
- [00:32:31.290]So we are not trying to compare here, but you know,
- [00:32:34.350]those are the equipments that we had access to.
- [00:32:36.660]So we collected data so that we can come up
- [00:32:39.420]with what is a kilowatt-hour, you know?
- [00:32:42.270]So, and then we have the battery energy densities, you know,
- [00:32:48.690]kilowatt-hour per cubic meter, you know,
- [00:32:51.630]335 and then that's the weight or mass energy density, okay?
- [00:32:58.410]So the bottom line is we know how much kilowatt-hour
- [00:33:06.540]1,117 kilowatt-hour, right?
- [00:33:10.260]And then we calculated the mass, alright,
- [00:33:15.990]with our 15 hours of operation.
- [00:33:18.540]That's how big a battery you need.
- [00:33:22.680]The only thing that works in favor of tractors
- [00:33:25.350]is we need to balance the tractor anyways, so we need more,
- [00:33:28.950]more mass on the tractor or weight on the tractor, right?
- [00:33:31.770]So that's the only advantage we have,
- [00:33:33.480]but still that's a 12 ton and five ton.
- [00:33:40.170]That's pretty big masses, you know?
- [00:33:42.570]Those are replacement batteries?
- [00:33:44.070]Yeah, replacement batteries, exactly.
- [00:33:46.200]Swapping, right? (chuckles)
- [00:33:47.033]You got to swap, you know?
- [00:33:49.470]Yeah.
- [00:33:50.505]Again, then comes the infrastructure, right?
- [00:33:52.087]You know, so are we going to address that?
- [00:33:55.500]So that's kinda where we are going with
- [00:33:57.330]in terms of electrification.
- [00:33:58.830]But here again the idea of Swarms
- [00:34:01.920]can help with electrification too,
- [00:34:04.530]because if you think about instead of one 600-horse tractor,
- [00:34:09.270]you have 10, 60-horse tractors,
- [00:34:11.820]and let's assume they're all electric, right?
- [00:34:14.790]So maybe six are working, four are charging
- [00:34:18.750]and then you schedule them, right?
- [00:34:21.510]So that is a possibility.
- [00:34:23.730]So if we are going the route of Swarms,
- [00:34:26.100]then some of them are charging and some of them are,
- [00:34:29.571]you know, working, but the other alternative
- [00:34:31.650]is to have a swapable battery infrastructure, you know?
- [00:34:34.170]So that's another way.
- [00:34:36.450]So we are continuing to collect a lot more data,
- [00:34:39.300]and this is just for planting,
- [00:34:41.070]which is a low draft application,
- [00:34:43.383]very low draft application compared to
- [00:34:45.660]if you're doing tilling or any type of, you know,
- [00:34:48.583]soil tool interaction at a deeper level,
- [00:34:51.420]then these power requirements are going to be
- [00:34:53.700]much, much higher, yeah.
- [00:34:59.160]One of the other things is the cybersecurity
- [00:35:03.210]of this equipment.
- [00:35:05.430]So we are talking about IOT and connected,
- [00:35:07.710]everything is connected, but the problem is
- [00:35:10.890]there are security issues right now.
- [00:35:15.090]So you've, so the JBS packing facility, you know,
- [00:35:21.115]it had a ransomware.
- [00:35:23.624]So the ransomware is where like these unauthorized parties
- [00:35:27.780]will hack into a system and they'll not give you access
- [00:35:30.600]until you pay them something.
- [00:35:32.940]That's why it's called a ransomware.
- [00:35:34.680]It actually already happened on AGCO planting equipment.
- [00:35:39.540]So 15 days during the planting season,
- [00:35:44.610]your million dollar equipment is sitting like a brick,
- [00:35:49.260]not a good situation, right?
- [00:35:51.150]So these are some of the things
- [00:35:55.124]that we need to be thinking about,
- [00:35:55.957]especially if you're talking about connected hardware
- [00:35:58.950]and connected equipment and autonomous equipment.
- [00:36:03.136]The problems are exacerbated because there is
- [00:36:06.676]human intervention is much lesser,
- [00:36:09.570]so it's hard to monitor, okay, am I applying something,
- [00:36:12.300]or is the machine just moving?
- [00:36:13.620]You know, so we are doing
- [00:36:14.453]a case study on nitrogen application.
- [00:36:16.530]What happens if a nitrogen applicator is compromised,
- [00:36:21.510]even though it's saying, okay, a hundred pounds per acre
- [00:36:24.210]or whatever the perfect rate,
- [00:36:25.470]but it's not actually doing it,
- [00:36:27.330]and you wouldn't realize until after you go to harvest.
- [00:36:32.790]Yeah.
- [00:36:33.623]So that's why the cybersecurity aspect is important,
- [00:36:37.200]you know, so whether it's a precision operation
- [00:36:39.600]or application rate, you know,
- [00:36:42.423]you have an autonomous tractor getting onto the Interstate,
- [00:36:47.610]whether, you know, it's hacked or is there a malfunction?
- [00:36:50.970]You know, it can be anything, right?
- [00:36:53.370]And then the ransomware, right?
- [00:36:55.410]So that's why connected equipment is not always, you know,
- [00:37:01.170]it's important for remote diagnostics and everything,
- [00:37:04.170]but we need to put, you know,
- [00:37:09.120]lot of preventive measures in place, you know,
- [00:37:12.150]because people are not even thinking
- [00:37:13.500]about security at all today.
- [00:37:18.360]So I think in the future we would be seeing, you know,
- [00:37:24.840]the traditional conventional precision AG equipment.
- [00:37:27.480]And when I say future in the next 10 to 15 years,
- [00:37:30.570]I would say, you will see robotic equipment
- [00:37:34.170]working hand in hand.
- [00:37:36.000]And then there will be drones, which already happened today.
- [00:37:40.620]But I think, if you want to go
- [00:37:42.930]towards that climate smartness,
- [00:37:45.960]how do we increase biodiversity on a farm?
- [00:37:49.770]Why not put multiple crops, crop mix, you know?
- [00:37:52.350]So can we use robots to do that?
- [00:37:54.570]I think the answer is yes, and then the computing,
- [00:37:59.790]you know, some of the computing might be happening
- [00:38:02.160]on the edge on the equipment, and some of them
- [00:38:06.210]could be offloaded to the Cloud.
- [00:38:08.910]And then there are many bottlenecks here,
- [00:38:11.100]and one of the thing is the connectivity piece, right?
- [00:38:13.590]So here today in Lincoln, we have 5G, right?
- [00:38:17.310]But you go out, you know, to the field,
- [00:38:19.800]there's not even one G, right?
- [00:38:22.440]So, as robotic researchers and designers,
- [00:38:27.510]we need to be thinking we think about that,
- [00:38:29.490]so if there is no connectivity, how does a robot react?
- [00:38:33.240]What are the onboard resources that it has? You know?
- [00:38:37.500]And then I say animal systems here,
- [00:38:40.950]because that's also an important aspect, is, can we use,
- [00:38:52.177]over the last three, four years have seen
- [00:38:53.820]like going away from synthetic fertilizer, right?
- [00:38:56.070]So that's where the, okay, can we do integrated crop
- [00:38:59.460]and animal systems, right?
- [00:39:01.500]So just think about this just for corn and soybeans,
- [00:39:07.020]if you wanna talk to a farmer during the planting season,
- [00:39:09.660]they'll be like, "No, I don't have any time
- [00:39:12.990]to do anything, right? (chuckles)
- [00:39:14.520]Now imagine you are putting crops and animals together,
- [00:39:18.453]your logistics and stress will be exponential, right?
- [00:39:23.460]So I think you need to, when you're thinking about
- [00:39:25.680]these integrated crop system, you need to have
- [00:39:27.690]highly automated systems that actually work in reality.
- [00:39:31.530]So that's one of the ways to envision this type of a system,
- [00:39:36.180]you need to have highly automated systems
- [00:39:38.040]or else it's very difficult
- [00:39:39.750]to manage the logistics, you know?
- [00:39:41.460]So, and now, like I was talking about, you know,
- [00:39:45.660]how do we train students, right?
- [00:39:48.574]We talk, we are talking about robotics,
- [00:39:50.100]we are talking about agriculture, soil science, weeds.
- [00:39:55.890]So I think the next generation workforce, you know,
- [00:39:58.770]have to be trained well, and here is an example.
- [00:40:04.730]So this is a robotics competition
- [00:40:07.304]that we participated last year.
- [00:40:09.150]So this is actually a cotton picking robot.
- [00:40:12.132]So here's a camera.
- [00:40:14.280]It's looking at a cotton and white background,
- [00:40:19.260]and white background, cotton is white and the background
- [00:40:21.446]is white, so it's very difficult to differentiate, right?
- [00:40:25.500]So that's why we are using something called a depth camera,
- [00:40:28.350]so that it not only gives you the color of the cotton,
- [00:40:32.100]but also how far it is from the camera
- [00:40:35.280]so that we can send this robotic arm
- [00:40:39.210]to actually go and pick the cotton.
- [00:40:43.410]Okay, so apparently we can do some cotton research too,
- [00:40:47.550]in Nebraska, is there any cotton here? (laughs)
- [00:40:50.250]There are Texas A&M, there's Clemson, we beat all of them,
- [00:40:53.640]you know, (chuckles) so a lot of southern states
- [00:40:58.080]did not do very well because we won the competition.
- [00:41:00.090]So we have an opportunity here. (chuckles)
- [00:41:02.670]So I think one of the other competition
- [00:41:08.040]that's coming up is this farm robotics challenge.
- [00:41:11.580]And thanks to Amit and Mandeep
- [00:41:14.460]we're actually growing weeds in the greenhouse.
- [00:41:17.400]I never thought we'll be growing weeds.
- [00:41:19.245]We always, I think about getting rid of weeds, right?
- [00:41:22.770]So it's like we are growing weeds
- [00:41:23.970]to get rid of them, actually. (chuckles)
- [00:41:26.490]But the important thing is
- [00:41:28.568]we need to train our machine vision models, right?
- [00:41:31.830]So how does a weed look like versus corn plant?
- [00:41:35.430]So yeah, thanks to Amit, we are using his greenhouse
- [00:41:38.750]to grow the weeds so that we can train
- [00:41:41.400]our machine vision models.
- [00:41:43.200]And our goal is, you know, can we use this robot here,
- [00:41:47.430]the Flex-Ro Mini, you know, to actually,
- [00:41:51.050]either spray or mechanically pull.
- [00:41:54.270]And I think in the future we will be thinking about
- [00:41:57.240]like integrated weed management, you know.
- [00:41:59.970]If it's a herbicide resistant weed, what are you gonna do?
- [00:42:03.333]We got to pull them or do some other technique, right?
- [00:42:07.380]So we just got started with this, but you know,
- [00:42:11.370]May 10th is just for the competition,
- [00:42:13.380]but we'll continue to work in this area, you know, so,
- [00:42:18.660]and then, yeah, here is a team that I have,
- [00:42:22.410]obviously they're the one who's doing all the work.
- [00:42:24.420]I'm just talking here.
- [00:42:25.680]So yeah, it's a pretty good team.
- [00:42:29.250]So yeah. With that, yeah, this is the Rogers Memorial Farm.
- [00:42:35.580]So here's what's happening is
- [00:42:36.960]that drone is taking off autonomously.
- [00:42:39.210]We are shooting it with another drone
- [00:42:40.680]that we are actually manually running.
- [00:42:43.680]So...
- [00:42:47.310]Yeah, with that, yeah.
- [00:42:48.720]I'll take any questions you have, yeah.
- [00:42:50.850]We do have plenty of time for questions.
- [00:42:52.590]Yeah! Any folks here in the room,
- [00:42:54.150]those of you online, again, typed in?
- [00:42:57.420]I'll pass the mic around to anyone here in person.
- [00:43:08.340]Very interesting.
- [00:43:10.650]I'm intrigued with all of the pictures-
- [00:43:12.960]Yeah? And sensors that...
- [00:43:15.480]I work with corn. Yeah.
- [00:43:17.010]And so I think about having this little robot
- [00:43:22.530]go through the field and identify something different.
- [00:43:26.970]Yeah. Now, to what extent
- [00:43:29.067]do you envision this being able to tell you
- [00:43:31.590]whether it's nitrogen, phosphorous, potassium, pH,
- [00:43:36.447]all those other variables or soil type?
- [00:43:41.010]Where will that decision be made?
- [00:43:43.740]How do you put that together?
- [00:43:45.360]Yeah, that's a really great question. (chuckles)
- [00:43:49.506]I don't have a good answer probably, but what I can say is,
- [00:43:52.320]so far I've focused on building these robotic equipment,
- [00:43:55.530]you know, and I'm trying to identify use cases.
- [00:43:59.130]So lot of sensing work that is done today, right?
- [00:44:03.930]Using manual methods or using, let's say spidercam
- [00:44:08.340]or any other methods, we can easily put that
- [00:44:11.370]on this equipment so that we can actually,
- [00:44:14.700]lot of times, you know, the frequency of data collection
- [00:44:20.040]is limited by how laborious the process is, right?
- [00:44:23.610]So we want to take that out of the equation.
- [00:44:25.500]So we have this robot, one of the things I envision is,
- [00:44:29.460]we'll have this robot 25 by seven in the field
- [00:44:32.640]always monitoring for things.
- [00:44:35.160]You know, so if there is scientists
- [00:44:38.790]who are developing sensors, we can easily
- [00:44:40.549]put them on here and collect data.
- [00:44:46.890]Let me add one more. Yeah?
- [00:44:48.450]The thought here is that in the case of corn-
- [00:44:52.620]Yeah? within a leaf
- [00:44:56.010]there can be color changes and you envision
- [00:44:59.250]having sensors that can scan for potassium deficiency
- [00:45:03.360]versus nitrogen, potassium turns brown on the margins,
- [00:45:08.790]nitrogen down the middle.
- [00:45:10.537]Do you think you'll be able to get that out of it?
- [00:45:15.150]Yeah. So again, yeah, that's a great question.
- [00:45:18.690]I'm not into sensing specifically,
- [00:45:21.210]but if there are proven methods, right?
- [00:45:25.020]Like one thing I heard is like the drones
- [00:45:27.720]cannot see the side view of the plants, right?
- [00:45:30.840]So, I mean, we can put cameras, you know,
- [00:45:34.329]so that you are sensing this way, looking sideways,
- [00:45:40.530]as you're going between the rows.
- [00:45:42.360]And we can put sensors at multiple heights too,
- [00:45:45.420]so that you are sensing that corn, you know,
- [00:45:47.550]multiple times throughout the growing season
- [00:45:49.500]to see if we can characterize that deficiency.
- [00:45:52.350]Yeah.
- [00:46:01.350]Thanks. Thanks Santosh.
- [00:46:02.970]One of the things that I saw an autonomous
- [00:46:06.090]dry spreader in the field a year, maybe a year or two ago.
- [00:46:10.170]Yeah, and here's the farmer sitting on a four-wheeler
- [00:46:12.540]at the edge of the field watching it.
- [00:46:14.520]The farmers aren't saving any time,
- [00:46:16.710]but they are saving labor maybe.
- [00:46:18.987]Yeah, yeah, yeah. But I'm wondering,
- [00:46:20.700]you know, when it's this transition from,
- [00:46:24.630]how do we save the time by being able
- [00:46:27.150]to let that autonomous thing happen autonomously.
- [00:46:29.670]I'm not sure we're ready for that. Yeah.
- [00:46:32.203]From techno technological standpoint,
- [00:46:33.930]nor are we from a liability standpoint. Yep.
- [00:46:35.831]I mean, we've had GPS stops on pivots for how long?
- [00:46:41.160]No, but we can't trust GPS stops on pivots.
- [00:46:43.387]Yeah, yeah. We gotta have manual stops.
- [00:46:46.440]Yeah. Because the GPS stop
- [00:46:47.580]doesn't always work. Yeah.
- [00:46:48.690]So talk just about that transition from-
- [00:46:51.397]Yeah. Watching it happen-
- [00:46:54.780]Yeah? To let it happen.
- [00:46:56.400]Yeah. While you're not there.
- [00:46:57.300]Yeah, no, that's a great point because,
- [00:47:02.130]you know, so this is where it's like
- [00:47:08.790]if you have a, let's say you have animal operation also,
- [00:47:13.620]then innovate makes sense because okay,
- [00:47:16.530]I program my planter to do planting
- [00:47:19.380]and I'm going to take care of my cows, so that's it.
- [00:47:22.400]So, you're- Yeah.
- [00:47:25.200]Right. Exactly, right.
- [00:47:26.580]So I can turn it off, I'm monitoring, but I'm not,
- [00:47:28.830]not fully focusing on, is the planting happening correctly?
- [00:47:32.760]I'm just monitoring my phone,
- [00:47:33.870]but I'm tending to my animals too.
- [00:47:36.300]So here, all of a sudden we are talking
- [00:47:38.550]about providing opportunities to do integrated
- [00:47:41.940]cropping and animal systems.
- [00:47:43.620]You know, so that's a possibility.
- [00:47:46.237]I think even when I talk to some farmers,
- [00:47:50.820]you know, some farmers, they have like
- [00:47:52.380]only 200 or 300 acres and they have another job.
- [00:47:56.010]So they're like, "Hey, you know, I could use
- [00:47:58.050]this machine on a weekend, you know,
- [00:47:59.790]I can let it do its thing.
- [00:48:01.230]I can do my chores."
- [00:48:02.580]You know? So I think there are some models like that,
- [00:48:05.670]that we could look into.
- [00:48:06.777]And coming to your spreader and then the liability.
- [00:48:11.430]So that's one of the driving factor
- [00:48:13.800]to think about Swarm robots, smaller robots,
- [00:48:17.654]is that is like a 600 horse tractor can do a lot more damage
- [00:48:21.780]than a 60 horse tractor if things go wrong.
- [00:48:24.780]You know? So I totally agree with you.
- [00:48:27.420]I think going back to smaller equipment,
- [00:48:30.750]but fully autonomous helps with a lot of things,
- [00:48:35.130]you know, compaction, liability.
- [00:48:38.040]Then also it's easy to transport
- [00:48:39.750]from one field to another field, right?
- [00:48:41.790]Now, today, if you want to try, you know,
- [00:48:44.760]transport a big piece of equipment,
- [00:48:46.320]man, you need multiple trips or multiple equipment, so yeah.
- [00:48:55.890]One more, sure? Yeah.
- [00:48:57.660]This is just a comment. Yeah.
- [00:48:59.444]When I was a student back in 1965, we toured John Deere.
- [00:49:03.480]Yeah. And they had a lawnmower.
- [00:49:06.990]Yeah? Unmanned lawnmower
- [00:49:09.450]roaming around out in the front yard.
- [00:49:12.540]It was not guided, the only thing it did was stayed away
- [00:49:15.690]from trees and- Yeah.
- [00:49:17.269]And all this kind of stuff.
- [00:49:18.102]So this is not totally new. Yeah. Nope, nope.
- [00:49:24.630]Just a question.
- [00:49:26.070]In terms of taking these robots and being able to manipulate
- [00:49:32.490]topography, we've got a lot of hills and side hills and...
- [00:49:38.100]Yep? Ravines and whatever.
- [00:49:40.140]Yep.
- [00:49:40.973]In a lot of fields in Nebraska.
- [00:49:42.780]Yep.
- [00:49:43.613]Have you gotten to that point yet?
- [00:49:45.900]Not, not yet, but totally like the stability
- [00:49:49.320]and the center of gravity in slopes,
- [00:49:52.318]all of that is so important.
- [00:49:54.540]So that's where I think there are advantages
- [00:49:59.280]of drones or ground machines.
- [00:50:02.700]That's some of the things we're looking into is,
- [00:50:05.700]you know, we are looking into a hybrid robot
- [00:50:08.790]where it'll go on the ground for some time,
- [00:50:11.730]but if it has to fly, it can fly, you know?
- [00:50:15.582](chuckles) So, but totally, you know, it's an important
- [00:50:19.946]center of gravity of the equipment,
- [00:50:21.720]especially on the hills on,
- [00:50:23.370]yeah it is going to be very important, yeah.
- [00:50:25.070]We have not gotten to that point,
- [00:50:26.520]but totally agree with that.
- [00:50:29.370]And I think compared to bigger machines,
- [00:50:32.820]the smaller brands still have a chance.
- [00:50:35.730]So. (chuckles)
- [00:50:41.580]Mimi, is there anything online?
- [00:50:44.288]Yes, there is. Okay.
- [00:50:50.160]Milosh says a nice presentation,
- [00:50:54.720]except proper weed recognition would lead
- [00:50:59.490]towards site specific...
- [00:51:01.410]That would lead towards site specific weed management,
- [00:51:04.320]it seemed like there's a wide range of weeds
- [00:51:06.180]present in the photo you shared.
- [00:51:08.087]What are the system capabilities of employing multiple
- [00:51:10.940]herbicides and further on how decisions are made
- [00:51:16.560]that proper rates are delivered?
- [00:51:18.660]So multiple species. Yeah.
- [00:51:20.520]Multiple herbicides. Yeah.
- [00:51:21.870]Proper rates. Yeah.
- [00:51:22.950]My answer is I'll do whatever Amit ask me to do, (chuckles)
- [00:51:27.270]but no, I think, you know,
- [00:51:31.560]I just started working with weeds
- [00:51:33.420]just because there is so many projects funded on weeds
- [00:51:38.008]over the years, last 15 years I would say.
- [00:51:42.450]And today John Deere has a See & Spray technology
- [00:51:46.425]that's, I'm sure that's one of the reasons
- [00:51:49.800]why they have cameras on their tractor
- [00:51:53.190]so that they can keep collecting data
- [00:51:54.930]and improve the models.
- [00:51:56.579]So especially for machine learning
- [00:52:03.150]and identifying different species, you know,
- [00:52:05.040]weeds that are popular in Nebraska
- [00:52:06.930]from Amit and Mandeep I learned,
- [00:52:08.640]Palmer amaranth and Waterhemp are the main things, right?
- [00:52:11.760]So that's kind of what we are focusing on right now.
- [00:52:14.730]But then we definitely need a multi-state project
- [00:52:20.130]where we need to talk about what are the common weed types
- [00:52:24.300]in different states and come up with a database, you know,
- [00:52:28.080]so all this robotic, weeding and AI
- [00:52:30.990]is only good if the data is good.
- [00:52:35.100]So we need to be collecting a lot more data sets
- [00:52:38.070]of common types of weeds in different states
- [00:52:41.070]and have a database that researchers like me can access.
- [00:52:45.150]You know?
- [00:52:46.238]But I found out there is no database,
- [00:52:47.940]open database for weeds.
- [00:52:49.680]I mean, there's a lot of open database
- [00:52:51.090]for cats and dogs and you know, you name it, you know,
- [00:52:54.240]but there is no good database where you can say, okay,
- [00:52:58.350]I want the weeds, let's say Palmer amaranth.
- [00:53:01.530]I wanna download all the image data
- [00:53:03.330]so that I can train my robot.
- [00:53:04.890]That's the reason why we are growing Palmer amaranth
- [00:53:07.860]in the greenhouse so that we have good images, you know?
- [00:53:11.400]Yeah.
- [00:53:12.240]I don't know, Myla if I answered your question or not,
- [00:53:14.730]but I think the biggest bottleneck today
- [00:53:17.940]is a open database that has different species of weeds.
- [00:53:21.750]If we have that, I think we can easily train
- [00:53:24.870]a lot of different models and we can
- [00:53:27.168]feed that into the robot.
- [00:53:29.098]If you want to do spraying, if you, you can have
- [00:53:31.470]three or four herbicides, you know, for different,
- [00:53:34.320]and the robot identifies which weed it is,
- [00:53:36.780]and you spray only that, you know, today we can have
- [00:53:40.590]multi nozzle heads on the same sprayer.
- [00:53:43.410]So we can do all those engineering pretty easily.
- [00:53:47.040]I think the bottleneck is, we don't have
- [00:53:49.470]the database of which to train on.
- [00:53:52.080]Yeah.
- [00:53:52.913](attendee speaking softly)
- [00:53:54.480]What's that? (attendee speaking softly)
- [00:53:57.720]Yes, yes.
- [00:54:01.350]Santosh. That was a great presentation.
- [00:54:03.390]I really enjoy learning about the new technology
- [00:54:06.120]that you guys are like putting together
- [00:54:07.950]and developing for like helping farmers.
- [00:54:10.680]Yep. From, you know,
- [00:54:13.830]from experience from GPS adoption, we have seen
- [00:54:17.670]that it has taken like 30 or 50 years on like, you know-
- [00:54:22.240]Yeah. The public on adopting
- [00:54:24.270]that technology, right? Yeah.
- [00:54:25.790]So 50 years on adopting GPS. Yep.
- [00:54:30.030]Knowing that, right, that take some time
- [00:54:32.910]for like, you know, there's a lacking in technology adoption
- [00:54:36.000]in like production systems, like, you know-
- [00:54:38.340]Hm mm? farm production.
- [00:54:40.170]What do you see the challenges here of
- [00:54:42.750]like now switching for like, you know, smaller equipment?
- [00:54:47.133]Yep. And one, is that right,
- [00:54:49.440]the challenges.
- [00:54:50.273]The other one is saying that, you know, we,
- [00:54:54.120]next year or the next five years,
- [00:54:55.860]we certainly do the switch. Yeah.
- [00:54:58.740]Right? Yeah.
- [00:54:59.760]What are like, is there a program in place
- [00:55:03.960]to like close that gap of the knowledge
- [00:55:08.130]on managing lots of data? Yeah...
- [00:55:10.410]Because probably we could like even adopt
- [00:55:12.330]new sensors in like, all current equipment,
- [00:55:15.000]but now we are like swimming on these big data sets
- [00:55:18.169]and you mentioned something about like, you know,
- [00:55:22.320]overcoming this challenge, but, you know,
- [00:55:24.360]I would like to hear more about that.
- [00:55:25.590]Yeah, yeah. Thanks.
- [00:55:26.890]Yeah, I think that's a great question.
- [00:55:28.890]You know, the adoption, I'll just show you one slide.
- [00:55:31.470]I think I know what you're talking about.
- [00:55:34.890]So let's see here.
- [00:55:36.090]If I can...
- [00:55:46.830]So, you know, one of the first things you asked
- [00:55:50.250]is like, you know, adopting this robotics technology, right?
- [00:55:53.910]So that's where I have this slide.
- [00:55:56.040]I didn't go on in detail about it, but, so let's say
- [00:56:01.260]there's a farmer who bought this Case tractor, right,
- [00:56:04.110]just now, and he is pulling
- [00:56:06.900]a 16 row planter with that.
- [00:56:09.180]And I tell him like, "Hey, here is a robot,
- [00:56:11.310]it's awesome, you know, use it."
- [00:56:13.050]He'll be like, "I already invested a ton of money in this,
- [00:56:15.780]why would I do that?"
- [00:56:17.220]So every farm is different.
- [00:56:19.890]So I think what I can tell him is that, you know,
- [00:56:25.710]how would you like to transition to use robotics?
- [00:56:28.500]You know, what is the transition point?
- [00:56:30.180]You know, so, so instead of saying that,
- [00:56:33.300]okay, use a fully robotic planter, I could say,
- [00:56:36.127]"Hey, you know, what are you doing for core crop?"
- [00:56:39.067]"Are you drilling?"
- [00:56:39.900]Or, you know, so we need to survey and identify,
- [00:56:43.620]you know, the needs, you know, because again,
- [00:56:45.510]every farming operation is different.
- [00:56:47.490]But let's say there is a farmer
- [00:56:49.050]who's still using 30 years old equipment,
- [00:56:52.230]you know, for him, instead of asking him,
- [00:56:55.357]"Hey, get a new planter and maybe you could
- [00:56:57.510]think about robotics, you know,
- [00:56:59.460]because you have not invested in precision AG technology,
- [00:57:02.880]the last, you know, 15, 20 years,
- [00:57:05.340]then maybe it's time to skip one generation of technology
- [00:57:08.640]and directly go to this robotic equipment."
- [00:57:10.800]You know, so that's again, you know,
- [00:57:13.620]there's a lot of surveys that need to be done to understand,
- [00:57:16.740]you know, why would they,
- [00:57:17.880]what's the goal for them to use robotics?
- [00:57:20.010]You know?
- [00:57:21.810]So I dunno if I answered you the whole question,
- [00:57:24.030]but I'm just talking about the robotics piece there.
- [00:57:26.250]The data piece is a whole another story, right?
- [00:57:30.240]Because company, red company have their own data streams.
- [00:57:36.090]Green company will have their own data streams,
- [00:57:38.661]and if a farmer wants to use both equipment,
- [00:57:41.490]then there's no interoperability between the data sets.
- [00:57:44.550]So in an ideal world, you want interoperability
- [00:57:49.410]among all companies, but then that's not how
- [00:57:52.073]the AG business works, I guess, you know?
- [00:57:54.480]So open source is the way to go, in my view.
- [00:57:57.510]Open source technologies, you know, but then open source,
- [00:58:01.560]the problem is who is gonna maintain,
- [00:58:03.390]who's going to be the leader?
- [00:58:05.280]You know, who is going to maintain the open source?
- [00:58:07.710]So, yeah. Yeah.
- [00:58:12.660]Dr. Pitla, it was really interesting.
- [00:58:15.540]My question goes to teaching and training-
- [00:58:19.470]Yeah. Of the next generation
- [00:58:21.480]of farmers or whomever to use this.
- [00:58:25.920]Does it affect what we need to teach students
- [00:58:29.400]in this department?
- [00:58:30.540]Does it affect, you know, who the farmers themselves are
- [00:58:35.430]and how we train them?
- [00:58:37.440]How do you see that whole educational aspect?
- [00:58:40.950]Yeah, you know, even in,
- [00:58:44.070]even within the AG engineering department,
- [00:58:46.950]there is only so many credits our students can take,
- [00:58:49.770]you know, so I think the best way used to do this
- [00:58:53.460]co-curricular activities like this, robotic competitions.
- [00:58:56.610]And then
- [00:59:01.110]so this farm bar challenge probably is one of the first ones
- [00:59:04.380]I think is going to be very effective in training
- [00:59:07.800]different students from a variety of departments,
- [00:59:11.760]not just AG engineering, you know?
- [00:59:13.934]So here you can see in this picture,
- [00:59:15.480]there are students from agronomy,
- [00:59:18.090]there are plant pathology students, they're AG engineers,
- [00:59:21.360]all are working on the same problem of weed management.
- [00:59:26.910]So I think we need to create more opportunities like this
- [00:59:29.580]to bring like interdisciplinary education, you know?
- [00:59:34.020]So that would be one way to do,
- [00:59:35.280]because it's so hard to do everything
- [00:59:37.530]within your own department.
- [00:59:38.730]You know, even in AG engineering we have the same problem.
- [00:59:41.310]We cannot teach all the students to code, right?
- [00:59:45.000]We need to give the bigger picture of, okay,
- [00:59:48.150]here's how your code interacts with a piece of equipment,
- [00:59:51.420]and here are the troubleshooting points.
- [00:59:53.760]But we cannot expect AG engineers or agronomy students
- [00:59:56.580]to be the most efficient coders.
- [00:59:58.860]We need to bring in some computer science students too.
- [01:00:01.644]Yeah.
- [01:00:02.477]So I think core curricular activities is the way to go.
- [01:00:05.520]And also focusing on the bigger problem and how to solve it
- [01:00:09.660]through interdisciplinary skills is kind of
- [01:00:12.450]what we need to, you know, discuss and teach the students.
- [01:00:16.410]Yeah.
- [01:00:18.930]All right. Well, we are at 4:30 or a little past,
- [01:00:21.720]so please join me in thanking Dr. Pitla.
- [01:00:24.360]Again, great, great presentation.
- [01:00:30.030]Hope to have you back again soon with-
- [01:00:31.820]Yeah, thank you. Rapid...
- [01:00:33.379](audience applauding)
The screen size you are trying to search captions on is too small!
You can always jump over to MediaHub and check it out there.
Log in to post comments
Embed
Copy the following code into your page
HTML
<div style="padding-top: 56.25%; overflow: hidden; position:relative; -webkit-box-flex: 1; flex-grow: 1;"> <iframe style="bottom: 0; left: 0; position: absolute; right: 0; top: 0; border: 0; height: 100%; width: 100%;" src="https://mediahub.unl.edu/media/21028?format=iframe&autoplay=0" title="Video Player: Robotic Tools for Climate-Smart Agriculture" allowfullscreen ></iframe> </div>
Comments
0 Comments