Robotic Tools for Climate-Smart Agriculture

SANTOSH PITLA Associate Professor of Advanced Machinery Systems, Biological Systems Engineering, University of Nebraska–Lincoln Author

06/07/2023 Added

19 Plays

Description

Transitioning to climate-smart agriculture requires a paradigm shift in how we manage crop inputs and natural resources. Robotics, automation and AI-enabled autonomous systems offer promise for improving productivity of field operations. This talk focuses on the development of multi-robot ground and aerial robotic systems and their potential use cases for deploying climate-smart agricultural practices.

Searchable Transcript

Search:

[00:00:00.810]The following presentation is part
[00:00:02.700]of the Agronomy and Horticulture seminar series
[00:00:05.850]at the University of Nebraska Lincoln.
[00:00:08.340]Welcome, everyone, thank you for attending
[00:00:10.980]both online and in person, the Department of Agronomy
[00:00:14.130]and Horticulture Seminar series.
[00:00:17.010]This is our second to last week,
[00:00:18.510]and so continue to be excited about our talks,
[00:00:22.500]and today is no exception.
[00:00:24.990]Along with Dr. Guillermo Balboa and the entire
[00:00:30.042]seminar committee it's my pleasure to introduce
[00:00:35.354]this afternoon's speaker, Dr. Santosh Pitla
[00:00:36.570]from just across East Campus here,
[00:00:39.960]not even all the way across campus,
[00:00:41.460]Biological Systems Engineering.
[00:00:43.500]We've been fortunate to have several great talks
[00:00:45.720]from BSC folks this past several semesters.
[00:00:51.870]Today, Dr. Pitla is going to talk to us
[00:00:55.830]about perhaps the MAARS lab and some of their work
[00:01:00.720]in robotics, in particular in the area
[00:01:04.410]of climate smart agriculture,
[00:01:06.210]which we just continue to hear more and more about.
[00:01:09.750]So I'll note, as we complete the seminar, folks online,
[00:01:15.450]please feel free to pose your questions in the Q&A.
[00:01:19.320]We'll then read them and answer them live here.
[00:01:23.010]And per usual, for those of you in the room,
[00:01:25.200]I will pass this mic around to you
[00:01:26.723]so that the folks online can hear.
[00:01:29.730]All right, and with that, I'll turn it over to you.
[00:01:31.770]Thank you.
[00:01:33.510]All right.
[00:01:37.590]Everybody hear me okay? Online?
[00:01:39.930]Yeah? All right.
[00:01:40.980]Thanks for the kind introduction, Daniel. Appreciate it.
[00:01:43.380]So, and thank you all for coming.
[00:01:46.980]And today, like Daniel said, I'll be talking about
[00:01:51.090]robotic tools for conservation agriculture practices.
[00:01:55.410]So I am in the BSC department.
[00:01:58.230]I have both a teaching and research appointment.
[00:02:01.470]So with that, I want to kind of show
[00:02:05.790]what I'm gonna talk about here.
[00:02:07.650]The slide works, the slide advances.
[00:02:12.090]Oh, I have to click first here. There you go.
[00:02:16.650]Alright, so this is the overview of my presentation.
[00:02:19.350]So, Roll of Robotics in Conservation Agriculture Practices,
[00:02:23.250]and then the concept of Swarm robotics,
[00:02:27.690]and then some of the Nebraska robotic test-beds
[00:02:30.810]we are working on right now.
[00:02:32.160]So you can see the list there.
[00:02:33.930]And then, you know, next generation
[00:02:36.360]workforce for smart farms.
[00:02:38.130]So I wanted to touch a little bit upon,
[00:02:40.530]today in agriculture, we are talking about AI a lot,
[00:02:43.920]robotics, then agriculture, the fundamental
[00:02:46.950]agriculture sciences.
[00:02:47.970]How are we going to train the next generation workforce?
[00:02:50.820]We're going to work at this intersection, right?
[00:02:52.740]So I'm going talk a little bit about that.
[00:02:56.610]So, FAO says, conservation agriculture,
[00:02:59.310]there's a very standard definition.
[00:03:00.750]So how do you do minimum soil disturbance?
[00:03:04.050]How do you maintain a permanent soil cover?
[00:03:06.840]And then how do you diversify plant species?
[00:03:11.220]This is a very difficult thing to do at scale, right?
[00:03:13.890]So how do we do this at scale?
[00:03:16.560]And one of the things is, if we can do this with robotics,
[00:03:22.080]we can boost, you know, water and nutrient efficiency.
[00:03:25.698]So the way I see it is the robotics and automation,
[00:03:28.950]I think they're not just, you know, ways to eliminate,
[00:03:33.720]drudgery or hard work in the farms,
[00:03:35.850]but also they can contribute to climate smart practices.
[00:03:38.430]So today I'm going to talk about some of the things
[00:03:43.770]that we are doing in my MAARS lab.
[00:03:45.630]Again, MAARS stands for Machine Automation
[00:03:47.610]and Agricultural Robotics Lab.
[00:03:49.470]So if you look at precision agriculture today,
[00:03:51.720]we still talk about dollar per acre.
[00:03:55.410]With the help of robotics, can we go,
[00:03:59.220]per square foot management or per plant management
[00:04:01.890]instead of, you know, per acre?
[00:04:05.100]So we talk about pounds per acre, gallons per acre.
[00:04:08.220]When we are talking about application rate,
[00:04:10.470]can we talk about, micro doses per plant
[00:04:14.190]or per square foot using robotics?
[00:04:16.854]You know, this could be possible with robotics.
[00:04:19.031]So with that, you know, weeds is one of the big things.
[00:04:23.433]Can we, especially herbicide resistant weeds,
[00:04:26.850]can we use robotics to take care of weeds?
[00:04:29.580]Again, chemical savings, intercropping systems,
[00:04:33.000]again, biodiversity is important,
[00:04:35.490]but how do you do multiple crops in the same field?
[00:04:38.307]You have thousands of acres, right?
[00:04:40.830]So those are some of the things that could be possible
[00:04:43.980]if you have a good robotic systems.
[00:04:46.166]And then, improving fertilizer application efficiency,
[00:04:52.448]so robotics could be a good tool
[00:04:54.480]to do these kind of climate smart practices.
[00:04:59.490]Tell you the story of Charlie, the corn plant.
[00:05:02.160]So today, with the amount of data we have
[00:05:06.210]from drones and ground machines, we know everything
[00:05:08.730]about a corn plant, at least, you know,
[00:05:10.770]what are the needs of the corn plant?
[00:05:12.570]So more water, more fertilizer.
[00:05:14.820]You know, there are weeds, and we are talking about IOT.
[00:05:20.280]Everybody heard about IOT, Internet of Things,
[00:05:22.830]that's up and coming, right?
[00:05:24.300]So in the future, you might be getting a call,
[00:05:26.250]text or a feed from a corn plant,
[00:05:28.710]so that, you know, we,
[00:05:30.780]there is so much data and big data today
[00:05:33.090]that we know a lot about corn plant and its needs.
[00:05:36.300]So assuming that we have all this data,
[00:05:39.407]how do we take care of this corn plant, right?
[00:05:41.700]But you know, Charlie's not alone, right?
[00:05:43.740]So there are 30,000 plants an acre on average, you know,
[00:05:47.280]6 million plants in a 200 acre field.
[00:05:49.994]So there's a reason why we are in the realm
[00:05:52.830]of digital agriculture, right?
[00:05:54.900]So there's a lot zeroes and ones,
[00:05:56.490]there's a lot of matrixes of data,
[00:05:58.800]all need to be solved
[00:05:59.910]so that we can take care of the corn plant right?
[00:06:02.820]So, well, let's send a robot like this,
[00:06:07.710]even though this robot looks pretty cartoonish,
[00:06:10.170]there's a lot of intelligence here.
[00:06:14.400]It has a camera, it can see the obstacles,
[00:06:17.130]it is connected, it has a decision support,
[00:06:20.839]it has different types of arms, right?
[00:06:23.400]So one for pulling the weeds
[00:06:24.810]and one for spraying, spot spraying.
[00:06:27.750]Even today, we don't have a robot like this.
[00:06:29.940]You know, it's pretty hard to design a robot like this,
[00:06:33.420]especially when we know this is
[00:06:36.690]the current state of the art.
[00:06:37.920]We use bigger equipment, right?
[00:06:40.260]So to actually take care of the farm,
[00:06:43.620]because you want to cover a lot of ground
[00:06:45.870]in the shortest time window.
[00:06:47.550]So that's the reason why we are here,
[00:06:49.860]where we are using tractors up to 600 horse today, right?
[00:06:53.520]So how many of you know Nebraska tractor test lab?
[00:06:58.410]Alright, for some of you don't know
[00:06:59.700]Nebraska tractor test lab is here on East Campus,
[00:07:03.000]so that's the only one in the western hemisphere.
[00:07:06.420]So hundred years of tractor testing happened
[00:07:09.330]here on East Campus, so if you don't know, please, visit us?
[00:07:13.440]And actually with this July, we are celebrating
[00:07:15.420]a hundred years of tractor testing.
[00:07:19.548]So this is one of the tractors we recently tested
[00:07:21.750]on the track.
[00:07:22.583]Again the point I'm trying to make here is,
[00:07:26.190]it's 600 horse, 60,000 pounds of weight.
[00:07:32.160]So we know compaction is a problem, right?
[00:07:36.390]So, and then another important thing here
[00:07:40.950]is this tractor is made for 40 years,
[00:07:43.470]four zero on an average, let's say you're using this tractor
[00:07:46.650]probably 500 hours a year.
[00:07:49.440]This will be on the farm for 40 years.
[00:07:51.757]What can you say about software
[00:07:53.940]and technology on that machine?
[00:07:55.860]Will it be valid for 40 years?
[00:07:58.140]It's not going to be, right?
[00:07:59.820]So this will become technically obsolete pretty fast, right?
[00:08:03.060]So in my, in our view, like seven to 10 years
[00:08:06.360]is a good number, at which point replacing
[00:08:10.440]the entire technology with the new technology
[00:08:12.210]might be better.
[00:08:13.043]So same thing with wider planters, right?
[00:08:15.750]So if the fields are perfectly square, great,
[00:08:20.700]but if you have a field like this,
[00:08:23.580]which is highly irregular, wider implements
[00:08:26.100]are not always good, right?
[00:08:27.150]So you are either under spraying or overspraying,
[00:08:29.970]especially if you're using a sprayer.
[00:08:32.010]And if you have a really wide planter,
[00:08:35.310]guess what you're doing most of the time:
[00:08:37.740]either you're backing up or turning
[00:08:39.510]instead of actually planting.
[00:08:40.680]So I think it's very important that,
[00:08:45.150]what are the field attributes, not just about, you know,
[00:08:48.840]how big is the field, but it's also important,
[00:08:50.730]what are your field attributes?
[00:08:52.140]And you need to match your implement set
[00:08:54.450]or automation or equipment set to your field.
[00:08:57.870]So you know a little bit about trends,
[00:09:00.900]you know where we are going.
[00:09:02.820]So what you see here are two really good examples
[00:09:06.300]of full autonomy.
[00:09:08.250]So the first one is on the left side
[00:09:10.530]of the screen is auto cart.
[00:09:13.350]Both of them are actually bought by Case New Holland now.
[00:09:15.990]So both these technologies are owned by Case New Holland.
[00:09:19.200]On the right side, you can see
[00:09:21.300]that's called a DOT platform.
[00:09:22.740]Actually it was a Canadian company and then bought by Raven.
[00:09:26.310]And Raven got bought by Case New Holland.
[00:09:28.710]So the idea is the major OEMs have autonomous programs,
[00:09:35.610]you know, so everyone is ready
[00:09:37.230]to launch autonomous tractors.
[00:09:40.655]So you can see the one on the right side
[00:09:43.560]doesn't look like a tractor anymore.
[00:09:46.560]So in my view, in the future, you know,
[00:09:49.110]the last a hundred years, we know how
[00:09:50.940]the tractor looks like, but the next 25 years,
[00:09:53.370]the tractors could be completely different.
[00:09:54.990]They might not look like tractors.
[00:09:56.970]And this was actually by both the director
[00:09:59.267]of the tractor test lab and also a John Deere engineer
[00:10:03.780]who worked there for 40 years.
[00:10:05.430]So I agree with that.
[00:10:07.440]In 20 to 25 years, tractor might not look like a tractor.
[00:10:12.728]So we use drones.
[00:10:15.180]So now we are looking at drones for spraying also,
[00:10:18.420]I think primarily we collect a lot of images
[00:10:22.290]about the crops for, you know, decision making
[00:10:27.180]or identifying problems.
[00:10:28.590]But we are also looking into spraying.
[00:10:30.324]But in the future, we could be doing
[00:10:32.760]more different kind of operations.
[00:10:35.910]Again, a little bit of history.
[00:10:37.440]So 1920, we tested the first tractor
[00:10:42.000]at the tractor test lab.
[00:10:46.170]So today, you know, fast forward a hundred years,
[00:10:50.010]the tractor is a highly sophisticated machine,
[00:10:53.280]lot of software, a lot of hardware.
[00:10:55.410]We are talking about full autonomy
[00:10:57.030]where we are talking about artificial intelligence, right?
[00:10:59.460]So how does the next generation equipment look like,
[00:11:02.790]let's say in 2040.
[00:11:04.160]So it could look like this, you know,
[00:11:06.210]so there could be small robotic vehicles
[00:11:09.840]that are going in between the crop rows.
[00:11:11.580]There could be a slightly bigger one.
[00:11:14.070]So these are some of the test beds
[00:11:15.390]that I have in the MAARS lab.
[00:11:17.400]And the one on the right side,
[00:11:18.570]we call it as a flex straw robot.
[00:11:20.733]It's a 60 horsepower machine.
[00:11:22.670]So I think that to us, that's a good size machine
[00:11:26.160]that is very valuable for Midwestern row crop.
[00:11:30.600]So the 60 horsepower machine, you know,
[00:11:33.480]instead of having just one, you could have multiple, right?
[00:11:37.260]So there are some companies that are
[00:11:38.970]actually providing that farming as a service.
[00:11:41.550]So you see on this left side here, that's a Kubota
[00:11:44.727]and the right side is out of Australia.
[00:11:47.640]So the paradigm shift is happening
[00:11:51.000]where instead of using one big machine,
[00:11:53.277]can we use multiple smaller machines?
[00:11:56.250]If I said this, let's say 20 years ago, you'll be like,
[00:11:59.470]if you have 10 machines, you need 10 operators, right?
[00:12:02.603]So it doesn't make sense, but today not the case.
[00:12:05.580]So one person can manage 10 or 20 machines.
[00:12:09.000]If your control architecture, the Swarm architecture,
[00:12:12.120]is good, you know, so again, this is
[00:12:15.270]a paradigm shift that's happening today.
[00:12:19.590]So again, with that idea, can we replace
[00:12:23.130]one big machine with multiple smaller machines?
[00:12:26.520]And again, these are another ones that I've seen at the top,
[00:12:31.320]you can see there's a DOT platform.
[00:12:33.360]And then Sabanto the orange machine.
[00:12:36.030]And the really small one is an Earth Sense.
[00:12:38.370]I really don't know about that if it's going to work
[00:12:40.590]in a real field condition, but at least it could be
[00:12:43.710]a good field scouting tool, you know, so the smaller ones,
[00:12:48.617]so the idea is can we replace one 600 horse tractor
[00:12:53.700]with multiple tractors, in this case,
[00:12:57.600]let's say 10, 60 horsepower machines so that, you know,
[00:13:01.290]bigger the field, you just have more number of machines.
[00:13:06.510]So kind of a field scale neutral solution.
[00:13:09.180]If you have a smaller field use, let's say two,
[00:13:11.700]if you have a larger field, use 20 of this.
[00:13:14.040]So the technology is going to be the same,
[00:13:16.230]but you scale the number of machines up and down.
[00:13:19.170]Okay?
[00:13:20.086]So with that, we are looking at the coordination strategies.
[00:13:25.290]Your planting is going to be different
[00:13:27.030]from baling and picking operation.
[00:13:28.590]For example, on the planting side,
[00:13:30.870]they all are doing the same operation, right?
[00:13:33.030]These three robots on the left, they're all planting.
[00:13:35.767]So we call that as homogeneous operation
[00:13:38.640]because there's just planting, whereas baling and picking
[00:13:42.390]is heterogeneous because there's a baler that bales,
[00:13:45.930]there's a picker.
[00:13:46.763]So they're both, both those robots
[00:13:48.480]are doing different operations.
[00:13:50.310]And the harvesting is another big example
[00:13:52.950]where you need communication all the time.
[00:13:55.320]And probably the most complex, because once
[00:13:58.290]a grain cart one is full, it needs to go.
[00:14:01.920]And then grain cart two need to sync up with the combine
[00:14:04.800]so that it can transfer the grain.
[00:14:06.600]So we have to be thinking about all these
[00:14:09.330]and program all of this into the control architecture.
[00:14:13.260]So when we are talking about multiple robots,
[00:14:16.158]and homogeneity and heterogeneity is also important.
[00:14:21.540]So with that, I'll just go over
[00:14:25.920]some robotic test beds we have.
[00:14:28.980]So this is a smaller robot that we launch
[00:14:33.930]in between the crop rows, you know,
[00:14:36.120]primarily you can get corn population, you know,
[00:14:38.940]so what's the population count?
[00:14:40.620]And also you can put different types of sensors
[00:14:43.170]so you can sense the microclimate underneath the canopy.
[00:14:47.310]So drones are great, you can fly over and get a lot of data,
[00:14:50.100]but what's happening under the canopy, right?
[00:14:52.407]So you can use this smaller machines or robots.
[00:14:57.750]And again, under the canopy, again, as AG engineering,
[00:15:00.960]our challenge is under the crop canopy, there is no GPS.
[00:15:04.003]So how does a robot navigate?
[00:15:05.958]So we're using laser sensors to navigate between the crop.
[00:15:09.900]So...
[00:15:12.517]And then, so this is our
[00:15:14.820]close to 60 horsepower robotic machine.
[00:15:18.570]So it can run 14 hours because
[00:15:21.780]as long as there's gas in the tank, it can work.
[00:15:24.270]So I know there's a lot of talk about electrification,
[00:15:26.430]but I'll show you some slides
[00:15:27.690]why electrification might not be a great idea,
[00:15:31.620]especially for Midwestern agriculture.
[00:15:34.939]So this machine here, so this is a four-wheel steer,
[00:15:39.300]four wheel drive.
[00:15:40.133]So you see four wheel drive on a truck, right?
[00:15:42.240]So trucks are four wheel drive.
[00:15:43.440]But with this we can also steer all four wheel.
[00:15:47.070]So what I'm am I talking about?
[00:15:48.157]So if you can see, you can turn all the wheels 90 degrees
[00:15:52.680]and you can go to the next rows if you want.
[00:15:56.640]So the idea is one of the big inefficiency
[00:16:00.870]in the fields is turning time.
[00:16:03.270]So whenever you're planting, you know, or spraying,
[00:16:05.673]you have to turn around to get to the next row.
[00:16:09.090]So that's the biggest inefficiency in a field.
[00:16:12.180]So how do we improve the efficiency?
[00:16:14.190]By innovating steering mechanism.
[00:16:16.560]So that's what we are trying to show here.
[00:16:19.920]And then since it's autonomous vehicle, right?
[00:16:22.950]So we need to test for obstacle detection avoidance.
[00:16:27.000]So that's something we work on in the lab.
[00:16:29.910]And then, you know, one of the first application we did
[00:16:34.170]is the phenotyping, right?
[00:16:35.820]So how many of you know spidercam facility, right?
[00:16:39.570]So yeah, so we have a one acre spider cam facility,
[00:16:42.810]but the big thing with this is it's only one acre, right?
[00:16:47.160]So the camera can go in only one acre.
[00:16:49.680]So we have the same sensors on this robot.
[00:16:52.230]So if you want to phenotype your field on each campus,
[00:16:57.090]we can use this machine to phenotype.
[00:16:59.520]So we are using the RTB camera, the same type of sensors
[00:17:05.010]that we all we have on the spidercam.
[00:17:09.720]So you can see here when we took this robot
[00:17:14.280]to the spidercam, so on the right side
[00:17:19.320]is when we were actually collecting the data,
[00:17:21.540]and you can see the spidercam in the background.
[00:17:23.850]So we have very good correlation between the data
[00:17:26.070]we collected using a robot and the spider cam.
[00:17:28.980]So we were actually phenotyping the soybean field,
[00:17:32.087]so there were different varieties.
[00:17:33.990]You can see a little bit of data here,
[00:17:35.970]days after planting and the average plot height.
[00:17:39.570]So you can kind of see the differentiate between
[00:17:43.320]the varieties of soybean here using
[00:17:46.320]the ultrasonic height sensor that we had there.
[00:17:51.600]And then we can get the green pixel fraction.
[00:17:54.726]And again, the idea here is if you're doing
[00:17:58.380]a lot of field trials, why not use robots
[00:18:01.440]to accelerate your investigations, right?
[00:18:03.840]So, that's kind of the idea is we could automate
[00:18:08.310]some of these processes so that we can progress
[00:18:11.160]and advance investigations in the field. Yeah.
[00:18:16.320]And then we submitted a grant to the USFDA,
[00:18:20.460]we'll see if it works or not.
[00:18:23.580]But the idea is can we use this robot to do multiple things?
[00:18:27.180]Can we use this robot to do cover crop inter-seeding?
[00:18:30.180]Can we use it to do cover crop drill?
[00:18:32.940]Because we know that's one of the important
[00:18:34.786]climate smart agriculture practice
[00:18:37.126]and farmers run out of time after they harvest.
[00:18:40.530]You have a very short time window,
[00:18:42.390]you're strained for labor.
[00:18:43.834]And how do you do cover cropping at scale?
[00:18:48.300]So today, the common technique is you broadcast
[00:18:51.060]a cover crop seed, but if you're talking
[00:18:53.070]about no-till agriculture, there's a lot of biomass, right?
[00:18:56.670]If you're simply spreading, you know,
[00:18:58.140]the germination might not be ideal, you know, or perfect,
[00:19:01.530]so why not send a robot, right?
[00:19:04.200]That can actually drill into
[00:19:05.970]the no-till field after your harvest.
[00:19:09.990]So, and then we can add soil sensing to this robot, right?
[00:19:15.300]Phenotyping and air sampling.
[00:19:17.100]So that we can characterize, you know, soil health,
[00:19:20.430]plant health and disease medication.
[00:19:22.080]So we can have this system,
[00:19:26.010]we can activate this flex-ro robot with, you know,
[00:19:29.117]let's say a cover crop drill.
[00:19:30.720]And then if it follows a combine, you know, and drills,
[00:19:35.820]you know, because the time windows are so short,
[00:19:39.600]so you can use this robotic system.
[00:19:41.255]And again, I keep thinking about,
[00:19:45.621]if I have to offer a robotic system to the farmer today,
[00:19:49.320]what would be they interested in?
[00:19:52.020]Planting is a very high risk operation.
[00:19:53.910]You don't want to, right?
[00:19:55.080]So planting, the trust is not there yet, right?
[00:19:58.307]So, what else can we do?
[00:19:59.640]Maybe, you know, a cover crop drill is a perfect operation
[00:20:02.640]because they'll be okay to try out this technology.
[00:20:06.600]So that's something I'm looking into, you know,
[00:20:08.550]to work with farmers.
[00:20:09.840]And then soil sampling is a huge, huge thing
[00:20:12.570]that even today it's a very manual
[00:20:15.660]and laborious process, right?
[00:20:17.550]So can we use this robot, you know, and can we pick,
[00:20:21.360]you know, also in situ sends the leaves, you know,
[00:20:25.680]on a corn plant, like for example,
[00:20:27.750]this is in our greenhouse, so we use this robotic arm
[00:20:31.830]to know what is the chlorophyll content
[00:20:33.540]and leaf temperature.
[00:20:35.010]So we can fly drones, right?
[00:20:36.540]And get data, but we can ground through them
[00:20:39.270]with these actual robotic devices instead of a person
[00:20:41.760]going and ground trooping it.
[00:20:43.530]Maybe we can send robots to do that, you know,
[00:20:45.750]so, and then air sample collection,
[00:20:49.770]this is one of our ideas that went in the proposal.
[00:20:52.140]So we'll see, you know, so we can put this air sampling
[00:20:54.960]device on the equipment.
[00:20:56.670]We can collect images to look at the disease mitigation.
[00:21:00.690]This is an example from wheat spike counting.
[00:21:04.440]So using machine vision, we can count
[00:21:06.420]how many wheat spikes there are.
[00:21:08.460]So, same technique can be applied
[00:21:10.200]to look at disease pressure, yeah.
[00:21:17.250]So, with the...
[00:21:19.980]This year, next week is gonna be pretty exciting for us
[00:21:23.220]because we are trying to plant with this robot
[00:21:26.430]for the first time ever robotically.
[00:21:28.260]So we are going to do corn planting next week.
[00:21:31.410]I think we are ready to go if the weather permits.
[00:21:35.130]So I mean if you know Rogers Memorial Farm?
[00:21:40.050]Yeah, so we are going to plant there.
[00:21:42.180]So if it is successful, then it'll be the first five acre
[00:21:46.830]of robotically planted corn in our research fields.
[00:21:51.960]I don't want to claim Nebraska because I've not done
[00:21:54.420]my research yet, but at least in our research fields,
[00:21:57.720]this will be the first time we'll be planting some fully
[00:22:00.180]robotically or autonomously.
[00:22:02.010]So that's how many, how many rows?
[00:22:03.930]Two rows, Two rows.
[00:22:04.763]Yeah, just two rows.
[00:22:05.910]We can actually put four rows on this equipment,
[00:22:08.070]but we are starting with the two rows
[00:22:09.900]because we are trying to do an innovative mechanism here.
[00:22:13.260]I can talk about it later, but yeah,
[00:22:15.660]so we are going with only two rows here, yeah.
[00:22:19.800]So we also have something called Flex-Ro Mini,
[00:22:22.860]with the goal of automating the operations
[00:22:26.700]in a vegetable farm too.
[00:22:28.020]So I'm working with Sam Wortman
[00:22:31.530]and we are sketching a plan to see
[00:22:34.620]if we can plant lettuce robotically in August.
[00:22:39.030]So if it all works, we'll be able to do that.
[00:22:42.060]And also we can use this machine as a dump wagon.
[00:22:44.580]So you harvest produce, put it on this machine
[00:22:47.550]and this is going to go drop the produce, you know,
[00:22:50.490]at the edge of the field and come back, you know,
[00:22:52.740]so be like a robotic helper, yeah.
[00:22:56.516]All done by seed, right?
[00:22:58.230]Yeah, seed lettuce by seed, yeah.
[00:23:03.210]So now we're talking about these robots, correct?
[00:23:07.860]But these robots are smaller compared to
[00:23:09.930]your 12 row 16 row planters, right?
[00:23:13.650]The tanks are obviously smaller.
[00:23:16.320]So how are you going to refill the seed?
[00:23:19.170]We don't want these ground robots to go back and forth.
[00:23:23.460]Why not use drones to refuel the seed, right?
[00:23:27.060]So that's kind of the idea here.
[00:23:29.400]So can we use that drone to drop off the seed
[00:23:33.810]on this ground machine so that this ground robot
[00:23:37.080]is always moving, right?
[00:23:39.270]So that, you know, the drones will go, take off,
[00:23:43.080]land on the machine, drop off the seed and go back.
[00:23:46.050]So we are utilizing the full potential
[00:23:48.815]of both type of ground robots, both ground robots
[00:23:53.220]and then aerial robots.
[00:23:54.630]Aerial robots can cover long distances in a short time,
[00:23:57.720]but then they can be there only for 20, 25 minutes.
[00:24:00.660]Ground machines don't have the problem.
[00:24:02.790]So we're using the best of both worlds, you know,
[00:24:05.070]so the big limitation is drones can carry
[00:24:09.390]only up to 20 pounds or so right now,
[00:24:13.230]but then we want to prove out the concept,
[00:24:15.510]the coordination strategy.
[00:24:17.730]We want to do a scale model and prove it
[00:24:20.370]so that when drones get bigger, you know,
[00:24:22.350]so we can use the same technique.
[00:24:25.830]So here's an example, nothing special here
[00:24:29.070]we are flying a drone,
[00:24:33.480]hard to see too.
[00:24:34.950]So...
[00:24:39.510]Yeah, there's a little drone,
[00:24:40.830]you see it's landing there. (chuckles)
[00:24:45.090]The special thing there is,
[00:24:47.040]it's took off autonomously and landed autonomous.
[00:24:51.450]So the typical DJI drones that we use in agriculture, right?
[00:24:55.980]So the software is closed,
[00:24:57.930]so we cannot make any modifications to the software.
[00:25:00.750]So what we are trying to do is developing these drones
[00:25:03.570]from the scratch so that we can program it.
[00:25:05.940]So we have, we program the flight controller
[00:25:08.880]so that we can take off and land.
[00:25:10.830]If you have to increase the capacity,
[00:25:12.330]we select different propellers, you know,
[00:25:15.030]so we need that type of customization
[00:25:17.490]or else we cannot put our algorithms on those drones.
[00:25:20.760]So that's kind of, we are in early stages of doing that.
[00:25:24.150]And here is another example.
[00:25:27.447]So there are two drones.
[00:25:29.820]The one in the front, close to the door is the leader.
[00:25:33.720]If that leader moves to the right,
[00:25:36.300]the follower moves to the right too.
[00:25:38.250]So we are trying to do coordination of drones,
[00:25:41.430]not just coordination of ground robot and aerial robot,
[00:25:44.610]but aerial robots also.
[00:25:47.100]So all of this becomes very important because again,
[00:25:49.320]going back to that example of ground robot planting,
[00:25:54.330]maybe you need two or three drones to service one machine,
[00:25:57.150]you know, and vice versa.
[00:25:58.380]So we are talking about
[00:26:00.000]multi-robot systems and coordination.
[00:26:02.670]So that's the reason why we are
[00:26:05.040]looking into these algorithms so that we can program
[00:26:08.970]the drones like we want to.
[00:26:15.197]So, let's see.
[00:26:19.290]So a lot of talk about driverless cars.
[00:26:23.940]I don't know why we need driverless cars
[00:26:25.920]because I don't wanna buy a Ferrari
[00:26:27.420]and let it drive itself.
[00:26:28.920]I want to drive it,
[00:26:30.420]but (chuckles) there is real need on the farm though.
[00:26:33.840]So, because we cannot find skilled labor
[00:26:37.170]or there's no labor, you know, so we need, if we need that,
[00:26:44.040]it's important that the robot not just,
[00:26:48.420]like detecting the obstacle and stopping
[00:26:50.280]like you saw before, but it also need
[00:26:52.260]to positively identify what that obstacle is.
[00:26:55.020]Is it a bale? Is it a cow?
[00:26:57.360]Is it a center pivot?
[00:26:58.680]They're very hard to detect, you know,
[00:27:00.990]so we are training these objects that are typically,
[00:27:04.680]that typically happen on a farm.
[00:27:06.450]So that if my flexor robot is going
[00:27:09.480]and it sees another equipment, it says, oh, it's a friendly,
[00:27:12.330]I need to go close and work with it, not run away from it,
[00:27:15.540]you know, so we are training these models
[00:27:18.300]so that we know what are the actual objects
[00:27:21.240]that the robot is seeing, okay?
[00:27:23.670]So for this, we need a lot of data sets,
[00:27:26.160]and I was surprised we, even though we are the headquarters
[00:27:30.810]of the center pivot manufacturers,
[00:27:32.640]there's no public database where we can get pictures
[00:27:35.580]of center pivot, you know, so this summer
[00:27:39.210]we want to do some data collection
[00:27:41.160]of all these obstacles from our farms
[00:27:43.440]so that we can train our machine learning models, okay?
[00:27:47.880]So center pivots are very hard, so you need to identify,
[00:27:51.330]because sometimes the model thinks it's a tractor
[00:27:54.030]because it has wheels, you know,
[00:27:55.620]sometimes because of the structure, it's so different.
[00:28:01.200]And then, so this is a project called "Cloud -E"
[00:28:07.230]that we are looking at, assuming that we solve
[00:28:12.720]the problem of bad connectivity on the farms, right?
[00:28:17.820]We can, if all the robots can be connected
[00:28:21.000]through the Cloud, then we can actually do cloud computing.
[00:28:25.110]So today, if you want to deploy AI, or if you want to do
[00:28:28.410]a lot of computation on individual machine,
[00:28:31.320]the problem is you need expensive hardware on each robot.
[00:28:35.310]So we say that, okay, we need 10 robots,
[00:28:38.580]but you have expensive hardware on each robot
[00:28:42.000]to do this artificial intelligence,
[00:28:43.830]then it's not going to work out because
[00:28:46.260]that's not going to be, that's gonna be very expensive.
[00:28:49.110]So if we solve the problem of this connectivity,
[00:28:52.230]then we can do computation,
[00:28:54.000]a lot of computation on the Cloud.
[00:28:56.370]So that's where this is, this is an architecture
[00:29:00.390]of how we can take data from the drones,
[00:29:03.500]how we can take data from the tractors and the robots,
[00:29:07.530]send the data to the Cloud so that instead of doing
[00:29:10.980]heavy computation on individual machine, you know,
[00:29:13.590]the Cloud is doing that computing for you
[00:29:15.480]and sending the result, you know,
[00:29:17.909]so at ENREEC we have good connectivity.
[00:29:20.460]Now there is internet throughout the ENREEC field,
[00:29:23.550]Eastern Nebraska Research and Extension Center, right?
[00:29:25.890]So, we'll try some of this out, you know,
[00:29:29.850]especially with nitrogen management,
[00:29:32.220]I know many of your work in that area.
[00:29:34.860]So the nitrogen, can we send the data from the drones
[00:29:39.024]and high clearance sprayers of the crop status
[00:29:42.930]to the Cloud so that the Cloud is going
[00:29:45.360]to continuously compute, you know, okay,
[00:29:48.030]this is the nitrogen status and here's the prescription,
[00:29:51.060]you know, and the robot can go do that.
[00:29:53.915]So you're continuously monitoring the crop status
[00:29:58.200]and whenever there is a need for it to go do application,
[00:30:01.980]it can go to that.
[00:30:03.300]So...
[00:30:06.480]And then human machine interface and gesture control.
[00:30:10.260]So, today, okay, you have a remote,
[00:30:14.793]you can take the robot to the field edge, launch it,
[00:30:18.720]and you give it a path using the GPS,
[00:30:21.150]it'll follow the path and do the operations.
[00:30:23.070]But how can we seamlessly integrate humans, right,
[00:30:27.960]into the mix, you know, into the control loop?
[00:30:31.770]So what we are trying to do is we are converting
[00:30:34.320]standard hand signs into robot signals.
[00:30:37.980]So in this case, if he, this is a forward sign we program,
[00:30:42.000]because we are going to have cameras anyways on the robot,
[00:30:44.460]so might as well use that, right?
[00:30:46.200]So use a hand gesture to move the robot
[00:30:49.110]instead of having a separate remote.
[00:30:51.870]So you can see, so when he is doing
[00:30:55.860]the hand gestures, right?
[00:30:57.360]So it moves or stops based on that.
[00:31:00.480]So we have programed more than 10 hand signs now.
[00:31:03.660]So we'll test it in this summer, you know,
[00:31:05.640]in the field to see how it goes.
[00:31:10.020]So the next thing is the, you know, we talk about robotics,
[00:31:15.000]climate smart agriculture, anytime we say,
[00:31:17.550]and then we say diesel engine, people will be like,
[00:31:19.447]"Oh, it's not electric, why is it not electric?"
[00:31:22.770]You know, the reason is the power density is not there yet
[00:31:28.440]for batteries, especially if you're talking
[00:31:31.050]about Midwestern farming, right?
[00:31:32.970]Because planting season, you'll be there out in the field
[00:31:38.430]15 to 16 hours a day, hours a day, right?
[00:31:41.400]So, and then you have, you need a lot of power
[00:31:44.550]and there is no charging time, right?
[00:31:46.230]So it's not like you can take your tractor
[00:31:48.840]and charge overnight, so there's no overnight,
[00:31:51.060]you know, so we cannot wait for that.
[00:31:53.373]So what we did is we collected data from this equipment,
[00:31:59.217]something called canvas data, and also we instrumented
[00:32:03.000]the drawbar, PTO and hydraulics to figure out,
[00:32:06.270]you know, so how much power are these implements
[00:32:09.450]taking from the tractor?
[00:32:12.480]So you can see here on the y-axis is kilowatt-hour,
[00:32:17.310]that's the energy.
[00:32:18.870]So these are the two different tractors.
[00:32:20.820]And again, Case B is a much bigger tractor.
[00:32:24.780]You can see it's a-48 row planter.
[00:32:29.760]This is a 16-row
[00:32:31.290]So we are not trying to compare here, but you know,
[00:32:34.350]those are the equipments that we had access to.
[00:32:36.660]So we collected data so that we can come up
[00:32:39.420]with what is a kilowatt-hour, you know?
[00:32:42.270]So, and then we have the battery energy densities, you know,
[00:32:48.690]kilowatt-hour per cubic meter, you know,
[00:32:51.630]335 and then that's the weight or mass energy density, okay?
[00:32:58.410]So the bottom line is we know how much kilowatt-hour
[00:33:06.540]1,117 kilowatt-hour, right?
[00:33:10.260]And then we calculated the mass, alright,
[00:33:15.990]with our 15 hours of operation.
[00:33:18.540]That's how big a battery you need.
[00:33:22.680]The only thing that works in favor of tractors
[00:33:25.350]is we need to balance the tractor anyways, so we need more,
[00:33:28.950]more mass on the tractor or weight on the tractor, right?
[00:33:31.770]So that's the only advantage we have,
[00:33:33.480]but still that's a 12 ton and five ton.
[00:33:40.170]That's pretty big masses, you know?
[00:33:42.570]Those are replacement batteries?
[00:33:44.070]Yeah, replacement batteries, exactly.
[00:33:46.200]Swapping, right? (chuckles)
[00:33:47.033]You got to swap, you know?
[00:33:49.470]Yeah.
[00:33:50.505]Again, then comes the infrastructure, right?
[00:33:52.087]You know, so are we going to address that?
[00:33:55.500]So that's kinda where we are going with
[00:33:57.330]in terms of electrification.
[00:33:58.830]But here again the idea of Swarms
[00:34:01.920]can help with electrification too,
[00:34:04.530]because if you think about instead of one 600-horse tractor,
[00:34:09.270]you have 10, 60-horse tractors,
[00:34:11.820]and let's assume they're all electric, right?
[00:34:14.790]So maybe six are working, four are charging
[00:34:18.750]and then you schedule them, right?
[00:34:21.510]So that is a possibility.
[00:34:23.730]So if we are going the route of Swarms,
[00:34:26.100]then some of them are charging and some of them are,
[00:34:29.571]you know, working, but the other alternative
[00:34:31.650]is to have a swapable battery infrastructure, you know?
[00:34:34.170]So that's another way.
[00:34:36.450]So we are continuing to collect a lot more data,
[00:34:39.300]and this is just for planting,
[00:34:41.070]which is a low draft application,
[00:34:43.383]very low draft application compared to
[00:34:45.660]if you're doing tilling or any type of, you know,
[00:34:48.583]soil tool interaction at a deeper level,
[00:34:51.420]then these power requirements are going to be
[00:34:53.700]much, much higher, yeah.
[00:34:59.160]One of the other things is the cybersecurity
[00:35:03.210]of this equipment.
[00:35:05.430]So we are talking about IOT and connected,
[00:35:07.710]everything is connected, but the problem is
[00:35:10.890]there are security issues right now.
[00:35:15.090]So you've, so the JBS packing facility, you know,
[00:35:21.115]it had a ransomware.
[00:35:23.624]So the ransomware is where like these unauthorized parties
[00:35:27.780]will hack into a system and they'll not give you access
[00:35:30.600]until you pay them something.
[00:35:32.940]That's why it's called a ransomware.
[00:35:34.680]It actually already happened on AGCO planting equipment.
[00:35:39.540]So 15 days during the planting season,
[00:35:44.610]your million dollar equipment is sitting like a brick,
[00:35:49.260]not a good situation, right?
[00:35:51.150]So these are some of the things
[00:35:55.124]that we need to be thinking about,
[00:35:55.957]especially if you're talking about connected hardware
[00:35:58.950]and connected equipment and autonomous equipment.
[00:36:03.136]The problems are exacerbated because there is
[00:36:06.676]human intervention is much lesser,
[00:36:09.570]so it's hard to monitor, okay, am I applying something,
[00:36:12.300]or is the machine just moving?
[00:36:13.620]You know, so we are doing
[00:36:14.453]a case study on nitrogen application.
[00:36:16.530]What happens if a nitrogen applicator is compromised,
[00:36:21.510]even though it's saying, okay, a hundred pounds per acre
[00:36:24.210]or whatever the perfect rate,
[00:36:25.470]but it's not actually doing it,
[00:36:27.330]and you wouldn't realize until after you go to harvest.
[00:36:32.790]Yeah.
[00:36:33.623]So that's why the cybersecurity aspect is important,
[00:36:37.200]you know, so whether it's a precision operation
[00:36:39.600]or application rate, you know,
[00:36:42.423]you have an autonomous tractor getting onto the Interstate,
[00:36:47.610]whether, you know, it's hacked or is there a malfunction?
[00:36:50.970]You know, it can be anything, right?
[00:36:53.370]And then the ransomware, right?
[00:36:55.410]So that's why connected equipment is not always, you know,
[00:37:01.170]it's important for remote diagnostics and everything,
[00:37:04.170]but we need to put, you know,
[00:37:09.120]lot of preventive measures in place, you know,
[00:37:12.150]because people are not even thinking
[00:37:13.500]about security at all today.
[00:37:18.360]So I think in the future we would be seeing, you know,
[00:37:24.840]the traditional conventional precision AG equipment.
[00:37:27.480]And when I say future in the next 10 to 15 years,
[00:37:30.570]I would say, you will see robotic equipment
[00:37:34.170]working hand in hand.
[00:37:36.000]And then there will be drones, which already happened today.
[00:37:40.620]But I think, if you want to go
[00:37:42.930]towards that climate smartness,
[00:37:45.960]how do we increase biodiversity on a farm?
[00:37:49.770]Why not put multiple crops, crop mix, you know?
[00:37:52.350]So can we use robots to do that?
[00:37:54.570]I think the answer is yes, and then the computing,
[00:37:59.790]you know, some of the computing might be happening
[00:38:02.160]on the edge on the equipment, and some of them
[00:38:06.210]could be offloaded to the Cloud.
[00:38:08.910]And then there are many bottlenecks here,
[00:38:11.100]and one of the thing is the connectivity piece, right?
[00:38:13.590]So here today in Lincoln, we have 5G, right?
[00:38:17.310]But you go out, you know, to the field,
[00:38:19.800]there's not even one G, right?
[00:38:22.440]So, as robotic researchers and designers,
[00:38:27.510]we need to be thinking we think about that,
[00:38:29.490]so if there is no connectivity, how does a robot react?
[00:38:33.240]What are the onboard resources that it has? You know?
[00:38:37.500]And then I say animal systems here,
[00:38:40.950]because that's also an important aspect, is, can we use,
[00:38:52.177]over the last three, four years have seen
[00:38:53.820]like going away from synthetic fertilizer, right?
[00:38:56.070]So that's where the, okay, can we do integrated crop
[00:38:59.460]and animal systems, right?
[00:39:01.500]So just think about this just for corn and soybeans,
[00:39:07.020]if you wanna talk to a farmer during the planting season,
[00:39:09.660]they'll be like, "No, I don't have any time
[00:39:12.990]to do anything, right? (chuckles)
[00:39:14.520]Now imagine you are putting crops and animals together,
[00:39:18.453]your logistics and stress will be exponential, right?
[00:39:23.460]So I think you need to, when you're thinking about
[00:39:25.680]these integrated crop system, you need to have
[00:39:27.690]highly automated systems that actually work in reality.
[00:39:31.530]So that's one of the ways to envision this type of a system,
[00:39:36.180]you need to have highly automated systems
[00:39:38.040]or else it's very difficult
[00:39:39.750]to manage the logistics, you know?
[00:39:41.460]So, and now, like I was talking about, you know,
[00:39:45.660]how do we train students, right?
[00:39:48.574]We talk, we are talking about robotics,
[00:39:50.100]we are talking about agriculture, soil science, weeds.
[00:39:55.890]So I think the next generation workforce, you know,
[00:39:58.770]have to be trained well, and here is an example.
[00:40:04.730]So this is a robotics competition
[00:40:07.304]that we participated last year.
[00:40:09.150]So this is actually a cotton picking robot.
[00:40:12.132]So here's a camera.
[00:40:14.280]It's looking at a cotton and white background,
[00:40:19.260]and white background, cotton is white and the background
[00:40:21.446]is white, so it's very difficult to differentiate, right?
[00:40:25.500]So that's why we are using something called a depth camera,
[00:40:28.350]so that it not only gives you the color of the cotton,
[00:40:32.100]but also how far it is from the camera
[00:40:35.280]so that we can send this robotic arm
[00:40:39.210]to actually go and pick the cotton.
[00:40:43.410]Okay, so apparently we can do some cotton research too,
[00:40:47.550]in Nebraska, is there any cotton here? (laughs)
[00:40:50.250]There are Texas A&M, there's Clemson, we beat all of them,
[00:40:53.640]you know, (chuckles) so a lot of southern states
[00:40:58.080]did not do very well because we won the competition.
[00:41:00.090]So we have an opportunity here. (chuckles)
[00:41:02.670]So I think one of the other competition
[00:41:08.040]that's coming up is this farm robotics challenge.
[00:41:11.580]And thanks to Amit and Mandeep
[00:41:14.460]we're actually growing weeds in the greenhouse.
[00:41:17.400]I never thought we'll be growing weeds.
[00:41:19.245]We always, I think about getting rid of weeds, right?
[00:41:22.770]So it's like we are growing weeds
[00:41:23.970]to get rid of them, actually. (chuckles)
[00:41:26.490]But the important thing is
[00:41:28.568]we need to train our machine vision models, right?
[00:41:31.830]So how does a weed look like versus corn plant?
[00:41:35.430]So yeah, thanks to Amit, we are using his greenhouse
[00:41:38.750]to grow the weeds so that we can train
[00:41:41.400]our machine vision models.
[00:41:43.200]And our goal is, you know, can we use this robot here,
[00:41:47.430]the Flex-Ro Mini, you know, to actually,
[00:41:51.050]either spray or mechanically pull.
[00:41:54.270]And I think in the future we will be thinking about
[00:41:57.240]like integrated weed management, you know.
[00:41:59.970]If it's a herbicide resistant weed, what are you gonna do?
[00:42:03.333]We got to pull them or do some other technique, right?
[00:42:07.380]So we just got started with this, but you know,
[00:42:11.370]May 10th is just for the competition,
[00:42:13.380]but we'll continue to work in this area, you know, so,
[00:42:18.660]and then, yeah, here is a team that I have,
[00:42:22.410]obviously they're the one who's doing all the work.
[00:42:24.420]I'm just talking here.
[00:42:25.680]So yeah, it's a pretty good team.
[00:42:29.250]So yeah. With that, yeah, this is the Rogers Memorial Farm.
[00:42:35.580]So here's what's happening is
[00:42:36.960]that drone is taking off autonomously.
[00:42:39.210]We are shooting it with another drone
[00:42:40.680]that we are actually manually running.
[00:42:43.680]So...
[00:42:47.310]Yeah, with that, yeah.
[00:42:48.720]I'll take any questions you have, yeah.
[00:42:50.850]We do have plenty of time for questions.
[00:42:52.590]Yeah! Any folks here in the room,
[00:42:54.150]those of you online, again, typed in?
[00:42:57.420]I'll pass the mic around to anyone here in person.
[00:43:08.340]Very interesting.
[00:43:10.650]I'm intrigued with all of the pictures-
[00:43:12.960]Yeah? And sensors that...
[00:43:15.480]I work with corn. Yeah.
[00:43:17.010]And so I think about having this little robot
[00:43:22.530]go through the field and identify something different.
[00:43:26.970]Yeah. Now, to what extent
[00:43:29.067]do you envision this being able to tell you
[00:43:31.590]whether it's nitrogen, phosphorous, potassium, pH,
[00:43:36.447]all those other variables or soil type?
[00:43:41.010]Where will that decision be made?
[00:43:43.740]How do you put that together?
[00:43:45.360]Yeah, that's a really great question. (chuckles)
[00:43:49.506]I don't have a good answer probably, but what I can say is,
[00:43:52.320]so far I've focused on building these robotic equipment,
[00:43:55.530]you know, and I'm trying to identify use cases.
[00:43:59.130]So lot of sensing work that is done today, right?
[00:44:03.930]Using manual methods or using, let's say spidercam
[00:44:08.340]or any other methods, we can easily put that
[00:44:11.370]on this equipment so that we can actually,
[00:44:14.700]lot of times, you know, the frequency of data collection
[00:44:20.040]is limited by how laborious the process is, right?
[00:44:23.610]So we want to take that out of the equation.
[00:44:25.500]So we have this robot, one of the things I envision is,
[00:44:29.460]we'll have this robot 25 by seven in the field
[00:44:32.640]always monitoring for things.
[00:44:35.160]You know, so if there is scientists
[00:44:38.790]who are developing sensors, we can easily
[00:44:40.549]put them on here and collect data.
[00:44:46.890]Let me add one more. Yeah?
[00:44:48.450]The thought here is that in the case of corn-
[00:44:52.620]Yeah? within a leaf
[00:44:56.010]there can be color changes and you envision
[00:44:59.250]having sensors that can scan for potassium deficiency
[00:45:03.360]versus nitrogen, potassium turns brown on the margins,
[00:45:08.790]nitrogen down the middle.
[00:45:10.537]Do you think you'll be able to get that out of it?
[00:45:15.150]Yeah. So again, yeah, that's a great question.
[00:45:18.690]I'm not into sensing specifically,
[00:45:21.210]but if there are proven methods, right?
[00:45:25.020]Like one thing I heard is like the drones
[00:45:27.720]cannot see the side view of the plants, right?
[00:45:30.840]So, I mean, we can put cameras, you know,
[00:45:34.329]so that you are sensing this way, looking sideways,
[00:45:40.530]as you're going between the rows.
[00:45:42.360]And we can put sensors at multiple heights too,
[00:45:45.420]so that you are sensing that corn, you know,
[00:45:47.550]multiple times throughout the growing season
[00:45:49.500]to see if we can characterize that deficiency.
[00:45:52.350]Yeah.
[00:46:01.350]Thanks. Thanks Santosh.
[00:46:02.970]One of the things that I saw an autonomous
[00:46:06.090]dry spreader in the field a year, maybe a year or two ago.
[00:46:10.170]Yeah, and here's the farmer sitting on a four-wheeler
[00:46:12.540]at the edge of the field watching it.
[00:46:14.520]The farmers aren't saving any time,
[00:46:16.710]but they are saving labor maybe.
[00:46:18.987]Yeah, yeah, yeah. But I'm wondering,
[00:46:20.700]you know, when it's this transition from,
[00:46:24.630]how do we save the time by being able
[00:46:27.150]to let that autonomous thing happen autonomously.
[00:46:29.670]I'm not sure we're ready for that. Yeah.
[00:46:32.203]From techno technological standpoint,
[00:46:33.930]nor are we from a liability standpoint. Yep.
[00:46:35.831]I mean, we've had GPS stops on pivots for how long?
[00:46:41.160]No, but we can't trust GPS stops on pivots.
[00:46:43.387]Yeah, yeah. We gotta have manual stops.
[00:46:46.440]Yeah. Because the GPS stop
[00:46:47.580]doesn't always work. Yeah.
[00:46:48.690]So talk just about that transition from-
[00:46:51.397]Yeah. Watching it happen-
[00:46:54.780]Yeah? To let it happen.
[00:46:56.400]Yeah. While you're not there.
[00:46:57.300]Yeah, no, that's a great point because,
[00:47:02.130]you know, so this is where it's like
[00:47:08.790]if you have a, let's say you have animal operation also,
[00:47:13.620]then innovate makes sense because okay,
[00:47:16.530]I program my planter to do planting
[00:47:19.380]and I'm going to take care of my cows, so that's it.
[00:47:22.400]So, you're- Yeah.
[00:47:25.200]Right. Exactly, right.
[00:47:26.580]So I can turn it off, I'm monitoring, but I'm not,
[00:47:28.830]not fully focusing on, is the planting happening correctly?
[00:47:32.760]I'm just monitoring my phone,
[00:47:33.870]but I'm tending to my animals too.
[00:47:36.300]So here, all of a sudden we are talking
[00:47:38.550]about providing opportunities to do integrated
[00:47:41.940]cropping and animal systems.
[00:47:43.620]You know, so that's a possibility.
[00:47:46.237]I think even when I talk to some farmers,
[00:47:50.820]you know, some farmers, they have like
[00:47:52.380]only 200 or 300 acres and they have another job.
[00:47:56.010]So they're like, "Hey, you know, I could use
[00:47:58.050]this machine on a weekend, you know,
[00:47:59.790]I can let it do its thing.
[00:48:01.230]I can do my chores."
[00:48:02.580]You know? So I think there are some models like that,
[00:48:05.670]that we could look into.
[00:48:06.777]And coming to your spreader and then the liability.
[00:48:11.430]So that's one of the driving factor
[00:48:13.800]to think about Swarm robots, smaller robots,
[00:48:17.654]is that is like a 600 horse tractor can do a lot more damage
[00:48:21.780]than a 60 horse tractor if things go wrong.
[00:48:24.780]You know? So I totally agree with you.
[00:48:27.420]I think going back to smaller equipment,
[00:48:30.750]but fully autonomous helps with a lot of things,
[00:48:35.130]you know, compaction, liability.
[00:48:38.040]Then also it's easy to transport
[00:48:39.750]from one field to another field, right?
[00:48:41.790]Now, today, if you want to try, you know,
[00:48:44.760]transport a big piece of equipment,
[00:48:46.320]man, you need multiple trips or multiple equipment, so yeah.
[00:48:55.890]One more, sure? Yeah.
[00:48:57.660]This is just a comment. Yeah.
[00:48:59.444]When I was a student back in 1965, we toured John Deere.
[00:49:03.480]Yeah. And they had a lawnmower.
[00:49:06.990]Yeah? Unmanned lawnmower
[00:49:09.450]roaming around out in the front yard.
[00:49:12.540]It was not guided, the only thing it did was stayed away
[00:49:15.690]from trees and- Yeah.
[00:49:17.269]And all this kind of stuff.
[00:49:18.102]So this is not totally new. Yeah. Nope, nope.
[00:49:24.630]Just a question.
[00:49:26.070]In terms of taking these robots and being able to manipulate
[00:49:32.490]topography, we've got a lot of hills and side hills and...
[00:49:38.100]Yep? Ravines and whatever.
[00:49:40.140]Yep.
[00:49:40.973]In a lot of fields in Nebraska.
[00:49:42.780]Yep.
[00:49:43.613]Have you gotten to that point yet?
[00:49:45.900]Not, not yet, but totally like the stability
[00:49:49.320]and the center of gravity in slopes,
[00:49:52.318]all of that is so important.
[00:49:54.540]So that's where I think there are advantages
[00:49:59.280]of drones or ground machines.
[00:50:02.700]That's some of the things we're looking into is,
[00:50:05.700]you know, we are looking into a hybrid robot
[00:50:08.790]where it'll go on the ground for some time,
[00:50:11.730]but if it has to fly, it can fly, you know?
[00:50:15.582](chuckles) So, but totally, you know, it's an important
[00:50:19.946]center of gravity of the equipment,
[00:50:21.720]especially on the hills on,
[00:50:23.370]yeah it is going to be very important, yeah.
[00:50:25.070]We have not gotten to that point,
[00:50:26.520]but totally agree with that.
[00:50:29.370]And I think compared to bigger machines,
[00:50:32.820]the smaller brands still have a chance.
[00:50:35.730]So. (chuckles)
[00:50:41.580]Mimi, is there anything online?
[00:50:44.288]Yes, there is. Okay.
[00:50:50.160]Milosh says a nice presentation,
[00:50:54.720]except proper weed recognition would lead
[00:50:59.490]towards site specific...
[00:51:01.410]That would lead towards site specific weed management,
[00:51:04.320]it seemed like there's a wide range of weeds
[00:51:06.180]present in the photo you shared.
[00:51:08.087]What are the system capabilities of employing multiple
[00:51:10.940]herbicides and further on how decisions are made
[00:51:16.560]that proper rates are delivered?
[00:51:18.660]So multiple species. Yeah.
[00:51:20.520]Multiple herbicides. Yeah.
[00:51:21.870]Proper rates. Yeah.
[00:51:22.950]My answer is I'll do whatever Amit ask me to do, (chuckles)
[00:51:27.270]but no, I think, you know,
[00:51:31.560]I just started working with weeds
[00:51:33.420]just because there is so many projects funded on weeds
[00:51:38.008]over the years, last 15 years I would say.
[00:51:42.450]And today John Deere has a See & Spray technology
[00:51:46.425]that's, I'm sure that's one of the reasons
[00:51:49.800]why they have cameras on their tractor
[00:51:53.190]so that they can keep collecting data
[00:51:54.930]and improve the models.
[00:51:56.579]So especially for machine learning
[00:52:03.150]and identifying different species, you know,
[00:52:05.040]weeds that are popular in Nebraska
[00:52:06.930]from Amit and Mandeep I learned,
[00:52:08.640]Palmer amaranth and Waterhemp are the main things, right?
[00:52:11.760]So that's kind of what we are focusing on right now.
[00:52:14.730]But then we definitely need a multi-state project
[00:52:20.130]where we need to talk about what are the common weed types
[00:52:24.300]in different states and come up with a database, you know,
[00:52:28.080]so all this robotic, weeding and AI
[00:52:30.990]is only good if the data is good.
[00:52:35.100]So we need to be collecting a lot more data sets
[00:52:38.070]of common types of weeds in different states
[00:52:41.070]and have a database that researchers like me can access.
[00:52:45.150]You know?
[00:52:46.238]But I found out there is no database,
[00:52:47.940]open database for weeds.
[00:52:49.680]I mean, there's a lot of open database
[00:52:51.090]for cats and dogs and you know, you name it, you know,
[00:52:54.240]but there is no good database where you can say, okay,
[00:52:58.350]I want the weeds, let's say Palmer amaranth.
[00:53:01.530]I wanna download all the image data
[00:53:03.330]so that I can train my robot.
[00:53:04.890]That's the reason why we are growing Palmer amaranth
[00:53:07.860]in the greenhouse so that we have good images, you know?
[00:53:11.400]Yeah.
[00:53:12.240]I don't know, Myla if I answered your question or not,
[00:53:14.730]but I think the biggest bottleneck today
[00:53:17.940]is a open database that has different species of weeds.
[00:53:21.750]If we have that, I think we can easily train
[00:53:24.870]a lot of different models and we can
[00:53:27.168]feed that into the robot.
[00:53:29.098]If you want to do spraying, if you, you can have
[00:53:31.470]three or four herbicides, you know, for different,
[00:53:34.320]and the robot identifies which weed it is,
[00:53:36.780]and you spray only that, you know, today we can have
[00:53:40.590]multi nozzle heads on the same sprayer.
[00:53:43.410]So we can do all those engineering pretty easily.
[00:53:47.040]I think the bottleneck is, we don't have
[00:53:49.470]the database of which to train on.
[00:53:52.080]Yeah.
[00:53:52.913](attendee speaking softly)
[00:53:54.480]What's that? (attendee speaking softly)
[00:53:57.720]Yes, yes.
[00:54:01.350]Santosh. That was a great presentation.
[00:54:03.390]I really enjoy learning about the new technology
[00:54:06.120]that you guys are like putting together
[00:54:07.950]and developing for like helping farmers.
[00:54:10.680]Yep. From, you know,
[00:54:13.830]from experience from GPS adoption, we have seen
[00:54:17.670]that it has taken like 30 or 50 years on like, you know-
[00:54:22.240]Yeah. The public on adopting
[00:54:24.270]that technology, right? Yeah.
[00:54:25.790]So 50 years on adopting GPS. Yep.
[00:54:30.030]Knowing that, right, that take some time
[00:54:32.910]for like, you know, there's a lacking in technology adoption
[00:54:36.000]in like production systems, like, you know-
[00:54:38.340]Hm mm? farm production.
[00:54:40.170]What do you see the challenges here of
[00:54:42.750]like now switching for like, you know, smaller equipment?
[00:54:47.133]Yep. And one, is that right,
[00:54:49.440]the challenges.
[00:54:50.273]The other one is saying that, you know, we,
[00:54:54.120]next year or the next five years,
[00:54:55.860]we certainly do the switch. Yeah.
[00:54:58.740]Right? Yeah.
[00:54:59.760]What are like, is there a program in place
[00:55:03.960]to like close that gap of the knowledge
[00:55:08.130]on managing lots of data? Yeah...
[00:55:10.410]Because probably we could like even adopt
[00:55:12.330]new sensors in like, all current equipment,
[00:55:15.000]but now we are like swimming on these big data sets
[00:55:18.169]and you mentioned something about like, you know,
[00:55:22.320]overcoming this challenge, but, you know,
[00:55:24.360]I would like to hear more about that.
[00:55:25.590]Yeah, yeah. Thanks.
[00:55:26.890]Yeah, I think that's a great question.
[00:55:28.890]You know, the adoption, I'll just show you one slide.
[00:55:31.470]I think I know what you're talking about.
[00:55:34.890]So let's see here.
[00:55:36.090]If I can...
[00:55:46.830]So, you know, one of the first things you asked
[00:55:50.250]is like, you know, adopting this robotics technology, right?
[00:55:53.910]So that's where I have this slide.
[00:55:56.040]I didn't go on in detail about it, but, so let's say
[00:56:01.260]there's a farmer who bought this Case tractor, right,
[00:56:04.110]just now, and he is pulling
[00:56:06.900]a 16 row planter with that.
[00:56:09.180]And I tell him like, "Hey, here is a robot,
[00:56:11.310]it's awesome, you know, use it."
[00:56:13.050]He'll be like, "I already invested a ton of money in this,
[00:56:15.780]why would I do that?"
[00:56:17.220]So every farm is different.
[00:56:19.890]So I think what I can tell him is that, you know,
[00:56:25.710]how would you like to transition to use robotics?
[00:56:28.500]You know, what is the transition point?
[00:56:30.180]You know, so, so instead of saying that,
[00:56:33.300]okay, use a fully robotic planter, I could say,
[00:56:36.127]"Hey, you know, what are you doing for core crop?"
[00:56:39.067]"Are you drilling?"
[00:56:39.900]Or, you know, so we need to survey and identify,
[00:56:43.620]you know, the needs, you know, because again,
[00:56:45.510]every farming operation is different.
[00:56:47.490]But let's say there is a farmer
[00:56:49.050]who's still using 30 years old equipment,
[00:56:52.230]you know, for him, instead of asking him,
[00:56:55.357]"Hey, get a new planter and maybe you could
[00:56:57.510]think about robotics, you know,
[00:56:59.460]because you have not invested in precision AG technology,
[00:57:02.880]the last, you know, 15, 20 years,
[00:57:05.340]then maybe it's time to skip one generation of technology
[00:57:08.640]and directly go to this robotic equipment."
[00:57:10.800]You know, so that's again, you know,
[00:57:13.620]there's a lot of surveys that need to be done to understand,
[00:57:16.740]you know, why would they,
[00:57:17.880]what's the goal for them to use robotics?
[00:57:20.010]You know?
[00:57:21.810]So I dunno if I answered you the whole question,
[00:57:24.030]but I'm just talking about the robotics piece there.
[00:57:26.250]The data piece is a whole another story, right?
[00:57:30.240]Because company, red company have their own data streams.
[00:57:36.090]Green company will have their own data streams,
[00:57:38.661]and if a farmer wants to use both equipment,
[00:57:41.490]then there's no interoperability between the data sets.
[00:57:44.550]So in an ideal world, you want interoperability
[00:57:49.410]among all companies, but then that's not how
[00:57:52.073]the AG business works, I guess, you know?
[00:57:54.480]So open source is the way to go, in my view.
[00:57:57.510]Open source technologies, you know, but then open source,
[00:58:01.560]the problem is who is gonna maintain,
[00:58:03.390]who's going to be the leader?
[00:58:05.280]You know, who is going to maintain the open source?
[00:58:07.710]So, yeah. Yeah.
[00:58:12.660]Dr. Pitla, it was really interesting.
[00:58:15.540]My question goes to teaching and training-
[00:58:19.470]Yeah. Of the next generation
[00:58:21.480]of farmers or whomever to use this.
[00:58:25.920]Does it affect what we need to teach students
[00:58:29.400]in this department?
[00:58:30.540]Does it affect, you know, who the farmers themselves are
[00:58:35.430]and how we train them?
[00:58:37.440]How do you see that whole educational aspect?
[00:58:40.950]Yeah, you know, even in,
[00:58:44.070]even within the AG engineering department,
[00:58:46.950]there is only so many credits our students can take,
[00:58:49.770]you know, so I think the best way used to do this
[00:58:53.460]co-curricular activities like this, robotic competitions.
[00:58:56.610]And then
[00:59:01.110]so this farm bar challenge probably is one of the first ones
[00:59:04.380]I think is going to be very effective in training
[00:59:07.800]different students from a variety of departments,
[00:59:11.760]not just AG engineering, you know?
[00:59:13.934]So here you can see in this picture,
[00:59:15.480]there are students from agronomy,
[00:59:18.090]there are plant pathology students, they're AG engineers,
[00:59:21.360]all are working on the same problem of weed management.
[00:59:26.910]So I think we need to create more opportunities like this
[00:59:29.580]to bring like interdisciplinary education, you know?
[00:59:34.020]So that would be one way to do,
[00:59:35.280]because it's so hard to do everything
[00:59:37.530]within your own department.
[00:59:38.730]You know, even in AG engineering we have the same problem.
[00:59:41.310]We cannot teach all the students to code, right?
[00:59:45.000]We need to give the bigger picture of, okay,
[00:59:48.150]here's how your code interacts with a piece of equipment,
[00:59:51.420]and here are the troubleshooting points.
[00:59:53.760]But we cannot expect AG engineers or agronomy students
[00:59:56.580]to be the most efficient coders.
[00:59:58.860]We need to bring in some computer science students too.
[01:00:01.644]Yeah.
[01:00:02.477]So I think core curricular activities is the way to go.
[01:00:05.520]And also focusing on the bigger problem and how to solve it
[01:00:09.660]through interdisciplinary skills is kind of
[01:00:12.450]what we need to, you know, discuss and teach the students.
[01:00:16.410]Yeah.
[01:00:18.930]All right. Well, we are at 4:30 or a little past,
[01:00:21.720]so please join me in thanking Dr. Pitla.
[01:00:24.360]Again, great, great presentation.
[01:00:30.030]Hope to have you back again soon with-
[01:00:31.820]Yeah, thank you. Rapid...
[01:00:33.379](audience applauding)

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Robotic Tools for Climate-Smart Agriculture

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments