Integration of remote‐weed mapping and an autonomous spraying unmanned aerial vehicle for site‐specific weed management

Abstract

BACKGROUND

Unmanned aerial vehicles (UAVs) have been used in agriculture to collect imagery for crop and pest monitoring, and for decision‐making purposes. Spraying‐capable UAVs are now commercially available worldwide for agricultural applications. Combining UAV weed mapping and UAV sprayers into an UAV integrated system (UAV‐IS) can offer a new alternative to implement site‐specific pest management.

RESULTS

The UAV‐IS was 0.3‐ to 3‐fold more efficient at identifying and treating target weedy areas, while minimizing treatment on non‐weedy areas, than ground‐based broadcast applications. The UAV‐IS treated 20–60% less area than ground‐based broadcast applications, but also missed up to 26% of the target weedy area, while broadcast applications covered almost the entire experimental area and only missed 2–3% of the target weeds. The efficiency of UAV‐IS management practices increased as weed spatial aggregation increased (patchiness).

CONCLUSION

Integrating UAV imagery for pest mapping and UAV sprayers can provide a new strategy for integrated pest management programs to improve efficiency and efficacy while reducing the amount of pesticide being applied. The UAV‐IS has the potential to improve the detection and control of weed escapes to reduce/delay herbicide resistance evolution. © 2019 The Authors. Pest Management Science published by John Wiley & Sons Ltd on behalf of Society of Chemical Industry.

The integration of UAVs for weed and pest detection with currently commercially available UAV sprayers can increase the efficiency of pesticide applications and help manage pesticide resistance.

1. INTRODUCTION

Precision agriculture is the concept of managing crop fields considering spatial variation and local field needs,1 and involves data collection to characterize field spatial variability, mapping, decision‐making, and management practice implementation.2 The increased adoption of precision agriculture has influenced the development of unmanned aerial vehicles (UAVs) due to their ability to conduct low‐altitude operations in small fields.1 UAVs have proved capable of acquiring remotely sensed data at higher spatial (centimeter) and temporal (daily) resolutions than satellites.2, 3 UAV technology offers a desirable platform for precision agriculture data collection that is highly flexible and easy to operate while collecting high spatial resolution data in a timely manner. UAVs also represent an advantageous platform over ground‐based, satellite‐based, or manned aircraft‐based remotely sensed data for detection of pests due to their spatial and temporal resolution capabilities and cost‐effectiveness.3, 4, 5, 6

In both Asia and the western hemisphere the use of remote sensing for agricultural management has been significant, but the adoption and commercial use of UAV spray technology has been considerably greater in the former than the latter.7 Aerial applications prevent damage to the crop from spray equipment and allow timely pesticide applications following adverse weather conditions that may leave fields inaccessible to ground‐based equipment.8 UAVs also create new management possibilities for agricultural systems that are too small for piloted aircraft to operate.1 In general, UAV aerial applications are considered a safer alternative to manned aircraft since the pilot is not in immediate danger.7, 9, 10

In recent years, UAVs have been developed to conduct aerial applications of crop protection chemicals.11, 12, 13, 14, 15, 16 UAV sprayers are capable of treating agricultural areas over complex geographic terrain not easily accessible to personnel or ground‐based machinery.15 Commercial UAV pesticide applications have become available in the USA even though a literature gap concerning the need to optimize the use of this technology presently exists.17 Currently, most UAV sprayers are designed to conduct broadcast aerial applications when operating in autonomous mode. However, recent advances in remote sensing for weed detection and real‐time weed identification using convolutional neural networks and machine learning have increased our ability to implement site‐specific weed management strategies.18, 19 UAV sprayers have the potential to improve the efficiency of applications when specifically treating previously mapped weed patches. For example, Huang et al.20 successfully developed a UAV spray system capable of completing autonomous spot sprays. Furthermore, Castaldi et al.21 used UAV‐generated weed maps and a variable‐rate tractor‐based spray system to achieve herbicide savings of 39% over conventional broadcast applications.

The next step for agricultural UAVs to use them in an integrated system (UAV‐IS) for pest control should have two complementary components: one for remote sensing, detection and mapping of the pest infested area and another with precision spraying capability.3 Although ideally these two components would be in a single UAV, as has been done for ground‐based weed sensor sprayers, flying with a heavy payload (i.e. a tank with pesticide solution) seeking weeds would significantly reduce the already limiting battery charge duration.22 Instead, using small, light UAVs with sensors for weed detection and a UAV sprayer that can fly directly to where the weed patches are can increase battery use efficiency and application time per charge. While UAVs have been shown to be capable of creating maps of weeds and other pests for site‐specific management and pesticide applications, the concept of conducting site‐specific pest management with a UAV‐IS has not been investigated. The present study utilizes a UAV‐IS consisting of two separate UAVs: one for remote sensing and weed mapping and another for site‐specific spraying. The main objective of the present study was to quantify the application efficiency and efficacy of a UAV‐IS compared with a conventional ground‐based broadcast spray system.

2. MATERIALS AND METHODS

2.1. Experimental approach

Field experiments were conducted on two irrigated sod fields in Eagle Springs, NC, USA (35.31 N, 79.71 W) in a Candor sand (sandy, kaolinitic, thermic Grossarenic Kandiudult) soil with 0.75% organic matter, and on two non‐irrigated sod fields in Willow Springs, NC, USA (35.57 N, 78.66 W). In the second location the soil was a Gritney sandy loam (fine, mixed, semiactive, thermic Aquic Hapludult) soil with 1.25% organic matter in one field and a mosaic of Gritney sandy loam and Wedowee sandy loam (fine, kaolinitic, thermic Typic Kanhapludult) with 1.8% organic matter in the other.

The studies were set up as randomized complete block designs with four replications and were repeated in space in two separate sod fields for each location during the summer of 2018. The treatments were UAV‐IS site‐specific herbicide application, ground‐based broadcast herbicide application (i.e. backpack sprayer), and a non‐treated control. A DJI AGRAS MG‐1 octocopter (DJI, Shenzhen, China) was flown at 1 m s⁻¹ at a height of 3 m above the ground (the lowest flight height in autonomous mode recommended by the manufacturer) for UAV‐based herbicide applications delivering 151 L ha⁻¹ with four air‐induction flat‐spray nozzles (TeeJet®, AIXR 11002VP, Spraying Systems Co., Wheaton, IL, USA) at 12 KPa with an effective spray swath of 1.5 m ± 0.5 m. The nozzles generated a droplet profile with 4% fine or smaller (<225 μm in diameter), 4% medium (226–325 μm), and 92% coarse or larger (>326 μm) droplets. This droplet profile made adjustments in the flight path due to wind speed or direction unnecessary. Ground‐based broadcast applications were done with a CO₂ backpack sprayer delivering 187 L ha⁻¹ at 22 KPa and a height of 0.5 m above the ground with a hand‐held spray boom with six air‐induction flat‐spray nozzles with an effective spray swath of 3.0 m ± 0.3 m. The droplet profile was similar to the UAV‐sprayer. Application volume selection was based on the maximum volume allowed by the UAV sprayer and the typical recommended volume for broadcast herbicide applications, respectively. All applications were done when the wind speed was 0.5–2.5 m s⁻¹ and no rain or irrigation occurred in the following 8 h after applications.

To detect and map weedy patches, images were collected at an altitude of 30 m using a DJI Phantom 4 Pro quadcopter (DJI) and the mission planning application Pix4D Capture (Pix4D SA, Lausanne, Switzerland) was run on an iPad mini 4 (Apple, Cupertino, CA, USA). Collected images were stitched together to generate an orthographic image exhibiting a spatial resolution of 0.82 cm per pixel for the entire experimental field using Agisoft PhotoScan 1.4.4 software (Agisoft, LLC, St Petersburg, Russia). Ground landmarks were used for coordinate confirmation.

2.2. Natural weed population study

The experiment was conducted at the Willow Springs, NC, USA location in an out‐of‐production sod field of mixed grasses with a relatively high weed infestation predominantly composed of common lespedeza (Kummerowia striata (Thunb.) Schindl.). Experimental units (10 × 10 m) were identified arbitrarily from the constructed field map using ArcMap 10.5.1 (Esri, Redlands, CA, USA) to include areas that had patches of common lespedeza. There was a buffer area of at least 10 m between experimental units to avoid any issues related to possible off‐target movement. Geospatial data constructed in the orthographic image provided coordinates for the vertices of each experimental unit. A Montana® 680t GPS (Garmin International, Inc., Olathe, KS, USA) was used to locate experimental unit vertexes in the field. Image collection was repeated to map weed patches within the experimental units (i.e. target area). Each experimental unit was subjected to supervised classification techniques and areas were consolidated into two classes: weeds present and weeds absent (Fig. 1). Classification was based on color differences between the turf and the weed using a six‐layer color system that adjusted for brightness variation due to differences in sunlight intensity and background color (e.g. bare ground vs. grass cover). Supervised classification accuracy of weed detection was visually confirmed by randomly selecting 1 × 1 m areas from multiple images. False positives and negatives combined represented less than 1% of the area. The resulting weed maps were used to develop a prescription map for UAV application (Fig. 2). Flight paths for UAV‐based site‐specific treatments were created based on weed density and location within experimental units and coordinates were annotated to guide the UAV application (Fig. 2(b)). Flight paths were determined based on the following criteria: (i) no more than three individual flight passes per experimental unit were allowed and (ii) only weed patches larger than 1 m in diameter and with over 35% weed coverage were targeted. Although we acknowledge these criteria are arbitrary, considering the scale of our plots and weed distributions they allowed herbicide use to be optimized while maintaining site‐specific applications. More flight passes or targeting smaller or less dense weed patches would have created almost a broadcast application, defeating the purpose of site‐specific weed management. Glufosinate (Liberty®, 280.39 g acid equivalent (ae) L⁻¹, BASF Corporation, Research Triangle Park, NC, USA) was applied at a rate of 594 g ae ha⁻¹ for site‐specific and broadcast applications. Since glufosinate is a non‐selective herbicide, its activity would be visual on the target and non‐target areas. Image collection was repeated 14 and 28 days after treatment (DAT) to map and quantify herbicide injury in the experimental area (Fig. 2(c)). Visual ratings of plant injury including chlorosis and necrosis (0 = no symptoms and 100 = plant death) were determined based on on‐site evaluations and compared with UAV images at 14 and 28 DAT.

Figure 1.

Aerial image of a sod field of mixed grasses with an infestation of K. striata (top image). Weed detection and mapping after using supervised classification based on color (bottom image). Identified weed patches are shown in bright green and non‐weedy areas are shown in brown.

Figure 2.

Steps for integrating UAV weed mapping and UAV herbicide application (UAV‐IS). (a) Supervised image classification was conducted to map weeds (Nicotiana tobacco plants as surrogate weeds in a fescue sod field) and determine target areas (blue shapes). (b) After identifying target areas, a prescription application map with specific flight paths (yellow arrows) was generated. (c) UAV images were collected post‐application to identify and map herbicide treated areas based on chlorosis and necrosis of weeds (target) and sod (off‐target). (d) and (e) Overlaying the weed detection (a) and spray (c) maps allowed areas to be quantified as true positives, true negatives, false positives, or false negatives. These classifications were used to determine the precision, accuracy, and efficiency of the applications.

2.3. Surrogate weed population study

This study was conducted in Eagle Springs, NC, USA in a sod field that had perennial tall fescue (Festuca arundinacea Shreb.) and no emerged weeds. Plots were arranged parallel to one another in both trials with an average plot size of 136 m2 and 10 m buffers between plots to avoid drift issues. Potted tobacco plants (Nicotiana tabacum L. ‘NC‐196’) 15 cm tall with three to four leaves were placed on the sod within each plot to simulate weed patches with different densities (3 to 15 plants m⁻²) and shapes (quadrilateral, circle, triangle, and straight line), which were randomly assigned to each experimental unit (Fig. 2). This variation in density and shape was included to create more challenging weed detection and spray situations.

Images were collected at an altitude of 30 m and processed as previously described to generate an orthographic image of the entire experimental area. Weed patches were identified visually from the orthographic map (same detection accuracy as supervised classification; data not shown). Geospatial data attached to the collected imagery provided the locality and extent of weed pressure for site‐specific applications. Flight maps for the UAV applications were designed to maximize efficiency by focusing on treating the target species and minimizing the treatment of non‐target species. All individual weed patches were targeted due to the high spatial aggregation of the target species. A maximum limit of three flight paths was set per experimental unit. GPS coordinates for weed patches were obtained from the aerial field maps and used for UAV applications. Glufosinate was applied at a rate of 594 g ae ha⁻¹ with a carrier volume of 151 L ha⁻¹ from 3 m height for site‐specific and 187 L ha⁻¹ carrier volume from 0.5 m height for broadcast applications.

UAV image collection was repeated at 14 and 28 DAT to map and quantify herbicide injury on the tall fescue. Tobacco plants were removed from the site after application once the herbicide on the leaves had dried and were maintained under greenhouse conditions. Tobacco plants were visually rated based on chlorosis, necrosis, and stunting (0 = no injury and 100 = plant death) for herbicide injury at 14 and 28 DAT. The shoot biomass of each individual plant was harvested 28 DAT and dried at 63 °C for 4 days before plant dry weights were recorded.

2.4. Data analysis

Post‐treatment field maps were uploaded to ArcMap and georeferenced to pre‐application data to be analyzed. Injury distribution and intensity were visually verified on the ground and mapped on the orthographic image (Fig. 2(c)). Maps with weed targets and injured areas were overlaid for data analysis (Fig. 2(c),(d)) to determine changes in the classification of each pixel after herbicide treatment (e.g. weed presence vs. weed absence). Spatial data for each experimental unit were classified into four classes based on the presence of weeds (i.e. target area) and whether or not it was affected by the herbicide (Fig. 2(e)). The four classes were no weeds and no spray (true negative), weedy and no spray (false negative), no weeds and spray (false positive), and weedy and spray (true positive). The total area within each category was added per experimental unit and used to estimate several indexes.23 Thus, application precision was determined per experimental unit as:

$$\text{precision}=\left(\frac{\text{weeds treated}}{\text{area treated}}\right)=\left(\frac{\text{true positive}}{\text{true positive}+\text{false positive}}\right)$$ (1)

Application recall was determined as:

$$\text{recall}=\left(\frac{\text{weeds treated}}{\text{weed infestation}}\right)=\left(\frac{\text{true positive}}{\text{true positive}+\text{false negative}}\right)$$ (2)

The Fscore allows for balancing the application's ability to hit the target with the minimum amount of herbicide required, and is estimated by combining the precision and recall to represent a resolution of application tradeoffs as follows:

$$\text{Fscore}=\left(\frac{2\times \text{precision}\times \text{recall}}{\text{precision}+\text{recall}}\right)$$ (3)

Additionally, a spray ratio was estimated as the total area sprayed per plot [inside and outside the plot (i.e., drift)] divided by the total area of the plot. This ratio was used to assess the overall application efficiency as follows:

$$\text{application efficiency}=\left[\frac{{\mathrm{F}}_{\text{score}}}{\left(\frac{\text{total sprayed area}}{\text{total plot area}}\right)}\right]$$ (4)

The aforementioned parameters, weed control, visual ratings, and dry weight, were analyzed using the GLIMMIX procedure and Tukey's honestly significant difference test for means separation in SAS (9.2 SAS® Institute Inc. Cary, NC, USA) with α = 0.05. Treatments were the only fixed effect for all analyses, while blocking and trial effects were considered random.

3. RESULTS AND DISCUSSION

3.1. Natural weed population study

Weed coverage before spray applications was approximately 36% and did not differ between treatments (Table 1). The UAV‐IS applications were 4% more precise than broadcast applications (P = 0.016). The precision analysis estimates the accuracy of the UAV‐IS to identify and treat weedy areas while leaving non‐weedy areas untreated. Although the broadcast treatment missed only 3% of the target weedy area (i.e. recall) and the UAV‐IS treatment left 15% of the non‐weedy area untreated, the higher precision in the UAV system was due to the fact that only 77% of the plot was treated, while the broadcast application covered 98%. Therefore, broadcast applications covered almost the entire plot, treating all weeds present, scoring a near maximum recall value of 100%. Treating less area with the UAV system, as opposed to the broadcast treatment, increased the weedy area missed by the application (false negatives) by over 10%. Using site‐specific management practices increases the risk of missing target weeds due to errors in detection, applications, or both.

Table 1.

Comparison of weed (K. striata) populations and application performance using different indexes after glufosinate applications with UAV‐IS and ground‐based broadcast systems in Willow Springs, NC, USA

Parameter	Ground‐based broadcast (%)	UAV‐IS (%)	P value
Weed coverage	37	34	0.41
Area sprayed	98	77	<0.0001
Precision	50	54	0.016
Recall	97	85	0.0001
Fscore	66	66	0.98
Application efficiency	68	87	0.0014
K. striata injury 14 DATa	96	81	<0.0001
K. striata injury 28 DAT	97	81	<0.0001

^aInjury estimates were based on a non‐treated control.

There were no differences in the F _score between the UAV and broadcast treatments (Table 1). The benefit of site‐specific management is it leaves non‐weedy areas untreated, increasing precision. In contrast, the benefit of broadcast applications is it ensures coverage over all weedy patches by treating the entire field, increasing recall. Therefore, despite their differences in precision and recall, both the UAV‐IS and broadcast treatments exhibited similar application results by obtaining an equivalent balance between these two parameters.

The UAV‐IS achieved 19% higher application efficiency over broadcast applications (Table 1). Site‐specific management practices using the UAV‐IS resulted in less herbicide applied since less non‐weedy area was treated, consequently increasing the UAV application efficiency.

3.2. Surrogate weed population study

The use of tobacco plants as surrogate weeds allowed weed distributions to be created that were more aggregated than the naturally occurring weed populations at the Willow Springs location (Table 2). Surrogate weed patches covered less than 10% of the area in both the UAV‐IS and broadcast systems. The UAV‐IS treated 60% less area compared to the broadcast application, which covered an average of 93% of the plot. This equated to the UAV site‐specific treatments being 20% more precise than the broadcast treatments.

Table 2.

Comparison of surrogate weed (Nicotiana tabacco) populations and application performance using different indexes after glufosinate applications with UAV‐IS and ground‐based broadcast systems in Eagle Springs, NC, USA

Parameter	Ground‐based broadcast (%)	UAV‐IS (%)	P value
Weed coverage	8	7	0.23
Area sprayed	93	33	<0.0001
Precision	51	71	<0.0001
Recall	98	74	0.0002
Fscore	67	72	0.30
Application efficiency	73	228	<0.0001
N. tobacco injury 14 DATa	82	94	<0.0001
N. tobacco injury 28 DAT	60	85	<0.0001
N. tobacco growth reduction 28 DATa	20	44	<0.0001

^aInjury estimates were based on a non‐treated control.

The broadcast application achieved almost perfect recall, being 24% higher than that for the UAV‐IS (Table 2). The UAV‐IS applications treated 33% of the plot, missing a quarter of the weedy area mapped as a target (data not shown). Conversely, broadcast applications only missed 2% of the mapped target weedy area.

The UAV‐IS and broadcast systems had equivalent F _score (Table 2). The broadcast system treated 92% of non‐weedy areas (false positives), while the UAV‐IS treated only 30%. The 26% of weedy areas missed (false negatives) from UAV applications that lowered its recall value were compensated for by the high precision of the application. Thus, the application efficiency of the UAV‐IS was three times higher than that of the broadcast system (Table 2), and this was favored by the high aggregation of weed coverage in this study, which required only treating a third of the total plot area with the UAV‐IS.

As aggregation of weed patches increases, the application efficiency of UAV systems also increases.24 This explains the dramatic differences in application efficiency between the surrogate weed and natural weed studies, with the former exhibiting highly aggregated and the latter larger and more uniformly distributed weed populations.

3.3. Herbicide efficacy

Broadcast applications caused 15–16% more K. striata injury than UAV‐IS applications at both 14 and 28 DAT (Table 1). It is likely that the differences in injury between treatments were due to the lower recall exhibited by the UAV‐IS, which allowed a considerably higher weedy area not receiving herbicide treatment than for the ground‐based broadcast system.

UAV‐IS applications caused higher tobacco injury than broadcast applications at both 14 and 28 DAT (Table 2). At 14 DAT, plants treated with the UAV‐IS exhibited 12% more glufosinate damage than broadcast treated plants, and this difference increased to 25% at 28 DAT. In both systems, injury decreased between evaluation rates by 9% and 22% for the UAV‐IS and broadcast systems, respectively. Dry weight was reduced twice as much in UAV‐IS sprayed than the broadcast sprayed plants when compared to the non‐treated control. Injury and growth reduction results confirmed that plants in UAV‐IS plots were more negatively affected by glufosinate than those in the broadcast system. These results were unexpected, but they can be explained by two factors. First, the carrier volume of the UAV‐IS (151 L ha⁻¹) was 20% lower and generated fewer droplets with a higher concentration of glufosinate compared to the 187 L ha⁻¹ carrier volume of the broadcast system, likely favoring herbicide absorption by the plant.25 Second, and more likely, the downwash air generated by the propellers of the UAV‐sprayer favored more N. tobacco leaf movement, so leaves were exposed to the herbicide on both the adaxial and abaxial sides, effectively increasing the amount of herbicide reaching deeper into the canopy of the target plants.26 This latter situation was more likely to occur for N. tobacco, which has erect growth and long leaves, than for K. striata, which exhibits prostrate growth and very small leaves.

In the present study we detected deviations from the target weedy area by the UAV‐sprayer of up to 1 m depending on wind conditions. These deviations were mainly due to the GPS accuracy of the UAV sprayer and not to the accuracy of the weed mapping. This geospatial precision was obtained without the use of real‐time kinematic (RTK) technology, which can increase the accuracy of UAV sprayers to 3 cm around the target when determined in relation to a specific geodetic datum.27 Perez‐Ruiz et al.28 utilized a tractor with an RTK‐GPS to achieve intra‐row chemical weed control while saving approximately 50% herbicide compared to broadcast applications. Incorporating RTK technology into a UAV‐IS would significantly increase the accuracy of pesticide sprays, reducing false negatives and increasing recall, but even with higher accuracy, wind speed and direction can greatly influence off‐target movement of UAV sprayers, so adjustments in flight path and special nozzle types might be needed to reduce off‐target movement.29

3.4. Applications of a UAV‐IS for herbicide resistance management

Due to the limitations in payload and battery charge duration,22 it is unlikely that UAV sprayers will replace ground‐based broadcast application systems in the near future, especially in large‐scale farming operations. However, they can greatly improve our ability to manage herbicide resistance evolution. A UAV‐IS can be used to map weed escapes after most conventional weed control practices and especially herbicide applications have been implemented.3 As in the present study, image collection with UAVs at different intervals after herbicide applications can clearly document the efficacy of the application and areas where weed mortality was lower than anticipated and where there is a higher probability of finding herbicide resistance mutations. These mapped escapes can then be quickly and efficiently controlled with a UAV sprayer using single or combinations of herbicides with sites of action that are effective against the potential resistance case.3 Furthermore, the use of spot applications makes it possible to increase the herbicide rate to compensate for the larger size of the escaped weed and thus ensure control.18 In this scenario, the impact on the crop or limitations regarding access due to canopy closure would be minimal. Therefore, the UAV‐IS can be a new tool to delay herbicide resistance and improve the stewardship of herbicide sites of action that are still effective in addition to increasing our ability to reduce pesticide use where weed distribution is clearly aggregated and broadcast applications are not justified. This latter point will have to be weighed against the risk of allowing weed seed production and seed bank increase if all weed patches are not controlled due to detection issues or minimum thresholds for action.30, 31

4. CONCLUSION

Results from the present study indicate integrating UAVs for weed mapping and site‐specific management (UAV‐IS) can create an efficient alternative to conventional broadcast weed management practices that can reduce pesticide use. However, the impact on weed control efficacy will depend on weed distribution and weed morphology. Furthermore, the UAV‐IS can be used for detection and mapping of weed escapes to direct UAV sprayers to control them before seed production occurs to eliminate any potential herbicide‐resistant individuals. Despite the potential of the UAV‐IS documented in the present study, we recognize that the current state of this technology requires the use of several software and technical processes that can be considered challenging by applicators. Therefore, the development of user‐friendly systems that automate image capturing and analysis for weed detection and mapping as well as the generation of prescription application maps is needed to ensure adoption and utilization of the UAV‐IS with a level of ease equivalent to conventional ground‐based systems.