Toward Autonomous Farming

While the automotive industry works diligently towards self-driving vehicles, it’s possible the carrots you’ve eaten recently were semi-autonomously planted and harvested with Case IH equipment by Bolthouse Farms, one of the largest carrot growers in the United States.

And the U.S. is hardly alone. Autonomous agriculture is coming everywhere, and it’s happening much faster than autonomous cars. There are fewer restrictions, far less likelihood of interactions with other farm equipment or people, and a measurable payback for large-scale farmers.

Over the past 15 years, farming equipment giant John Deere has been working to increase automation capabilities, according to Julian Sanchez, director of John Deere’s Technology Innovation Center. Sanchez has been involved in everything from system design to development of the human-machine interfaces that support Deere’s semi-autonomous vehicle systems. The company’s Intelligent Systems Group developed many of the components for Deere’s semi-autonomous vehicles, from the GPS receiver to the embedded controller that helps drive some of the vehicles subsystems to accomplish semi-autonomous jobs.

John Deere’s move into electronics dates back to the late 1990s, when the company rolled out a GPS-enabled parallel tracking feature for it’s vehicles “This is before the U.S. government really unconstrained the accuracy on GPS,” Sanchez said. “From there we basically began to build upon that system, and a few years later we came out with an auto-track feature that allowed the farmer to not even drive a first path. You just told the vehicle, ‘Drive from here to here,’ and it would do it.” In the mid 2000s, the company introduced a system that allowed a vehicle to turn around by itself at the end of a row, and by 2010, the company had fully self-driving vehicles.

“That still required an operator to be in the vehicle,” Sanchez said. “One of the reasons for that is with agricultural vehicles, it’s not just about driving like it is an automobile. It’s also about getting the vehicle tuned correctly to do a job, and that’s a whole different component of autonomy that we have yet to figure out. We have a parallel journey in trying to deliver autonomy to that part of the system. Basically, how do we make all parts of the system autonomous?”

John Deere’s Gen4 display, in action.
Source: John Deere

Grand View Research estimates will reach $243.4 billion by 2025, and farm equipment vendors are lining up for a piece of what could prove to be a very lucrative business.

Leo Bose, marketing manager for the Advanced Farming Systems (AFS) at Case IH, another heavy hitter in this industry, explained that the company has watched how automation and autonomy levels have compressed over the last couple of decades.

“We started steering tractors [autonomously] in the early 2000s,” Bose said. “Farmers get one chance a year to plant their crops, and technology can provide analysis to help them get the best yields.” For example, Case IH combines are equipped with yield monitors to geo-reference the yield in the field so farmers can see the actual moisture of that crop, and how many bushels per acre it is yielding.

“They can look at trend lines year over year. They look at different input practices. While that started back in 1994, now we’re trying to automate that whole process,” he said.

Fig. 2: Case IH’s guidance plowing patterns.

Technology + farming equipment
In fact, farming may provide a glimpse for how autonomous vehicle technology ultimately will fare on highways, starting with public transportation.

“Certain cities like Singapore or Dubai are places that are very well controlled, and you would expect those places to be early adopters of autonomy for public transportation and taxis,” said Marc Serughetti, director of business development at Synopsys. “You might see a separate lane or part of the street reserved for autonomous cars and buses. You need an environment that’s extremely controlled like those places. Based on what’s being talked about in the market, this is where it’s coming. There are also all of the areas we don’t see, and don’t know about such as agriculture, mining, and things like this. Again, these are very restricted and defined areas. There are a lot of autonomous technologies out there in those areas, but how do you bring them into a public space? It becomes a combination of legislation, of infrastructure and about technology in the vehicle — and actually the technology in the vehicle is not the one that’s lagging right now.”

Much like semiconductor design, advances in autonomous technology, machine learning and data analytics are changing the way farmers farm.

To this point, in 2016, Case IH gave a sneak peek of its autonomous, cab-less concept tractor. “You take the cab off and ask, ‘What if?’” said Case IH’s Bose. “We collaborated with Utah-based ASI (Autonomous Solutions Incorporated), a leading edge provide of autonomous vehicle control systems company to the agriculture and mining industries. The autonomous concept vehicle uses LiDAR, radar and camera technology to see the vehicle in the field. It is remotely supervised and geo-referenced in a fenced area that’s not on a public road, so we don’t have the regulations as we would have for on-the-highway versus off-highway.”

In the case of John Deere vehicles, in addition to GPS tracking, a variety of sensors are employed, Sanchez said. Gyroscopes and accelerometers are part of the sensor suite, as well as a camera with machine-vision capability. For example, in its sprayers, the camera-based steering RowSense feature that is mounted on the equipment uses a mono camera. GPS keeps track of the location of the vehicle, especially because sprayers travel at about 20 miles per hour — fast by farming speeds.

“In order to be able to make it really fast through a field of corn that’s already five feet tall, you have to make sure you stay within those rows. We install that camera and the camera basically uses machine vision to make sure that the sprayers stay within the rows. It’s essentially looking for the middle of the row and does those calculations real-time.”

John Deere’s Intelligent Solutions Group developed the machine learning algorithms in house. Some of the team is comprised of engineers and scientists from schools such as Carnegie Mellon and Stanford University, who participated in the original DARPA challenges. The company also has made acquisitions to bolster its technology talent pool.

In fact, technology development runs deep across the agriculture sector. “While the journey never ends, over the last seven years we’ve been constantly updating our whole embedded systems effort, such as the native display [within the equipment]. In the last few years we did a major re-architecture of that to include good capabilities with cloud connectivity such as telematics, the telematics hardware boxes, among other things. We had to make decisions to support all of the cellular chips, embedded WiFi capabilities, and Bluetooth capabilities to the point that equipment coming off the line today has all of those capabilities in the displays. We revamped the user interface. You never really quite keep up with this, but we wanted it looking like it was designed in this in this decade, and it looks that way now.”

John Deere also has opened the door inside its vehicles to embrace mobility, Sanchez said, which means an iPad can be plugged into the vehicles to provide a much higher resolution picture, as well as a new way to transmit data to the cloud.

Case IH has taken a similar path with LiDAR, radar and camera technology. The company also recognizes that while its customer base may not be ready for fully autonomous farming, the technology can be installed on a manned, production tractor today. “What that means is I can take an operator, I can put that operator in the cab, now that’s a manned vehicle going down the road, exclusive of autonomy. I can get that vehicle to that field and can even put multiple units in the field with maybe one supervisor — supervised autonomy in the field, which we would define as category 4 automation with category 5 being full autonomy. Today we have a lot of operators that are in the cab and the vehicle’s steering itself.”

Another player in the technology-agriculture space is Fraunhofer, which has a number of activities within its different sectors. One is a deepening partnership with a Portuguese research institution that aims to develop digitization solutions for precision agriculture and IT applications in agro-businesses to achieve more efficient farmland and forest management, more precise pest control, and better monitoring of plant growth. The goal is modern, sustainable agriculture.

Also, Fraunhofer’s Institute for Industrial Mathematics (ITWM) is developing methods and tools for modeling, simulation, and optimization of agricultural processes, including logic planning models and the transparent presentation of profit by means of a departure from the old planning models.

Semiconductor impact of autonomous agriculture
While the self-driving car has become the poster child for leading-edge technology, advanced automation efforts are underway in more prosaic working vehicles, as well. “Autonomous agriculture using ground-based vehicles looks rather like a combination of self-driving cars and industrial robots,” said Tom Anderson, technical marketing consultant at OneSpin Solutions. “The ability of harvesters and other large agricultural equipment to navigate safely will doubtless leverage experience with autonomous passenger vehicles. Yet their sheer size and potential for damage to humans may demand safety zones and operational guidelines closer to industrial automation,”

For semiconductors in this space, safety requirements will get even more complex, Anderson said. “For example, the well-known ISO 26262 standard for automobile electronics is being extended to include larger vehicles. Further, autonomous agriculture includes drones and larger aircraft performing seeding and crop-dusting operations. This adds airborne electronics standards such as DO-254 and DO-178C into the mix. The navigational demands for autonomous agricultural vehicles may be less than those of a self-driving car in heavy traffic, but other forms of complexity must be factored in and verified using certified formal techniques.”

This should come as no surprise to those familiar with the industrial sector. Autonomous operation and, more broadly, artificial intelligence/machine learning technologies have been closely associated with automotive and industrial electronics segments, and agriculture spans both worlds.

“While the applications are obvious, the characteristics of each segment from a business, technology and deployment standpoint could not be more different,” said Srikanth Rengarajan, vice president of products at Austemper Design Systems. “A few large, vertically integrated operators dominate the Level 3 and beyond efforts in automotive hoping to reach economies of scale at the system/OEM level with component prices, particularly semiconductors, relegated to second-order sensitivity. The industrial segment, which is racing to catch up, is expected by conventional wisdom to include a variety of providers working to ensure interoperability between vendors and across the value chain.”

The agricultural market that has seen a confluence of both trends. “At one end, the imaging subsegment is seeing drones and aerial-mapping vehicles utilizing a superset of the imaging technologies found in advanced driver assistance systems (ADAS) and autonomous cars –– stereoscopic cameras, infrared, multispectral and hyperspectral imaging –– to identify vegetative health,” said Rengarajan. “They are supported by the robotics and autonomous vehicle sub-segment with a host of technologies that enabled autonomous driving –– GPS, automatic steering and obstacle detection. The advantage of a controlled environment helped these segments march ahead of automobiles. At the other end, the sensors sub-segment has the distinguishing characteristics of a ‘many Internet of Things.’ A variety of low-power wireless communication devices hooked up to (increasingly MEMS) soil- and vegetation-measuring devices ranging from pH level and temperature to electrical conductivity and reflected ambient light. Some of the technologies employed for communication include LoRaWAN, SigFox, and the emerging 802.11ah Wi-Fi standard (HaLow). So pervasive is this ecosystem that a trial project to map and measure the tomato industry in New England is aptly termed ‘the Internet of Tomatoes’ project.”

The future, in 5G
An interesting pivot point here for many industries, including agriculture, automotive and semiconductors, is next-generation wireless technology.

“Once 5G really comes online, if you think about the pendulum swing with the real-time capabilities and the bandwidth capabilities of 5G, it really does kind of swing the pendulum again toward saying, ‘It’s viable to do real-time, on-vehicle optimization by leveraging the cloud,'” said John Deere’s Sanchez. “Theoretically, it’s possible.” Just like automotive, we’re exploring what that would look like, and if it promises to be everything that it can be. Of course for us there’s an element of coverage of connectivity. We always end up being the lucky ones that end up having to develop the connectivity architecture as well as the embedded architecture for when the connectivity is not there. But we’re definitely exploring that and looking toward having the opportunity to try that out.”

Gone are the days of the small, independent farm. But the upside is that autonomy and technology could create a future of more sustainable, efficient and optimized agriculture around the globe.

Related Stories
Agricultural IoT: Outstanding In The Field
The Internet of Things for agriculture. Yes, it’s a thing—and a huge market.
Smart Farming Accelerates (blog)
Applying technology to a very well established industry is picking up steam.
Investors Back IoT Startups
Total private funding was more than $1.35B in 2H 2017, but investors also are showing more caution about where they put their money.