What is machine vision?
It’s a two-step process of acquiring an image and then extracting information from it. Human vision is similar. Our eyes capture specific wavelengths of light, our brains process these into an image, and we attach meaning to those images.
Machine vision is a two-step process of acquiring an image and then extracting information from it.
However, machines can capture a much broader spectrum of light than humans can see, and machines have more raw processing power than human brains. “Machine vision is the group of techniques that can be applied to images to extract information, sometimes better than humans can,” confirms Poblete-Echeverría. “For example, with machines we can try to estimate chemical parameters, which is impossible for us with our human capacity.”
Just as the human eye has a light-sensitive retina, machine vision relies on sensors to collect light. The human retina and the machine sensor convert the light into a two-dimensional representation – an image.
This differs from spectroscopy, where light is measured at a single point. Think about it like this: thermal imaging, used for night-time surveillance, generates images based on temperature. Compare this with infrared thermometers, which became commonplace during the COVID-19 pandemic. Both systems utilise infrared light, but one generates an image, while the other produces a single value.
Poblete-Echeverría and Venter are investigating spectroscopy alongside machine vision. For their machine vision work, they are including multi- and hyperspectral imaging. Whereas the human eye sees three bands of light, which we perceive as red, green and blue, multispectral imaging measures three to 10 bands, and hyperspectral imaging measures hundreds – even thousands – of bands.
Optimal image capture
When light hits a surface, some of it is absorbed and some is reflected. The reflected light we see – and that imaging systems capture – carries information because objects differ in their absorption and reflection characteristics.
However, reflected light isn’t just a function of the reflecting surface. It’s also affected by the light hitting that surface in the first place. Furthermore, imaging systems typically capture a mixture of reflected light and light from the source illuminating the surface.
Human eyes evolved to adapt to different lighting conditions, and human brains excel at sorting through noisy information. Machines need help to do this.
“With all our imaging techniques, the external light is the most complicated part, because it changes every day, as well as during each day. And it’s affected by the canopy that covers the bunches but also has gaps,” says Poblete-Echeverría.
“With all our imaging tech-niques, the external light is the most complicated part, because it changes every day, as well as during each day.”
“Imaging something like a bunch that’s not simply spherical or flat is tricky,” adds Venter. “That’s why this project focused heavily on protocol development and investigating the effect of various factors on the quality of the images and the data extraction.”
The researchers have been experimenting to find lighting conditions that clearly differentiate berries and bunches from the background, while generating reflected light that carries useful information.
Poblete-Echeverría explains that imaging sensors are designed for specific wavelengths of light. “If you provide light in that part of the spectrum, the sensor will react and create an image. But if the light is a different wavelength, it registers as white or black. And if the light is too much, it saturates the system too quickly, and everything looks white.”
Researchers are laying the foundation for automated data collection in the vineyard
Vineyard and laboratory
To test different lighting conditions and imaging systems, the researchers worked on Crimson Seedless and IFG Eleven from a commercial farm in the Berg River Region. They applied multispectral imaging to bunches in the vineyard, as well as spectroscopy and multi- and hyperspectral imaging to bunches in the laboratory.
In the vineyard, they used a light box to overcome interference by ambient light. The box had a hole in the top, so it could be positioned around a bunch without removing the bunch from the vine. The box allowed the researchers to experiment with different lighting configurations and intensities.
In the laboratory, images were captured under a fixed light configuration, but the researchers tested different light intensities for the multispectral images. The multispectral camera captured visible (375–650 nm), red edge (710–740 nm), and near-infrared (830–870 nm) light in five bands. The hyperspectral camera captured continuous light in the visible (406–997 nm) and shortwave near-infrared (952–2 517 nm) spectra.
The researchers also tested a visible near-infrared (200–1 080 nm) and a shortwave near-infrared (900–1 700 nm) hyperspectral spectrometer. “Our first approach was to explore images to obtain more information at once,” says Poblete-Echeverría. “Handheld spectrometers are not able to generate an image – they give you the value of a specific spot – but the cost is much lower.”
Spectrometers hold promise for measuring characteristics such as total soluble solids or titratable acidity. However, estimating physical parameters such as bunch mass or berry size requires imaging.
The results so far
During the 2022/3 and 2023/4 seasons, the researchers imaged bunches and berries with different systems under different lighting conditions. They also measured total soluble solids, titratable acidity, pH, berry size and mass, bunch length, volume, mass and compactness, rachis length, and number of berries per bunch.
“We’ve identified the best method-ologies for capturing images and extracting data.”
“We’ve identified the best methodologies for capturing images and extracting data,” says Venter. “Now we’re seeing if we can create models that can accurately predict the chemical parameters. A second student is jumping on board to start unlocking the data set on physical parameters.”
The goal is to identify which technology is best for determining each bunch characteristic, and how to set it up.
For example, a handheld spectrometer could be an affordable option for measuring total soluble solids in the vineyard, while a multispectral camera could facilitate automated crop-load estimation. The practical implementation of either technology would have to factor in lighting and environmental conditions.
The final results, expected in 2026, can be used to guide the development of commercial solutions for characterising bunches.
Meanwhile, Poblete-Echeverría advises growers to familiarise themselves with the basics of machine vision. “We are sure that these technologies are coming to the vineyard,” he says. “There are many applications of these systems if they can be mounted on a tractor or in a packhouse. But before that can happen, you need to do all these tests that we’ve been doing.”
“We are sure that these technologies are coming to the vineyard.”
