Figure 2. Range, median, mean and USDA average values for 8 minerals measured by the BI labs by crop. The range equals the area from the minimum to maximum value after the top and bottom 5% of observations have been trimmed to remove outliers.

Figure 3. Range, median, mean and USDA average values for antioxidants and polyphenols by crop. The range equals the area from the minimum to maximum value after the top and bottom 5% of observations have been trimmed to remove outliers. There are on recommended daily allowances for antioxidants and polyphenols.

Sources of Variation

In 2018 and 2019, the BI identified climate region, soil type, crop variety and farm management practice as sources of variation for which we could collect meaningful sample metadata. An additional goal of the 2020 season, based on previous seasons results, was to better balance the samples coming from different regions, soil types and farm practices so that we could better understand these individual effects. Figure 4(a,b) displays the box and whisker plots for Ca in kale by climate region and crop variety. Looking at the number of observations for each climate region, we can see that we did balance the samples much better than in previous years, with many regions having a similar number of samples. We can also see that the climate regions with the most observations show a large range in Ca content. These results suggest that while climate regions may affect variation, they are not the driving force. Kale variety (Fig. 4b) does seem to impact variation, but many of the samples have few observations, making it difficult to separate out variety from other potential sources of variation, such as management. Figures 4c and 4d display the same parameters (Ca in Kale) from climate region and crop variety using the Data Explorer Dashboard.

In both the kale variety boxplot (Fig 4b) and variability plot for crop variety from the Data Explorer (Fig. 4d) it is clear that Lacinato kale, a common Kale variety, exhibits a large range of Ca values. Those values range from well below the USDA average to well above the average (Fig 5). This wide range within a single crop variety suggests that other factors beyond variety itself are driving those variations.

The next step in exploring the sources of variation in Ca content in kale was to use the same non-parametric analyses used in 2019 and in the 2020 Grains Report to further investigate if farm practices are affecting Ca content in Kale. As previously mentioned, the observational and non-standardized nature of this study makes using traditional statistical methods nearly impossible. Therefore, we use non-parametric analysis to test the “median shift” (percentage change) of individual nutrients grown under different practices compared to a reference set of samples. Results are split into three categories based on our level of confidence. Dark shading (green or red) indicates that the observed effect is more statistically likely to be real (p value < 0.1). Medium shading indicates that the observed effect was not quite statistically significant (0.1 < p < 0.5) but was close enough to warrant follow up investigation to prove or disprove that the effect is real. Finally, light shading effects are just as likely to be caused by random chance as by farm practice and therefore should not be investigated. The color represents the direction of the effect, green is positive and red is negative, and the number is the percent change.

By investigating median shifts, we can explore the impacts of specific management practices more effectively. For example, kale grown with “heavy tillage” (defined as tillage 6 or more inches deep) had Ca content that was nearly 30% lower than kale produced using no-till management. Likewise, comparing different no-till practices to tillage, kale grown using solarization or broadforking had 16-20% higher Ca and using sheet mulching had 32% higher Ca than tilled kale samples. In this example, all of the effects listed are statistically significant.

Linking farm practices, soil health, and nutrient density outcomes

Within the regenerative agriculture community there is a belief that “regenerative practices” will improve soil health, which will in turn improve the nutrient density of the crops grown in that soil (https://regenerationinternational.org/why-regenerative-agriculture/). However, given the number of factors that influence nutrient density--climate, soil type, crop variety, management practices, etc--it is difficult to test this hypothesis. By conducting the annual Food and Soil Survey and capturing detailed metadata on these factors, we are building the deepest library of this data that is currently available. By examining this library we can then identify trends and understand where to look next.

To set up this examination, we started 2020 with the goal of dramatically scaling up on-farm data collection. We accomplished this by increasing the number of samples directly from farms from 813 samples (40% of all samples) in 2019 to 2260 (58.7% of all samples) in 2020 (Fig. 7). Additionally, the BI lab received 1,566 and 1,730 soil samples respectively from 0-10 cm and 10-20 cm soil depths, associated with 2,040 different produce samples (53% of all samples). This allows us to evaluate the impact of farm practice on soil properties measured by the BI labs (soil carbon and respiration), and those soil properties on nutrient density outcomes.

Farm practice and nutrient density outcomes

We began this investigation by examining the median shift that different amendments and tillage intensities produced directly on nutrient outcomes (Fig. 8). A close examination of the table reveals that the results are often contradictory, with a given practice positively affecting one crop and negatively affecting another crop. For example, using mulch resulted in 22% higher antioxidant content in beets but 63% lower antioxidant content in carrots. Likewise, the same practice or amendment may have a positive impact on one analyte but a negative impact on another analyte in the same crop. Again using mulch as an example, antioxidant content in zucchini grown with mulch was shifted down 5%, while polyphenol content for zucchini was shifted up 10%. One reason for these dramatic shifts in effect size and direction is the small sample sizes available. There were 35 beet samples grown with mulch, but only 9 carrot samples grown with mulch. 

Even though we drastically increased the number of samples analyzed in BI labs in 2020, we received between 36 and 480 samples per crop. This is within the same range of samples we received of individual crops in 2018 and 2019. Therefore, we did not gain any statistical power to analyze the effect of farm management on nutrient density outcomes in crop samples. In a few years, after aggregating numerous years worth of samples and building a larger library of outcomes, these median shift tables may be more informative. At this time, however, we need to identify ways to aggregate samples together to produce larger sample sizes. 

In 2019, we started experimenting with a Bionutrient Quality Index (BQI) value, which was developed from normalized values of a subset of nutrients and minerals measured in the BI lab (Box 1). An example of this would be to rank tomatoes and peppers from highest to lowest for a given nutrient (e.g. antioxidants), then the highest tomato and pepper samples are given a score of 100, and the lowest a score of 0. Once complete, all of the crops have the same range of values, 0-100, and can be compared to each other. Next, BQI combines multiple nutrients and minerals into a single value we can use to aggregate complex data to: 1) take a more holistic look at how nutritious a sample was and 2) combine a number of crops together to increase the statistical power of our comparisons and learn more from the data. In 2019, when the Food and Soil Survey included only six crops, we were able to analyze all six crops together to examine the relationships between farm practice, crop labels and nutrient density outcomes. However, after expanding the Food and Soil Survey to 20 crops, this analysis did not yield useful results (data not shown).

Important Note: The first goal of the Food and Soil Survey is to build a library of outcomes. We will engage with experts to continue to analyze the results from the library. This may mean refining the methods included in this report or changing our methodologies completely to provide the most robust analysis possible of the library

To refine the BQI approach, we decided to group crops with similar management together. We should not expect that a given farm practice will have the same nutrient density impact on a specialty crop grown on a partial acre as on a grain crop grown on fields ranging from dozens to hundreds of acres in size. Nor should we expect the relationships between tillage and nutrient density outcomes to be similar for perennial crops and tuber/root crops. Therefore, we aggregated wheat and oats as ‘grain’ crops and apple, blueberry and grapes as ‘fruit’ crops with similar perennial management (Table 3). 

Figure 9 displays the effects of farm practice on BQI for each of the five crop types outlined above. Like the median shift tables, we saw inconsistent effects across different crops. For example, cover crops seem to have a positive impact on vegetables and fruit, but negative or not significant effects on grains and tubers. Additionally, with the exception of no-till in grains, the size of the effect is usually very small, less than 5% different from the reference.

Note: The large increase in BQI in grains was, in part, related to a single farmer organization that participated in the BI and had high quality nutrient outcomes. However, we did not have enough samples from across the US to determine if the effects were due to no-till or other climate and management effects. For more details about no-till in grains, you can read our grains report here.

This section has mostly focused on the challenges that present themselves when trying to identify quantitative relationships between farm practices and nutrient density outcomes in specific crops. This is not surprisingly given the complexity of natural systems or soil ecosystems in particular. All of the results in this section will be affected by the sources of variation laid out previously--climate, soil type, and crop variety--as well as small sample size effects from the large number of crops included in the survey. Therefore, in the next section we will focus on the relationships between soil parameters and nutrient density outcomes and between farm practices and soil health, independent of the crops being grown.

Impact of Farm Practice on Soil Parameters

To begin this examination, we used multiple linear regression analysis to predict antioxidants, polyphenols and BQI for crop samples using only the total carbon, respiration and pH values generated for soil samples in the BI lab. Both total carbon and respiration are recognized as indicators of soil health. Eleven out of the twenty crops tested in 2020 had sufficient samples to test this relationship, which is presented in Table 4. While the strength of the relationships ranged from very weak (r2 < 0.1) to quite strong (r2 = 0.61), there were many statistically significant relationships between soil properties and crop quality outcomes. In beets, for example, soil carbon and respiration from the 10-20 cm depth range were positively correlated with antioxidant, polyphenol and BQI outcomes. 

These results indicate that soil health does influence nutrient density outcomes, but not always in a positive way. We will explore why some of these relationships may be negative a little later in this section. More importantly, this statistical link between soil health parameters and nutrient density allows us to focus our examination on the connection between management practices and soil parameters, instead of looking for direct links between management practice and nutrient density outcomes in each crop.

To provide an example of the type of analysis that is currently possible with the library of data on hand, we will examine the effects of tillage intensity on soil carbon and respiration. Figure 10 presents the median shift of soil carbon and respiration values, compared to a no-till reference, for crops grown using light and heavy tillage. For the purposes of this analysis, a crop was grown using heavy tillage when at least one of the tillage passes was six or more inches deep. Light tillage is defined as when none of the tillage passes reached 6 inches in depth. We choose these definitions because it is easy to process the existing management data captured by producers to automatically categorize tillage as “light” or “heavy.” For grain crops, soils grown using both heavy and light tillage had less soil carbon than soils under no-till management. However, in produce farming systems, soils with light tillage had more carbon than no-till soil, while soils under heavy tillage had significantly lower soil carbon. Also, we see that in grain systems, soil respiration increased with the use of tillage while it decreased in the produce farming systems.

Tillage as a practice is known to destroy the continuous soil pore network and infrastructure that has been created by soil organisms. Conversely, no-till supports the activities of the soil biota, improves soil's ability to store moisture, and increases soil carbon content.[1] Additionally, tillage mixes the soil, bringing organic matter into contact with soil microorganisms, often stimulating biological activity, and re-distributing organic matter throughout the plow layer. These seemingly conflicting results are a strong reminder that context matters. The average grain grower partner owned 3,173 acres. In contrast, the average produce grower partner owned 53 acres, and over 50% of our produce grower partners owned less than 5 acres of land. This difference in scale will affect management practices, and therefore the impacts of those management practices. For example, while it was common for small-scale produce partners to add numerous organic amendments to the soil, most grain partners added little or no organic amendments. Therefore, in grain systems the primary source of organic material was the crop residue, so the act of tilling in the residue brought the residue into contact with, and provided a food source for, soil microorganisms. The end result was that the residue was mineralized more quickly, with a significant portion of the carbon from that residue being converted into CO2 by the soil biology, so total carbon went down and soil respiration went up. Conversely, the regular organic additions to soils in smaller diversified vegetable operations were enough to offset carbon loss from light tillage, leading to increased soil carbon at the 0-4 inch depth. When sampling, producers were asked to move aside organic amendments and mulch layers and to sample only the mineral soil, so layers of thick organic materials that may build up in no-till systems were not sampled. For produce samples under heavy tillage the significant reduction in soil carbon is most likely driven by two factors: increased mineralization of organic materials due to soil disturbance, and mixing of carbon below the 8 inch depth that was sampled via deep tillage operations. 

The varying impacts of tillage practices on respiration for different crop groups may be driven by initial differences in the soil carbon levels in those fields. Figure 11 presents the median soil carbon and respiration values by crop at the 0-4 inch depth. Many of the specialty crops included in the BI have higher than average soil C and respiration levels. For example, the average soil that produced mizuna, bok choy, spinach, kale and lettuce had soil C values greater than 5%, whereas wheat, oats, potato and blueberry had average soil C values between 2.5 and 3.5%. The same trend held true for respiration, with specialty crops generally averaging greater than 25 ug C g soil-1 and wheat, oats, potato and blueberry having less than 25 ug C g soil-1. These results suggest that when soils are carbon-limited, the practice of mixing organic materials into the soil increases respiration. Conversely, when soil carbon is not limiting, tillage-based disruptions of microorganisms reduce respiration.

The previous example shows the types of analyses that can be done using the data in the BI library. Realistically, however, there are so many different management practices captured on so many crops that we would struggle to include them all here. Over the next year, the Bionutrient Institute will continue to engage with soil scientists, agronomists and other researchers to investigate the relationships between farm practices and soil health parameters. Those results will be shared to the public in the form of infographics and peer-reviewed scientific journal publications. In the meantime, members of the public can use the Data Explorer to investigate relationships of interest to them. For example, figure 12 uses the Data Explorer to look at the effect of mulch, synthetic fertilizer and organic amendments on soil respiration in potato fields. Not surprisingly, mulches and organic amendments increased soil respiration over the average while synthetic fertilizer decreased soil respiration.

Predicting Variation

In 2018 and 2019, the BI evaluated the feasibility of using a low-cost handheld sensor--the Bionutrient meter--to predict nutrient density in crops. To do this, each food sample analyzed in the BI lab had its reflection spectra measured with the Bionutrient meter (10 channel spectrometer ranging from 365-940 nm) and the Siware spectrometer (1300-2500 nm range, 30 nm resolution). First, the surface reflectance was scanned and then the sample was juiced and the reflection spectra of the juice was measured. This allowed the BI lab to generate a large database of spectral data and nutrient outcomes. Prediction models for nutrients were then tested based on this database using linear regression and the Random Forest ensemble machine learning methods. Some of the key questions we sought to answer were: 

• Can the lower-cost, but less accurate, Bionutrient meter capture enough spectral data to generate predictions?

• Does the increased resolution of the SiWare spectrometer translate into significantly better nutrient predictions?

• Can we develop prediction models based on raw samples (eg. whole leaves or fruit) or do we need to juice the sample to develop a usable prediction model?

1. In evaluating the database and prediction models in 2019 we determined that:
   The Bionutrient meter, combined with appropriately attainable metadata, is just as effective at predicting nutrient quality as the benchtop Siware device.

2. Using variety data may provide a large boost to the predictive capacity of the Bionutrient meter. Therefore, the BI should put more emphasis on capturing variety data in 2020.

3. The best predictive capacity came when attempting to categorize samples as above average, average, or below average, not when predicting absolute values.

This year, instead of evaluating the potential to predict nutrient concentrations in crops, we evaluated prediction models through the lens of which models we would feel comfortable releasing to the public as functional prediction models. More specifically, we sought to develop prediction models that used the Bionutrient meter to estimate the level of three nutrients in crops: antioxidants, polyphenols and BQI.

To guide the release of prediction models we developed a set of criteria that we could easily communicate to the community:

1. The models would only require data that could be easily collected by a non-scientist member of the community when they were measuring the spectra of the food with the Bionutrient meter.

2. We will predict the ranking of a nutrient and not its absolute value. At this early stage of model deployment it is more useful to tell a consumer where an item of food ranks in our database than to display the absolute value (ex: Kale leaf ‘X’ has 255 FRAP units of Antioxidants). This gives the community a sense of the relative quality that is easy to interpret.

3. We will provide the user with a value range at an 80% accuracy level. For example, a model will output a ranking (ex: 72) and a range (ex: 60 - 85). These values reflect the model's best guess of the percentile of the nutrient and the 80% confidence range (we are 80% confident the value falls within the given range).

4. The maximum allowable average value range, or width, was set to 35%. Therefore, any model whose width was greater than 35 at the 80% accuracy threshold will not be deployed.

Furthermore, in 2021 we will target sampling of crops that fell just short of our deployment criteria to boost the number of observations and (hopefully) release models for these crops early in 2022

In addition to developing a subset of functional prediction models for a beta release, we tested 20 different prediction models for each crop nutrient using different sets of spectral data and metadata. More details about all of these prediction models are available in the detailed data analysis briefs developed for each crop type.

- Grains Data Brief.
- Leafy Greens Data Brief.
- Vegetables, tubers and fruit Data Brief.

One theme in 2020, that is consistent with results from 2018 and 2019, is that using the Siware spectrometer or homogenizing the samples through juicing or grinding samples did not significantly increase the quality of the prediction models (data not shown).

Next Steps for model deployment

Releasing a set of functional models to the public in 2021 is the realization of many years of work within the Bionutrient Institute. However, it is only the first step in what is sure to be a long process. In order for the Bionutrient meter to be a useful tool within the food supply chain, building transparency and trust in the outcomes are long-term requirements. The first requirement, transparency, has been a key principle of the lab and meter development process since 2018. All of the data and models used are open source and available to the public. The second component, trust, will require testing and interaction with the community of users. 

We are working with Citizen Science and Grower partners who purchased a Bionutrient meter to validate the in field predictions. To do this, partners will submit a subset of samples that they test using the Bionutrient meter to the BI lab for lab analysis of the same samples. This will allow us to compare predictions to actual lab outcomes. This data will help us optimize the prediction models that we have released to:

- Increase the accuracy of the prediction models
- Reduce the width for the 80% accuracy predictions
- Expand the number of nutrients we can predict

Furthermore, throughout the 2nd half of 2021, we will work with our 2020 data set to release more prediction models. In particular, we will target crops that fell just short of our deployment criteria to boost the number of observations and (hopefully) release models for these crops by the end of 2021.