The Illusion of High Precision
Dr. Nan Pond
Dr. Nan Pond
21 December, 2018 min read

This article was first published in the December 2018 issue of the Forestry Source.

In the October 2018 Biometrics Bits article, Zack Parisa and Max Nova discussed precision forestry and a framework for putting a dollar value on inventory precision. In this article, I’ll take a look at a subtle but important issue about how precision is often truncated at each step of the forest-planning process, leading to an illusion of much higher precision in management plans than is warranted.

Creating a forest-management plan starts with an inventory, but then requires further analyses, such as growth projection and harvest scheduling, to arrive at an estimate of final profitability (or some other metric that you’re optimizing for, like quality and quantity of deer habitat). Past Biometrics Bits articles have addressed the concept of a confidence interval around inventory estimates, and many foresters have a good idea of what 100ft2 +/- 10ft2 of basal area means in the context of a timber cruise. The “+/- 10ft2” represents the precision of the estimate.

But how does that precision get propagated through growth projection and harvest scheduling? Usually, it doesn’t. At each step of the analysis chain, it is com-mon for the precision of the incoming data to be discarded. It’s easy to miss this, because the output of each step typically includes a confidence interval, but it’s almost always a confidence interval that exclusively reflects the uncertainty in the projection and excludes the uncertainty in the input data. This results in an artificially tight confidence interval on the out-put—you’ll think you’re operating with a lot more precision than you actually are. Needless to say, this can result in some un-pleasant surprises.

Truncating Imprecision in Growth Models

Let’s look at how growth projection typically works. There are lots of different types of growth models with all sorts of different parameters. Much of assessing and comparing growth models is focused on whether they are individual-tree growth models, such as Forest Vegetation Simulator (FVS), which was developed by the US Forest Service, and Forest Projection and Planning System (FPS), developed by Jim Arney of the Forest Biometrics Re-search Institute. There are also stand-level growth models, such as FASTLOB from the Forest Management Research Co-operative. Other differentiating features are distance-independent (like FVS), or distance-dependent models (like FPS)—whether or not the spatial structure of a stand is explicitly accounted for in growth projections. Rather than get bogged down in the differences, however, I’ll generalize the approach.

I’ll start with the cruise data. Let’s say I’ve got 10 plots in my stand. Each plot has some number of trees—on average, maybe 10 trees per plot. I could work this up just like a normal cruise and get an estimate of the average basal area and its precision (for example, 100ft2 +/- 10ft2).

Now I’ll use the growth model to predict the basal area of the stand five years from now. A typical growth model will take the average basal area (the 100ft2 calculated above) and project it forward to something like 120ft2. You might believe this growth model is usually very accurate and feel comfortable that the estimate is 120 ft2+/- 5ft2. It looks like I’ve still got a precision estimate, so where’s the problem?

That +/- 5ft2 is actually just an im-pression of the model—not a reported value! No growth model is perfect; there should be some precision reported. That precision needs to take into account the sample precision, but typically it does not.

The estimate of 120 ft2+/- 5ft2 does not include the imprecision from the original sample (the +/- 10ft2). The actual precision of these two steps combined might be considerably worse—perhaps more like +/- 18ft2 (the exact calculation will depend on the particular growth model). But if you don’t realize that, you’ll think that the projection five years into the future is a lot more precise than it really is. The graphical representation in Figure 1 can help you understand how this works.

Figure 1. Artificially tight reported confidence interval for growth. The actual confidence interval must account for the sampling imprecision in the original basal area estimate.

Many foresters are quite careful to select a growth model that is well-calibrated for their region, forest type, and silvicultural practices. It’s also important to know how the growth model handles imprecision. The variation in cruise data is a valuable piece of information about a stand—we do ourselves a disservice by setting aside that information when we move on to the next level of decision making.

Propagating Precision through The Analysis Chain

This isn’t just a problem in growth models. At each step in the forest decision making chain, important information about precision is discarded. Instead of compounding uncertainty, analyses often pretend like it doesn’t exist. While you end up with a final number for profitability or species habitat, the confidence interval on that number is deceptively tight (see Figure 2).

Furthermore, in my examples, the error bars around each estimate are symmetric, but in real life, they may not be. This is critical, because if a growth model is actually biased in one direction, or per-haps inaccurately calibrated for one of the forest types on a property, the true trajectory of your forest might diverge from an artificially precise future condition.

Figure 2. Imprecision should be compounded and propagated through each step of the analysis process, otherwise the end result will appear far more precise than it actually is.

At this point, overconfidence in the precision of your projections can lead to seemingly mysterious outcomes (or un-pleasant surprises!) that appear to diverge from what the original management plan predicted. Accounting for uncertainty can help you generate more-realistic fore-casts and expectations. As a profession, as we work to improve the tools we use to analyze our inventory data—growth models, harvest scheduling, and habitat modelling—we should work to explicitly account for uncertainty at each step, to improve the overall understanding of what we do and do not know about our current and future forests.

Special thanks to colleagues at the recent Northeastern and Southeastern Mensurationists Meeting for productive conversations on this topic.

The NCX Newsletter

Subscribe for the latest thinking and news from NCX.

about the author

Dr. Nan Pond

Dr. Nan Pond

Director of Certification
Dr. Nan Pond serves as the Director of Certification at NCX. She is responsible for ensuring that our natural capital products reflect the highest quality science as we hone our existing methods and expand into new credit types and new geographies. She is the recipient of the 2020 SAF Young Forester Leadership Award and has held multiple leadership roles within the Society of American Foresters. Dr. Pond earned a PhD in forest biometrics from Michigan Technological University and a Bachelor of Science in forest ecosystem science from SUNY ESF.