Appendix 1: TIPS FOR CALIBRATING BIOLOGICAL MODEL: NOTES FROM THE 2015 ATLANTIS SUMMIT

by Isaac Kaplan and Atlantis community

New Atlantis modelers at the Summit requested additional guidance on the steps useful for calibrating models for new systems.

The main software tools used to calibrate the model are OLIVE or DIVE (for visualising spatial output maps in out.nc), a good text editor to look through log.txt, Excel files that use lookup tables to compare model output with observations and assessments (as appropriate) or R (where scripts can be run to do spatial or time series plotting for comparison purposes).

At the Atlantis Summit, two useful R packages were unveiled to help with calibration:

Additional R packages are also being posted on Github:

Conversations about calibration are usually posted to the Atlantis google group (https://groups.google.com/forum/#!forum/atlantis-ecosystem-model), while solutions to problems are posted on the Atlantis wiki (https://confluence.csiro.au/display/Atlantis/Atlantis+Ecosystem+Model+Home+Page).

The general goals of calibration are, in this order:

  1. Prevent all species present in the model from going extinct (unless they do so in the time series being fit to)

  2. Have age structured groups grow such that size-at-age is reasonable (within 20% of initial conditions, typically)

  3. For species with historical data are available, have the model recreate observations of abundance from surveys or assessments

  4. For species with no historical data, the model should yield reasonable time series of abundance (especially under perturbation – such as environmental forcing or fishing pressure)

  5. Capturing observed spatial distributions

Practical experience suggests that tackling model problems in the order above is best, since this sequence minimizes recalibrating bits too many times.

Once size-at-age is within 20% of the von Bertalanffy curve expected from the literature, then growth is generally acceptable. When calibrating age structured groups, attempt to get relative SN and RN values reasonable before worrying about numbers (den results). Note though that if numbers are too high (by an order of magnitude or more) the group maybe starving itself so you may need to reduce recruitment or increase predation to get numbers under control before you will make any headway with the growth of the group.

During calibration, tuning typically focuses primarily on changing growth rates, consumption rates, linear and quadratic mortality, and recruitment (Beverton Holt) parameters. These variables are in the biology.prm file. Below, XX represents the functional group code (e.g. FVD, FPL, etc.) from the functional_groups.csv file:

A thorough exploration of parameter distributions within the Atlantis models would be broadly useful for error checking inputs and understanding reasonable bounds during calibration. As suggested by Ainsworth and Walters (2015), other authors have already built databases and conducted meta-analyses on Ecopath parameters, and comparison of Atlantis values to Ecopath values would be informative. Finally, automated parameter comparison across models lends itself well to basic checks on model assumptions about biomasses and vital rates, akin to the PREBAL diagnostic test of Link (2010).

Calibration with fishing

A major part of calibrating the model is comparing how it performs when faced with various levels of fishing intensity, both real historical values and hypothetical values. This sort of qualitative exploration can reveal underlying problems, such as parameterizations that involve too little recruitment (and therefore inevitable decline of a stock) or too much recruitment (and therefore unreasonable resilience to fishing). Just as testing Ecopath models with fishing is a crucial step (Ainsworth and Walters 2015), Atlantis models can also use fishing to test basic parameterization of productivity. For heavily fished target species, we roughly expect Atlantis to behave like a single species model (and if not, need to consider why). For lightly fished, nontarget or forage species, we expect ecosystem dynamics to matter a lot more, and Atlantis may (reasonably) diverge from a single species assessment prediction.

Beth’s suggestions on fishing scenarios to run are as follows (these in fact mirror some of the Common Scenarios runs for Atlantis Summit)

  1. Unfished - Atlantis spinup estimate of the unexploited system (where possible from virgin, but also what system would it head to from current state if unfished)

  2. Historical removals - can the Atlantis state support historically recorded catches (and suspected misreported catches); does the trajectory match assessment/survey data? (preferably survey data rather than assessment)

  3. Fish the target(s) at current levels (and do so constantly through time)

  4. Fish the target(s) at assessment estimate of RBC (Recommended Biological Catch, which may differ from MSY)

  5. Fish the target(s) at 5xassessment estimate of RBC

  6. Fish the target(s) at 0.2xassessment estimate of RBC

  7. Fish the target(s) at 0.2xcurrent levels

  8. Fish the target(s) at 20xcurrent levels (or more depending on current F)

  9. Shock the system in some other way (e.g. nutrient pollution, habitat degration, etc.)

  10. Combinations of shocks (typically high fishing + a non-fishing shock)

Tips for parameterizing biomass pools (invertebrates)

Benthic invertebrate abundance and distribution data are scarce, especially for non-harvested species, and generalized additive models are one approach to addressing this. Generalized additive models start from patchy observations or field sampling, and use habitat and oceanographic descriptors to predict invertebrate abundance or distribution. This was applied to SEAMAP data in the Gulf of Mexico (Ainsworth et al. 2015), and is being applied in the Bay of Biscay. Expert knowledge was also relied upon to parameterize these species.

Based on responses at the Atlantis Summit, invertebrate rate parameters such as consumption and growth rates are often taken from existing Ecopath models. These models serve as essential steps toward building Atlantis models, yet the assumptions and methods to estimate Ecopath parameters (PB and QB) were not clear to all Atlantis Summit participants. Furthermore, consumption and growth rates for invertebrates were often tuned heavily in Atlantis. A best practice would be to first understand Ecopath parameterization, then document how it informs Atlantis, before beginning the Atlantis calibration.

Invertebrate groups have been particularly hard to calibrate in many Atlantis models. It is not clear if this is due to poor data or noisy population dynamics. Cephalopods, shrimp, jellyfish, and some benthos are problematic. Cephalopods and shrimp often are parameterized with simple juvenile vs adult biomass pools, and users with the setup should view and test the associated code.

Most Atlantis modelers convert from carbon or dry weight to nitrogen using the Redfield ratio. However, greater attention to these conversions is warranted as the same value may not actually be appropriate across vertebrates and invertebrates of different kinds.

Tips for parameterizing age-structured (vertebrate) groups

Starvation and respiration

Parameterization of starvation is a topic that many Atlantis modelers have not focused upon, but the mortality rate for starving fish can be an influential parameter. Most participants at the Atlantis Summit felt that they should investigate higher values of starvation-induced mortality, as well as testing a model run where all diets are set to 0 (with starvation expected to result).

Very few Atlantis models currently include explicit respiration. In theory, respiration can be included implicitly by reducing assimilation efficiencies (from ~0.8 to 0.3 or less) (but see details on respiration in the manual above). Similarly, most models used a standard temperature effect on metabolism (Q10 = 2), but this should be questioned for new models and applications, particularly those related to climate change.

Movement

A mix of approaches to age structured group movement has been applied within Atlantis, and the exact approach needs to be tailored to the research question and model goals. Density dependent movement has been particularly useful for modeling marine mammals and birds in the Baltic Sea and Northeast USA (Link et al. 2010) and some fish groups in Guam (Weijerman et al. 2015). Temperature-dependent movements have less commonly been applied. Often density dependent movement for some species has been combined with prescribed quarterly spatial distributions for other species; these prescribed movements can be based on expert opinion and rough categorization of species into a few general movement patterns.

Reproduction

To model recruitment (reproduction) of fish, most Atlantis models to date primarily rely on one of two forms of the Beverton-Holt relationship. This omits the possibility of ecological effects such as cannibalism on larvae or pre-recruits. When using Beverton-Holt, a starting point for parameterization is to use the spreadsheet available on the wiki, which converts from stock assessment quantities to Atlantis Beverton-Holt parameters. However, subsequent calibration is essential, in large part because ‘recruits’ in Atlantis are much younger than ‘recruits’ in stock assessments, and the intervening mortality means that stock assessment estimates must be scaled up to Atlantis numbers-of-recruits. In general, mammals and birds were often parameterized with fixed offspring per adult.

Growth, consumption, and predator-prey functional response

There are at least three methods for estimating consumption rates, prior to using these within Atlantis. For mammals and birds, Gompertz growth curves have often been used to specify size-at-age. For fish, size-at-age is often taken from von Bertlanffy growth relationships. Then one option is that consumption may be estimated from allometric relationships
Ci = a · (RNi + SNi) 0.7

Where C is consumption , i indexes age, a is a constant, and RN and SN are weight-at-age in terms of reserve and structural nitrogen. This approach is used within the California Current Atlantis model.

A second alternative is that consumption may be estimated from growth:

Ci = b · ((RNi + SNi) - (RNi-1 + SN i-1))

where b is a conversion efficiency of ~10. This approach is presently used within the Icelandic Atlantis model.

A final alternative in ecosystems with better stomach sampling is that daily ration (consumption per day) can be directly taken from field studies. This approach is possible for the Baltic Sea Atlantis. Finally, it was noted that consumption estimates in Atlantis are maximum consumption, and the estimates above were therefore scaled up by 20%-300% to account for this.

A strong recommendation arose from Atlantis Summit conversations regarding predator-prey functional responses: Atlantis modelers should look at the code, understand the functional response chosen, and test alternative responses. Most models are using modified Holling Type II relationships for all or most of the age structured groups, though some species (such as menhaden and marine mammals in Northeast US) use modified Holling type III. The Holling Type II relationship in Atlantis was originally used for filter feeders, and assumptions need to be understood for age structured groups. An Excel functional response demo is available for common functional response.

All users were encouraged to carefully consider and test the functional response options they choose to use in the code. If the traditional modified Holling Type II relationship is chosen, the mum maximum growth rate parameter can usually be inferred from the equations listed above for consumption, and an assumption about assimilation or conversion efficiency. The ‘clearance’ (C_) parameter would be related to water volume filtered by a filter feeding invertebrate; for a fish it is roughly related to search volume. However, modelers in some systems have found that clearance= 0.1 * mum was a good starting point for calibration, while others have had better success with clearance = 10 x mum. Each modeler should test and plot the functional response and the parameters they are using, and consider alternative formulations as well.

During calibration of the model, most users experienced situations with low growth of age structured groups, and in these cases increasing mum was the recommended first step, before adjusting other functional response parameters.

Other mortality

In most Atlantis models, linear and quadratic mortality (mL and mQ in Atlantis) are generally used as calibration parameters of last resort, to represent mortality effects or carrying capacity not explicitly handled the by the model.

Tips on parameterizing biogeochemistry

Atlantis requires initial conditions for nutrients, primary producers, zooplankton, and detritus. Several modeling groups have used Nutrient-Phytoplantkon-Zooplankton-Detritus (NPZD) model output as the basis for these Atlantis initial conditions. These include the Baltic Sea and Strait of Sicily Atlantis models. This was suggested as a useful approach, as long as the NPZD models have been evaluated for skill against field observations. Some regions have extensive in-situ sampling and accessible databases, which can also be used as initial conditions for Atlantis. An example is http://gulfatlas.noaa.gov/ in the Gulf of Mexico.

Nutrient loading has been used in Atlantis models (forcing with nutrient time series), including the, California Current, Guam, Gulf of Mexico (Brand et al. 2007; Ainsworth et al. 2015; Weijerman et al. 2015) and Chesapeake Bay and Strait of Sicily model. One advantage of this approach is that it can be used as a proxy for multi-decadal trends in ocean productivity, even if full multi-decadal oceanography (ROMS etc.) is not available and must be ‘looped’. Within the US, the Environmental Protection Agency has provided time series of point-source and non-point-source nutrients and sediments to Atlantis modelers.

Tips on handling ocean acidification

Several distinct approaches to handling ocean acidification impacts have been tested within Atlantis:

  1. adding linear mortality to calcifying species, across all model polygons (Kaplan et al. 2010; Griffith et al. 2011).

  2. allowing pH to evolve within Atlantis, but forced by atmospheric pCO2 (Weijerman et al. 2015),

  3. Importing pH from a ROMS that includes biogeochemistry (California Current work in prep).

A different approach would be to predict pH or aragonite saturation state from temperature, oxygen, salinity, depth (Alin et al. in revision; Juranek et al. 2009). Based on experience with Common Scenarios at the Atantis Summit, we suggest the first option, before delving into the more detailed approaches.

After pH (or aragonite saturation state) fields are generated within or outside Atlantis, the biological response to pH must be parameterized. This is challenging and involves a high degree of uncertainty; global meta-analyses provide some guidance (Kroeker et al. 2013; Wittmann and Pörtner 2013) but local conditions may require additional expertise. One concern was that these biological responses should be updated periodically as new information becomes available.