Post by Christine Rollinson, Post-doc at Boston University working with Michael Dietze.
This post is the second in a 3-part series describing the PalEON Model Inter-Comparison Project (MIP). The first post provided an overview of why we are using models to study paleoecology. This post will describe the process and series of decisions modelers make and the challenges we face when running a model.
The modeling process actually is similar to designing experiments, and as with any research project, the process begins with a testable central question or hypothesis about the way the world works. Like field studies, model simulations can fall into two classes: 1) sensitivity analyses are similar to manipulative experiments where individual model parameters such as maximum photosynthetic capacity are altered one at a time to determine its influence on ecosystem dynamics; or 2) observational studies where a given set of conditions, such as future, or in our case past climate change, is given to the model and the ecosystem dynamics are described. The paleoecologists, field ecologists, and each PalEON model member are all using the second approach to answer the question of how climate change has affected forest ecosystems over the past millennium. However, the MIP (Model Inter-comparison Project) team is borrowing an element of the model sensitivity analysis approach and instead of varying individual model parameters, is comparing how ecosystems in entirely different models are influenced by past climate change.
The modeling process can be broken down into three steps: 1) preparation of inputs; 2) running of the model; and 3) analyzing the model output. This blog post will focus on the first two steps and the third and final post will discuss what we do once the models have been run.
Model Inputs
At the center of the PalEON MIP is the central idea that despite the myriad of differences among the participating models, they are all being run with common meteorology drivers. Meteorology drivers are a set of environmental conditions, including temperature, precipitation, solar radiation, and CO2 that vary through time and provide external forcing on the models. In the context of scientific experimental design, the meteorology drivers are the independent variables driving the responses changes through time in the models.
Developing the model inputs is not merely a technical obstacle. Several of the PalEON MIP models simulate ecosystem dynamics on sub-hourly time scales and require sub-daily meteorology drivers for the full 1,200-year PalEON MIP temporal domain. This posed an incredible scaling challenge that is common in ecology. Our meteorology drivers originate with two sets of daily data from CMIP5 plus 6-hourly CRUNCEP data. Previous PalEON postdocs over the past several years had to temporally downscale each data set to 6-hourly data and align and bias-correct these data sets spatially to provide a continuous meteorology product.
Choices regarding model-specific representation of plant physiology or ecosystem processes are other inputs that must be carefully considered from an ecological point of view. Most models rely on broad generalizations of plant functional types that often include groupings by green period (evergreen or deciduous), leaf type (broadleaf or needle), and generic biome (tropical, temperate, boreal). The model I’ve been working with (ED) requires a number of physiological parameters about each plant functional type such as cold- and shade-tolerance, leaf cuticular conductance, and root respiration. These are the types of parameters that are often “tuned” or manually adjusted to produce patterns that we ecologists deem reasonable, even if it means the values of the parameters themselves are not. In other words, manipulation of these variables can cause “the right answer for the wrong reasons”. However, because empirical data on many of these processes is surprisingly sparse, determining what is a “reasonable” value is not as straightforward as it may seem.
Model Runs & Debugging
Once the model inputs and settings are squared away, it’s time to actually run the model and let its internal processes work themselves out. When I joined PalEON, I had naively assumed that because many of these models have been in use for decades all of the major kinks and “bugs” would have been ironed out and they would be able to easily handle multi-centennial simulations. What I didn’t realize is that all of these models, are still rapidly growing and there are many technical and conceptual bugs to be fixed. Some bugs have existed for years, but don’t become a problem except in very rare situations. For example, this spring we discovered a slight bug with temperature and light thresholds for phenology that could cause all deciduous vegetation to die if a certain pattern existed in late winter temperatures. Since the PalEON temporal domain is much longer than most other simulation attempts, based on pure probability we are more likely to generate and experience these rare exceptions that cause the model to break. 
Other more challenging bugs are those where everything looks reasonable over the course of a few years or decades, but slight drifts overtime can result in some very strange and difficult-to-explain patterns. While some bugs are simple typos or oversights in the code, some turn out to be conceptual flaws with how ecology is represented in the model. These bugs in particular illustrate why model- and field-based ecologists must work together. Ecologists are needed not only to help identify the bugs, but present reasonable solutions or alternatives for the models.
Model Outputs
Just like field studies, the scientific process isn’t over once the data, or output in the case of models, are produced. For me, analyzing the output to determine what’s actually happening in an ecosystem is where the fun begins. Although one might think that because ecosystem models are all mathematically structured, linking cause and effect would be easy. However, the complexity of relationships and feedbacks within each model means that disentangling relationships between multiple climatic drivers, disturbance, and observed responses can be extraordinarily challenging and full of surprises.
These surprises will be the topic of the next post that will talk more about how the PalEON MIP is being used to answer classic questions in ecology.