The development of environmental laws and regulations is often grounded in studies produced by researchers who examine human impact on nature. However, the methods behind much of this research are flawed, assuming a correlation between a policy and a result without sufficiently analyzing all the available data.

Consider analyses of national park designations: some researchers might reach the conclusion that a park designation saves the land from degradation. Such conclusions ignore the probability that the land probably was chosen for a park because it lacked economic value and therefore would not have been a candidate for development, with or without the park designation.

Failures to verify the foundation on which scientific conclusions are being drawn are a common shortcoming across much of the coupled human and natural systems (CHANS) literature. These omissions make it difficult to gauge the legitimacy of the studies’ findings and undercut their credibility with policymakers.

To make credible inferences about causal links in complex systems, researchers often make two very basic assumptions when analyzing data: excludability and no interference. These assumptions rarely are satisfied in most published studies on CHANS. In CHANS, multiple confounding variables, both observed and unobserved, create complex feedback across social and environmental dimensions.

Researchers from Johns Hopkins University, Duke University and the University of California at Davis reviewed one of the largest CHANS literature studies: measuring the impacts of marine-protected areas on fisheries. Among nearly 200 studies that aimed to determine if protected areas helped or harmed fisheries and the communities that depended on them, only one study addressed whether the two basic assumptions of excludability and no interference were satisfied.

The research is published in the Proceedings of the National Academy of Sciences.

Schematic of the coupled human-natural system with and without biological dispersal. Source: Johns Hopkins University, Duke University, University of California