Dark Emergence

Dark Emergence is the process by which self-organizing and locally rational dynamics within a complex adaptive system produce globally destabilizing patterns as feedback loops amplify beyond intended operating conditions, absent deliberate harm.

You may have heard some of these other terms. While similar, they are conceptually different:

OPTIMIZATION-INDUCED DYNAMICS

Endogenous Amplification

• Dark Emergence
• Neutral Emergence: Complex patterns arising naturally from local interactions without inherent positive or negative value.

Incentive and Proxy Failures

• Goodhart’s Law: When a measure becomes a target, it stops being a good measure because optimizing it distorts the underlying goal.
• Perverse Incentives: Incentive structures that unintentionally encourage undesirable or counterproductive behavior.
• Tragedy of the Commons: Resource depletion caused by individuals rationally pursuing self-interest in a shared-resource system.

SHOCK-TRIGGERED DYNAMICS

• Systemic Risk: The risk of widespread collapse due to fragility and interconnection within a system.
• Black Swan: A rare, unpredictable external event with large-scale impact.