Defensive behavior is a function of specific survival circuits, the “aversive brain system”, that are thought to be conserved across vertebrates, and involve threat detection and the organization of defensive responses to reduce or eliminate threat. In mammals, these circuits involve amygdalar and hypothalamic subnuclei and midbrain circuits. The increased interest in teleost fishes as model organisms in neuroscience created a demand to understand which brain circuits are involved in defensive behavior. Telencephalic and habenular circuits represent a “forebrain circuit” for threat processing and organization of responses, being important to mounting appropriate coping responses. Specific hypothalamic circuits organize neuroendocrine and neurovegetative outputs, but are the less well-studied in fish. A “midbrain circuit” is represented by projections to interneurons in the optic tectum which mediate fast escape responses via projections to the central gray and/or the brainstem escape network. Threatening stimuli (especially visual stimuli) can bypass the “high road” and directly activate this system, initiating escape responses. Increased attention to these circuits in an evolutionary framework is still needed.