Cerebral visual impairment (CVI) is a leading cause of disability in children. Diagnosing CVI is difficult, however, since patients often have cognitive impairments that preclude the use of traditional visual assessment procedures. We addressed this problem by designing a computer-based system that measures vision in children based on the reflexive tracking of spatially defined visual stimuli. Visual stimuli are moved horizontally across a large computer screen while an eye tracker measures the gaze position of a subject facing the screen. An algorithm determines in real time whether the eyes are smoothly following stimulus movement. When the algorithm detects smooth tracking, which confirms that the subject can see, the behavior is rewarded with musical feedback. To determine visual thresholds, such as acuity and contrast sensitivity, the system automatically adjusts the stimulus characteristics until the limit of ability is identified. We tested children with brain injury that varied in cognitive function. In the majority of children, the system was able to detect eye movements and determine thresholds of visual ability. Measures in healthy children and children with brain injury who can follow commands, showed that tracking thresholds correlated well with classical measures of vision. Our system enables the detection and quantification of CVI from patients who lack the verbal or cognitive skills to participate in standard visual assessment procedures. The measurement of visual thresholds in children with brain injury should enable future studies to determine incidence, natural history and treatment of CVI.