It has been shown that the left and right cerebral hemispheres (LH and RH) have preferences for processing qualitative (or “categorical”) and metric (or “coordinate”) spatial relations respectively. However, categorical spatial information could be divided into semantic- and visuospatial-categorical information. We examined whether semantic- and visuospatial-categorical information were different, and whether such a distinction was differentially lateralized. We manipulated the colors and positions of the standard traffic light sign so that either the semantic- or visuospatial-categorical information changed in a non-verbal format and used these stimuli in a sample-to-match task coupled to the divided visual field method. In the semantic-categorical matching task a LH’s advantage for processing semantic-categorical information was observed even in a non-verbal format. In the visuospatial-categorical matching task, however, neither left nor right visual field advantage was obtained. These results suggest the processing of semantic-categorical information is lateralized in LH and it is distinct from visuospatial-categorical information.