The purpose of this study was to investigate the neuropsychological process and timing of appreciating humor upon reading a set of Japanese riddles which were made up of 4 distinct phases: "Given A (1st phase), I'd say B (2nd phase). Do you know why? (3rd phase) It is because X (4th phase)." To investigate how the brain responds when an individual finds the answer to the riddle ("It is because X") humorous, this study used the fMRI method. We found that when a participant judged the answer to the riddle humorous, the bilateral amygdalae had been significantly activated at the 4th phase. When the participant judged the answer to the riddle banal (non-humorous), by contrast, the left amygdala was found to be significantly activated earlier at the 3rd phase. Therefore, we argue that in both cases, activation of the amygdala is related to the detection of optimal relevance.