Many everyday activities involve the use of one action to modify the effects of another: When driving, shifting gears modifies the influence of pressing the gas pedal on acceleration; when cooking, the rate of adding a particular ingredient modifies the influence of stirring on viscosity. Here, we investigate a general ability to learn how to use actions to control schedules of reinforcement. In Experiment 1, participants quickly discovered the optimal rate of responding on an action that controlled the rate of reward contingent on performing a different action. In Experiment 2, when the modifying action was itself rewarded, participants failed to discover the optimal rate. Implications for formal theories of instrumental behavior are discussed.