We consider Video Object Segmentation from a Meta-Learning perspective where each task consists of segmenting objects in a video given a single annotation. We learn how to adapt the activations of a neural network to segment a given object via two modulation networks conditioned on the available information at test time.