Previous work has shown that state abstraction can be an efficient way to plan in robotic environments with continuous actions and long task horizons. Although some of these works learn predicates for state abstractions, they often neglect an important part needed for generalization. Namely objects and their properties, our work shows that object properties can be exploited to increase the generalizability of learned models. We do this by extending a framework based on predicates for state abstraction and introducing affordances. Affordances allow us to group objects in a way that also considers possible actions. In order to learn affordance interpretations we make use of querying. First, the agent is tasked with solving a task made to test a specific affordance, then based on its uncertainty about an object's affordance interpretation it may query an expert. For example, when presented with a task where it needs to pick up an object it may query the expert on whether the object is actually grippable. We compare our approach with a baseline without affordances and show that our approach needs fewer operators to plan and that our approach is able to generalize to novel objects.