In this talk, I will discuss an integrated learning and planning approach for flexible and general robotic manipulators. I will primarily focus on the technical idea of leveraging compositional abstract representations built on top of two important spatio-temporal structures: factorization and sparsity structures in state representations (the physical state can be represented as a collection of object states and their relational configurations), and hierarchical structures in plans (a high-level goal can be decomposed into subgoals). I will talk about the design of such representations and the overall architecture in the context of robot manipulation, present methods for learning them automatically from data, and showcase various types of generalization enabled by such a framework.