The Birkhoff polytope is a long-studied polytope connected to many areas of mathematics. In this paper, we generalize it by considering convex hulls of subsets of its vertices. The vertices chosen correspond to avoidance classes of permutations. We study the structure of two special cases, leading to connections with shellable order complexes, toric ideals, standard Young tableaux, and (P,ω)-partitions. We also find that these polytopes have palindromic and unimodal h^*-vectors.