We define the class of high dimensional graph manifolds. These are compact smooth manifolds supporting a decomposition into finitely many pieces, each of which is diffeomorphic to the product of a torus with a finite volume hyperbolic manifold with toric cusps. The various pieces are attached together via affine maps of the boundary tori. We require all the hyperbolic factors in the pieces to have dimension 3. Our main goal is to study this class of graph manifolds from the viewpoint of rigidity theory. We show that, in dimensions 6, the Borel conjecture holds for our graph manifolds. We also show that smooth rigidity holds within the class: two graph manifolds are homotopy equivalent if and only if they are diffeomorphic. We introduce the notion of irreducible graph manifolds. These form a subclass which has better coarse geometric properties, in that various subgroups can be shown to be quasi-isometrically embedded inside the fundamental group. We establish some structure theory for finitely generated groups which are quasi-isometric to the fundamental group of an irreducible graph manifold: any such group has a graph of groups splitting with strong constraints on the edge and vertex groups. Along the way, we classify groups which are quasi-isometric to the product of a free abelian group and a non-uniform lattice in SO(n,1). We provide various examples of graph manifolds which do not support any locally CAT(0) metric. Several of our results can be extended to allow pieces with hyperbolic surface factors. We emphasize that, in dimension =3, our notion of graph manifold does not coincide with the classical graph manifolds. Rather, it is a class of 3-manifolds that contains some (but not all) classical graph 3-manifolds (we don't allow general Seifert fibered pieces), as well as some non-graph 3-manifolds (we do allow hyperbolic pieces).