Although digital fabrication has made it possible to replicate complex digital models as physical objects, these technologies are not well suited to unpredictable materials or environments. We describe a mixed reality application that improves the ability of human construction teams to rapidly fabricate complex structures by intuitively working from approximate holographic guides coupled with precise digital models generated from as-built conditions. A pavilion built from scrap timber illustrates the application of our system both in generating, fabricating and assembling non-standard timber elements using hand held tools, and as a collaborative environment for simultaneously designing and making