Integrated Assessment Models (IAMs) couple representations of the natural climate system with models of the global economy to capture interactions that are important for the evaluation of potential climate and energy policies. The U.S. federal government currently uses such models to derive the benefits of carbon mitigation policies through estimates of the social cost of carbon (SCC). To remain tractable these models often utilize highly simplified representations of complex natural, social, and economic systems. This makes IAMs susceptible to oversimplification by failing to capture key features of the underlying system that are important for policy analysis. In this paper we focus on one area in which these models appear to have fallen into such a trap. We consider three prominent IAMs, DICE, FUND, and PAGE, and examine the way in which these models represent the transient temperature response to increases in radiative forcing. We compare the highly simplified temperature response models in these IAMs to two upwelling diffusion energy balance models that better reflect the progressive uptake of heat by the deep ocean. We find that all three IAMs are unable to fully capture important characteristics in the temporal dynamics of temperature response, especially in the case of high equilibrium climate sensitivity. This has serious implications given that these models are often run with distributions for the equilibrium climate sensitivity that contain a positive probability for such states of the world. We find that all else equal the temperature response function utilized in the FUND model results in estimates of the expected SCC that are up to 25% lower than those derived with the more realistic climate models, while the functions used in DICE and PAGE lead to expected SCC estimates up to 40% and 50% higher, respectively.Paper submitted to the special issue The Social Cost of Carbon