Agent-based modeling for decision making in economics under uncertainty

,


Terms of use:
Documents in EconStor may be saved and copied for your personal and scholarly purposes.
You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public.

Introduction
Ever since the emergence of economics as a distinct scientific discipline, policy makers have turned to economic models to guide policy interventions.If policy makers seek to enhance sustainable economic growth, the economic system (in our case, an open but regulated capitalist economy) and the behavior of economic agents populating it need to be taken into account.As technological change is the primary driver of economic growth (Fabricant, 1954;Solow, 1957), the dynamic efficiency of the neo-Schumpeterian process of technological competition of firms should be of particular concern to policy makers.Dynamic efficiency of the economy as a whole coincides with opportune balancing of exploitation of the current product range and exploration, absorption, and development of radically new products (Cyert & March, 1963;Tushman & O'Reilly, 1996;Pyka, 2015).In this paper, we associate exploitation and exploration by firms with, respectively, (i) microeconomic management of their daily production and sales & procurement operations in creating immediate value with existing products, and (ii) management of research & development activities in creating new products to generate value in the future.The managerial decisions in both daily operations as well as innovation activities are characterized by different sorts of uncertainty (largely due to bounded rationality) and by inefficiencies, non-optimal allocations, and market failures that are (partly) caused by those uncertainties.A natural starting point for policy makers then is to seek to reduce uncertainties, repair market failures, improve timing and technological focus decisions, and increase efficiencies through innovation and production system design.Ironically, policy makers too suffer of bounded rationality and their interventions enhance the economy non-optimally (if enhanced, in the first place) and, moreover, by doing so, government thereby becomes an integral part of the economic system, distorting the very functioning of Schumpeterian mechanisms in place.Now then, there is the formidable challenge for the policy maker to, firstly, fathom the complexity of the economic system (including its own role) and, secondly, devise and deploy instruments that enhance the system in desired ways.Despite the existence of advanced system dynamics models and formal, equation-based models, we can actually only think of one proper research model casu quo policy evaluation tool, namely: agent-based modeling (ABMing).In our own research, we use neo-Schumpeterian agent-based models (ABMs) in two ways.Firstly, for abductive inquiries of possible mechanisms or behavior responsible for a particular (simulation) outcome.Concretely, we use an ABM as a research instrument to discover and formulate hypotheses on the behavior of real-world agents that render particular empirical realities (cf.Axelrod 2007;Brenner and Werker 2007).As such, ABMs can be used for economic theorizing.Secondly, we use conceptually encompassing and empirically calibrated AMBs for policy experimentation and evaluation.Although ABMs may be classified on the degree of abstraction applied, the trump card of ABM is that, unlike the other approaches mentioned, that (many conceptualizations of) the real world can be modeled largely unabridged and, potentially, calibrated to empirically data (cf.Boero and Squazzoni 2005).With that, such an in silico virtual world offers unprecedented liberties for policy evaluation and experimentation.However, apart from touting the merits, benefits, and promises of using ABMs for economic theorizing and/or policy making, this paper will also elaborate on the pitfalls, disadvantages, and challenges.Given the big impact of policies that may come out of policy evaluation exercises, we particularly focus on ensuring validity of ABMs such that the policies found are suited to intervene in the complex real world and not just the ABM stylization of it.The structure of the paper is as follows.In Section 2, we provide an exposition of decision making in microeconomics (or, rather, how it was modeled, historically, and which normative recommendation came out of those models) and how firms struggle with elementary uncertainties therein.In Section 3, we elaborate on innovation activities and how firms struggle with uncertainties inherent to technology search, neo-Schumpeterian competition, and the present day organization in innovation systems.We also discuss how policy makers may repair market failures, reduce the uncertainties, and enhance the efficiencies of the economy at hand.In Section 4, we present agent-based modeling as a policy making tool and discuss how it meets both the requirements of the policy maker, while, at the same time, the features of the economic system at hand (notably the inherent uncertainties in operations and innovation decisions).In Section 5, we discuss the challenges of using agent-based models for policy making purposes.In Section 6, we reflect on the value of agent-based modeling as a tool in policy making in (innovation) economics under uncertainty and whether it is suitable for theorizing.

Decision making in microeconomics under uncertainty
In the third quarter of last century, a lofty cohort of young researchers (among them several future Nobel laureates) sought to rationalize decision making, in general.Operational methods for management sciences and operations research were soon developed and applied in a wide range of activities.With the fanning out of operational decision methods in the late 1950s and 1960s into a variety of other fields, repeated intellectual endeavors revealed that application in fields like microeconomics and industrial organization required strong assumptions.Sequential decision making in an economic system of real-world complexity showed to be cumbersome because, firstly, there are various sources of uncertainty impossible to take into account quantitatively, and, secondly, agents cannot anticipate the decisions of other economic agents because these decisions become nonoptimal and more or less risky due to these uncertainties.To appreciate the dimensions of decisions under uncertainty for the type of economic systems we study, we provide a brief overview.

Sequential decision method
Arguably urged by the operational challenges of full-scale warfare during WW II, governmental and military initiatives spurred the development of operational decision methods in the fields of management science and operations research.This led inclusion of sequentiality in decision methods, with the sequential statistical tests of Abraham Wald (1945Wald ( , 1950) ) and recursive adjustment rules of Holt, Modigliani & Simon (see e.g.their work in 1955) as precursors.Richard Bellman together with several of his colleagues consolidated sequential decision making in the general dynamic programming method (cf.Bellman, 1953;1954;1957).Dynamic programming is based on the elegant principle of optimality: an optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decisions (Bellman, 1957, p.83).Note that, while the commonly used dynamic programming method requires discrete periods and a numerable state space, the discreteness is not required for the principle of optimality to hold; Lev Pontryagin showed that maximization problems in continuous time and with continuous state variables can be broken down into multiple, consecutive problems of lower dimension, much like the principle of optimality.The ideas behind sequential decision making fanned out into fields like microeconomics and industrial organization, in which multiple economic agents face repeated allocation and pricing decisions.

Stochastic uncertainty
Dynamic programming can cope with uncertainty in the state variables (and so does control theory, its continuous counterpart), at least, in as far as that uncertainty is of a narrow, stochastic nature.For one, Jacob Marschak sought ways to quantify uncertainty in utility and how to cope with uncertainty in sequential decision problems (cf.Arrow, Harris & Marschak, 1951).However, even stochasticity of uncertainty may already inhibit optimal sequential decisions.After all, if a particular decision limits (viable) subsequent options (as e.g. with purchasing assets used to fulfill demand in multiple periods), the decision taken may then be optimal given the probability distribution over states when these are still all possible to occur, but is generally non-optimal once a transition to one particular state has occurred (cf.Arrow, 1959).In a microeconomic setting, complete certainty would allow firms to oversee future events and allocate resources perfectly at marginal rates, which would render no surplus at all.So, uncertainty is required for profit to exist (Hicks, 1931) and the cause for the existence of liquidity (Radner, 1968).In microeconomics, uncertainty is taken to be the inability to predict demand and for the competitive firm to predict price.Richard Nelson (1961) aptly argues: "For there to be price uncertainty, demand must vary, and in addition the firm's ability to explain and predict the variation must be less than perfect".As such, through reductio ad absurdum, we see that the perfect rationality axiom in the general equilibrium theory is irreconcilable with stochastic uncertainty in price or demand.

Information uncertainty
A more general notion of uncertainty in sequential decision making is the lack of information on the options (now or in the future) or on the probability distributions of outcomes of actions.Uncertainty in this sense resembles the notions of Knight (1921) and may be popularly phrased as 'unknown unknowns'.Information on options or the probability distribution on effects of actions may possibly become available only over time.Decision makers are then faced with the problem that an action taken now may later appear to be a poor choice because it limits the options or causes undesired, irreversible effects.Under such information uncertainty, "good current actions may be those which permit good later responses" (Marschak & Nelson, 1962), i.e. a certain degree of flexibility in actions later may be preferred to be able to act upon new information when it becomes available.The aforementioned sequential decision methods cannot readily cope with action and state sets that are only partially known or expanding or shrinking over time, nor with only partially known or evolving transition probabilities.Off the shelf, dynamic programming does work, unabridged, with a transient, non-ergodic transition matrix and hence (a narrow form of) dynamics, yet it does require the state space and transition matrix to be constant.We note, moreover, that the sort of information mentioned above may become available only after specific, deliberate actions.Ying (1967) extends the dynamic programming framework with agents that can chose actions to update a possibly imperfect outcome probability matrix.However, actual optimization of activities requires the agent to know or be able to compute the value of that update.We stress that, generally, the economic value of such additional information can only be determined after make expenses on (meta-level) actions to acquire that information.

Behavioral uncertainty, risk tolerances and heterogeneity
Only whenever an economic agent has unlimited computational capacity, has perfect information, and is perfectly rational, it can rationally weigh all factors, devise a perfect contract spanning all future decisions, and thus arrive at an optimal choice (cf.Radner, 1968).However, "[..] rational behavior is a set of propositions that can be regarded either as idealized approximations to the actual behavior of men or as recommendations to be followed" (Marschak, 1950).It is a "normative model of an idealized decision maker, not a description of the behavior of real people" (Tversky & Kahneman, 1986).Simon (1955) stresses that humans cannot decide optimally because they are -what was later coinedboundedly rational: they may perceive only part of the options, may have either flawed information on which outcomes will actually occur (and with which probabilities), and may not have a strict preference for outcomes (so, struggle with picking one of the options).Indeed, individuals do not maximize expected utility (Machina, 1989).These "deviations of actual behavior from the normative model are too widespread to be ignored, too systematic to be dismissed as random error, and too fundamental to be accommodated by relaxing the normative system" (Tversky & Kahneman, 1986).Under (stochastic) uncertainty and limited predictive abilities, behavior of individuals may differ, since they have (different) tolerances for risks (cf.Knight, 1921;Arrow, 1959).In microeconomics, the risk preference of a firm affects both the quantity produced and price demanded (Baron, 1970), such that the heterogeneity of economic agents inhibits aggregation of decisions into those of a "representative" agent.

Interaction & problem structure uncertainty
The implications of bounded rationality are particularly substantial when it comes to making decisions in a system featuring multiple agents.Whenever economic agents are perfectly rational and possess complete information, agents cope with (stochastic) uncertainty due to interaction of their decisions (e.g. through the prices) by following the stochastic game theory model (Shapley, 1953).In this, agents know which strategy to follow (i.e. which actions to take) to maximize the expected payoff given the strategies of other agents.Given the heterogeneity in information, computational capabilities, and risk preferences of agents, there is uncertainty on the decision behavior of other agents.This inhibits rational sequential decision making; the decisions of agents may change the underlying structure of the decision problem (viability of certain options, the probability of certain transitions, etc.) and the decisions of other agents cannot be anticipated.

In sum: uncertainty in operational decisions
We thus find that decision making in operational microeconomics is plagued by four inevitable sources of uncertainty, each with different consequences.Firstly, stochasticity (having a known probability distribution), due to which past decisions may still well prove to be non-optimal in future states.Secondly, a lack of information, calling for postponement of decisions or picking options that leave a degree of flexibility in later decisions.Thirdly, a limited understanding and limited computing capacity, due to which -even if all relevant information is present-agents may make non-optimal decisions.Fourthly, given that economic agents are heterogeneous (not only in their risk preferences, but also in the information available), their computational power, their objective function, etc., agents can anticipate behavior of other agents only to a limited extent, and their decisions may structurally alter the decision problem for other economic agents.In the next section, we show that not only does the economic system under study feature these sources of uncertainty, but also that the very presence of that uncertainty triggers perpetual structural change, further exacerbating the problems of managing (partaking in) the system.

Decision making in innovation economics
In the 50s of last century, economists became keenly aware of the significance of technological change for economic growth.Fabricant (1954) observed that about 90 percent of the increase in output per capita over the years 1871 -1951 is to be attributed to technical progress.Solow (1957) observed that just under 90 percent of the doubling in the gross output per man-hour over the years 1909 -1949 is to be attributed to technical change, and only the remainder to an increase of capital.However, despite the significance of and the piqued interest in, there is yet a limited understanding of how technological change comes about, how entrepreneurs could or should direct research and development activities, and which role the public sector could or should play.Here, we elaborate on making decisions in research and development activities over time and the role of existential and technological uncertainty as well as -by bounded rationality induced-networking and dynamic inefficiencies.We do not provide an exhaustive list of types and sources of uncertainty in innovation processes (for an extensive literature survey, see Jalonen, 2011).We end with a discussion of the various ways in which policy makers could improve the efficiencies in innovation processes, 'repair' the market for technological research, and take away somewhat of the uncertainties.

Existential uncertainty
In the neo-Schumpeterian perspective, a capitalist economy is engaged in a self-propelled dynamics due to three mechanisms: (i) if (potential) profit margins in a particular (non-existing or insufficiently competitive) market are sufficiently high and entry costs can be recouped, entrepreneurs enter to reap profit opportunities, (ii) unfettered price competition among entrepreneurs erodes profit margins of existing products and increases the efficiency of production processes, and (iii) entrepreneurs seek to develop (breakthrough) inventions to restore mono-/oligopolistic profit margins or serve niche markets in which customers have a higher willingness-to-pay.Firms in a capitalist economy seek to technologically outrace one another, yet due to bounded rationality, they are (i) technologically uncertain of how and when to make better products, and (ii) uncertain of what products are (going to be) in demand.Firms are therefore forced to proactively decide on what to make and face the consequences later (cf.Knight, 1921).As we have seen in the previous section, firms have different characteristics (thereby different goals), do not know all options or are unable to perform certain actions, and may moreover not know how to realize such options with certainty.Given different probabilities of survival on the market, there is a natural selection of different types of firms (Alchian, 1950) and (thereby) particular routines of firms (Nelson & Winter, 1982).

Technological uncertainty
Due to the unfamiliarity of inventors with particular technological knowledge and the high rate of failure of combinations of such knowledge, the course of technology search is highly uncertain (Fleming, 2001).In operational models of technology search, a firm constructs an innovation by combining technological 'elements', thereby searching for new combinations that exceed the utility/ fitness of the yet best known combination (see e.g.Frenken, 2006;Gilbert, Ahrweiler & Pyka, 2001).In these models, firms have no a priori information on whether particular combinations of elements have higher utility/ fitness than other combinations or not.This reflects how firms act uninformed, are not able to determine the outcome ex ante, and how the outcome of particular actions is uncertain (cf.Dosi, 1988;Metcalfe, 1994).Due to this high technological uncertainty, there are inherently uncertain returns on investment in R&D.However, the outcome of research and development activities is not completely haphazard.Arrow (1962) argues that the initial a priori probability distribution of the true "state of nature" (e.g. the firm's understanding of the technological principles or the feasibility of combinations of capabilities) is relatively flat.With each successive research step, the a posteriori distribution becomes more defined.So, researchers make the best possible choice, given the capabilities and action options at their disposal, and given the information about the future and the consequences possible actions.In line with this, we argue that researchers enhance their understanding of the operational rationales of the technology studied, construct a functional and operational decomposition of a product to be developed, and use this to further guide research efforts.Of course, the risk of this adaptive, pathdependent search is that researchers may become locked-in in researching a particular (infeasible or subpar) operational model.
Due to the technological uncertainty, there is a market failure for research and development.Arrow (1962) observed that inventing new products relies on the production and use of (new) technological knowledge, but firms are bound to underinvest in the production of that technological knowledge for various reasons.Firstly, technology search is risky and there are uncertain returns on investment in R&D.Secondly, firms struggle to recoup R&D expenses due to the limited appropriability.After all, competitors can freely access the technological knowledge once produced, particularly so when this knowledge is embodied in the product.Any legal system (e.g.patent protection) falls short to protect how the knowledge is used in circumvention or extension.This is particularly true for basic research, as entrepreneurs in a wide variety of industries can and do actually use the scientific knowledge in their inventions.As particularly basic research thus generates a future stream of benefits that accrues to society, the stepping in of government to repair this market failure is justified (Nelson, 1959;Arrow, 1962).That said, although a market failure for publicly available basic research is plausible, firms may still invest in specific, applied research as reaping spillovers requires specific absorptive capacities that competitor may not have (cf. Cohen & Levinthal, 1989).

Networking efficiency
The present-day understanding of innovation processes is that research & development as well as commercialization of new products, processes, services, business models, etc. do take place in systems of innovation (for an early proponent, see Metcalfe, 1994).Regardless of whether an innovation system is national, regional, or sectoral, it features networks of firms exchanging and creating knowledge, possibly facilitated by governmental institutes like an exchange platform or competence center.Given the vertical specialization of firms on knowledge in their domain of application, it is crucial for any firm to seek (i) access to and synergistic cross-fertilize with complementary knowledge possessed by other economic agents (Grant & Baden-Fuller, 2004), and (ii) cooperate in redesign of products beyond the current technological interfaces (cf.Frenken, 2006).As such, arguably, networking capabilities are indispensable for individual firms.Note that, since researchers, follow their limited understanding and (imperfect/ partial) functional and operational decomposition of a product to be developed, innovation networks (and even innovation systems) can be organized around infeasible or inferior product technology.
The distribution of technologically related knowledge over multiple agents, possibly spread out geographically (e.g. in another country) or organizationally (e.g. a high shortest path length), etc. substantially hampers innovative capabilities.Although evolutionary forces may enhance the organizational properties of innovation systems that survive, there may very well be both static as well as dynamic inefficiencies and inadequacies in existing innovation systems in the form of (i) the absence of firms with particular capabilities/ knowledge, (ii) the lack of ties between particular firms or poor communication across existing ties, or rather (iii) the existence of too strong ties or asymmetries in power causing a lock-in in existing designs (cf. Boschma, 2012;Tödtling & Trippl, 2005;Pyka, 2015).On top of these three sources of innovation system failures, firms may have a lack of information on whether particular knowledge is present somewhere or not, causing these firms to search for this information (possibly in vain) or duplicate research that has already been done by another firm.

Dynamic inefficiencies
From a technological perspective, firms cycle between a stage of inventive activities to open up a new market or to leapfrog existing products in an existing market, and a stage of incremental product innovation and enhancing production efficiency, where stage transitions are punctuated by breakthrough inventions and concentration and followed by a swarm-in and a shake-out of entrepreneurs and production houses respectively (cf.Utterback & Abernathy, 1975;Anderson & Tushman, 1990;Malerba, 2006).If firms are to survive the entries into and shake-outs over multiple product life-cycles, they have to strike a balance between exploiting (and incrementally innovating) the current product range, as well as exploring, absorbing, and developing radically new products (Cyert & March, 1963;Tushman & O'Reilly, 1996).Due to imperfect assessment of the a priori distribution of technological opportunities and profit prospects, firms risk inopportune changes in research focus.Indeed, firms need to strike a balance when to explore (through networking and technology search) and when to exploit (through incremental improvements and plain microeconomic everyday business).Evolutionary inefficiencies arise due to bounded rationality in deciding when and how much to widen or narrow the technological focus.If there is a bias to application and the focus narrows too soon or widens up too late, firms may be exploiting technology with poor profit prospects (explorative inefficiency).If the focus narrows too late or widens too soon, research funds are squandered on infeasible technologies (exploitative inefficiency) (Pyka, 2015).

Opportunities for policy makers
While the means to overcome the operational uncertainties and market failures described before are arguably limited, there are various ways for a policy maker to enhance dynamic efficiency of the innovation system at hand.Policy intervention is to resolve the above mentioned market failures, improve explorative and exploitative efficiency, and reduce uncertainties.We propose the following interventions.Firstly, repairing the market failure for fundamental research and improving appropriability of applied research.Given the importance for a wide variety of fields, government should financially support fundamental scientific research and research into general purpose technologies.Government should protect intellectual property in applications and ensure redistribution of economic value of spillovers.Secondly, facilitating the access to, recombination & diffusion of, as well as the application of knowledge.In the early phase of the research process, the efficiency of exploration is to be enhanced by stimulating a broad range of projects or decentralized search by a range of firms.To update the common understanding of the "state of nature" and the a priori probability distribution, firms and research institutes should be stimulated to share information on technological feasibilities.Related to this is that government should stimulate valorization of research in applications when technologically opportune.Thirdly, regulating research directions and applications.Arguably, policy makers should, in general, refrain from convoluting the probability distribution of opportunities, because they too suffer a limited understanding of technology (or even more than technology experts working in industry).That said, given their involvement in research activities of multiple firms, (public) knowledge platforms and competence centers may well have a sharper a priori distribution (or obtain one by delegating mutually exclusive research directions to a range of firms).Fourthly, managing the population dynamics.Given the importance of decentralized exploration, government should stimulate startups and entry of entrepreneurs.Moreover, government should combat anticompetitive behavior and lower entry barriers.Fifthly, building networks.Governmental or industry institutions may (i) establish links between firms, (ii) become a research agent seeking to tackle particular technological issues, (iii) become a knowledge sharing hub, (iv) stimulate entry of particular (types of) economic agents (at particular places) in the network, and (v) stimulate exploration of new product architectures with other than current network partners.Finally, safeguarding societal interests and curbing adverse effects of capitalism.Fierce price competition in a free market invites externalization of environmental costs, causes concentration of capital & social inequalities, features tragedies of the commons (e.g.overproduction, overexploitation), inefficiencies in R&D (e.g.due to duplication) and production, etc. Particularly the inefficiencies and tragedies can be address by information sharing, while internalization of costs can be ensured through regulation, laws and regulations, certificates, etc.
We see that policies thus target the design and inner workings of the innovation & production system (e.g.network and population management, establishing institutes), as well as explicitly giving direction to research and production activities both re-and proactively (e.g.financing particular research, stimulating startups in certain sectors, regulating waste streams, certification marks), etc.In general, present-day policy makers employ a policy mix with a handpicked selection of existing or ex novo created instruments tailored to the context in which they are applied (cf.Borras & Edquist, 2013).The policies and thereby instruments required depend strongly on the particularities of and the development patterns in the regional innovation system (Tödtling & Trippl, 2005;Camagni & Capello, 2013).For a framework to classify different innovation policy instruments and a range of examples, see Borras & Edquist (2013).

Policy evaluation with agent-based models
Even up to today, it is not uncommon that policies are ad hoc, based on individual case studies, and/or without unambiguous scientific support.That said, policy makers rather do not experiment with interventions in the real world if there may be substantial irreversible adverse effects.Economic models (of various sorts) allow policy makers to evaluate the effects of their policy interventions before actually implementing them in the real world.Over the last couple of decades, not only have the business models, modes of organization of production and innovation activities, and -accordinglypolicies changed drastically, so have the economic models to determine the policy interventions.In his 1957 paper, Solow writes: "As long as we insist on practicing macro-economics we shall need aggregate relationships".After the burial of the representative agent and the birth of heterogeneous agents, such undue aggregation is no longer desirable.Moreover, with the advent of computational means to simulate large populations of heterogeneous, interacting agents, aggregation is also no longer required.We use ABMs in research to, firstly, abductively formulate hypotheses on the behavior of real-world agents as cause for empirical realities (cf.Axelrod, 2007;Brenner and Werker, 2007).Agent-based models allow studying dynamics under ill-defined behavioral rules (heuristics) coping with uncertainty about the future (cf.Lempert, 2002) and implementing elements specific for the context and locality.Moreover, the real economic system can be implemented largely disaggregated and unabridged, as well as calibrated to empirical data (cf.Boero and Squazzoni, 2005).Secondly, we (advertise the) use of such empirically calibrated ABMs to evaluate the effects of particular policy interventions.For the remainder of this section, we make the case that and explain how policy makers can use ABMs to their advantage.

Implementing the required elements
For an extensive discussion on what defines a neo-Schumpeterian agent-based model and what are the essential components and different considerations in designing an agent-based model, the reader is referred to Pyka & Fagiolo (2007).For the experienced ABM developer, it is obvious that the elements mentioned in the previous two sections can be implemented without much ado.In fact, the field of neo-Schumpeterian modeling started with Nelson & Winter (1982) implementing an ABM with technological competition, entry and exit, heterogeneity, autonomous decisions, and technology search.Others have implemented ABMs for the formation of supply chains and knowledge-based innovation networks (Gilbert et al., 2011), research conglomerates (Scholz et al., 2010), regional innovation systems (Korber et al., 2009), as well as industry-university research networks (Ahrweiler et al., 2011;Triulzi & Pyka, 2011) under operational, technological and existential uncertainties, as well as network inefficiencies.For a further discussion of issues of implementation, the reader is referred to the subsection on challenges of operationalization.

Disaggregated, unabridged modeling for particular cases
One of the most obvious advantages of using ABMs for a virtual world is that the researcher-modeler is not forced to conceptually 'over-abstract' the real-world system that is studied.In general, the researcher-modeler breaks down the real-world system in, on the one hand, definitions of agents' behavioral rules 1 and, on the other hand, definitions of the elements in the environment (e.g.resources, infrastructure, geographical space, communication medium).Given that the agent-based model is thus a close representation of an actual real-world system, much more than equation-based aggregations thereof, policy makers can easily join in on defining the behavior of agents (e.g.collaboration heuristics, product-market strategy) as well as the economic setting in which these agents operate (e.g. the product market served, the patent system in place, the institutional framework).In the 'translation' from the real-world elements to the model operationalizations, the modeler does not need to bother about tractability of a solution, about the formal definition, about the dimensionality of the system as a whole, etc.Moreover, given the straight-forward associations of artificial to real-world elements, researchers and policy makers can interpret simulation outcomes straightforward and infer on the mechanisms at play.An additional reason to develop and use encompassing and realistically complex2 models is that policy interventions should not ignore particularities of the actual economic system sought to intervene in.Different innovation systems in the same industry may differ substantially on dimensions and internal dynamics, and, evidently, policy interventions in these key dimensions are then likely to have different effects, so may need to be differed for these systems.On top of the individual differences from one economic system to the other, the various economic systems are coupled through a shared market, links in value networks, etc., so it may be undesirable to uncouple economic systems and model just one, in isolation.Obviously, to properly evaluate the effects of policy interventions, not only should the agent-based model reflect the real-world in sufficient detail and complexity, so should the simulated policy instruments.
That said, also in formulating ABMs, the modeler-researcher has to strike a balance between a simple and a descriptive model by assuming away details deemed irrelevant, insignificant, or distracting in answering particular research questions (cf.Deichsel & Pyka, 2009).Particularly if the system exhibits high-dimensional and non-linear double dynamics (concepts that are discussed in detail below), the modeler-researcher best limits the model to the essential parts so as to be able to ascertain validity and interpret simulation results.Moreover, also in formulating an ABM it may be commendable to extend an existing model with established validity (cf.Pyka & Fagiolo, 2007).

Transparency and "God mode"
Another major advantage of using computer simulation is that the programmer can provide access to all the data of agents and can log each step in every procedure.As such, the virtual world is perfectly transparent.The modeler-researchers and policy makers can record and study events, flows of knowledge and resources, and decisions and interactions of agents in great detail.ABM simulation thus also allows studying in detail the connection of micro-level flows and decisions, non-linear interactions of agents at the meso-level, the translation of micro-and meso-activities into macro-level indicators and phenomena, and the subsequent feedback into micro-level behavior.As, on top of merely being able to conduct a black-box study of the effects of policy interventions, ABMs allow tracing structural interactions, collecting rich data, studying (non-linear) interaction of decisions, etc.Moreover, in the real world, not only are data and decision processes largely unobservable for policy makers, they also can -generally-not fully control the operational behavior of agents or the environment in which they operate.In contrast, in ABMs, modeler-researchers and policy makers have "God mode" power to change agent behavior instantaneously.In fact, modelers can even use the observable and controllable elements explicitly in agent behavior heuristics.On top of transparency and control over behavior, policy makers can use ABMs to study the immediate and emerging effects of interventions, initializations of the virtual world, and external events on the otherwise autonomously running virtual world.Such interventions may be 'ad hoc' experiments or rather a predefined procedure in the code that endogenously responds to the state of the artificial world.The experimental variables in such an intervention may be either funding of particular research activities of individual agents, the presence of particular knowledge sharing platforms, regulations on R&D network formation, etc.The researcher may also experiment with the initialization of the simulation run, e.g.different initial social or organizational network, a different distribution of technological opportunities, different collaboration propensities, etc. ABMs can be and also are used to study the effects of external events like disruptions in supply (e.g. of financial means, or of raw input material), discontinuation of social or organizational links (e.g.due to political decisions), or shifts in demand (e.g.due to a natural catastrophe, or a ban), etc.

Harnessing enthusiasm
While these advantages of using ABMs for policy evaluation (e.g.unabridged modeling, heterogeneity, particularity, transparency, God mode) may get policy makers interested, one still needs to win them over for a particular project by having them accept the limitations (discussed in the next section).To harness the enthusiasm of policy makers, modeler-researchers need to ensure that they subscribe to (possibly limiting) operational definitions and support the (possibly limited) research questions that can thus be answered.To do so, it is of paramount importance that policy makers are involved in the actual design and progressive adorning of the ABM.Whenever policy makers provide modeling input, they can interpret, appreciate, and use the output.Do note that this is in fact a serious challenge, because the modeler-researchers should then be involved early on in the policy research process and should also have policy makers dedicate time and resources in assisting in the actual modeling.

Challenges of using agent-based models
Occasionally, policy makers are somewhat skeptical about the value of ABM-based policy recommendations.Firstly, since the underlying technical implementation of the model is largely a black box for the policy makers, they contest the internal and external validity of outcomes.Secondly, simulation outcomes are poorly understood and misinterpreted because of the presence of temporal disturbances like the simulation onset behavior (due to ABM initialization) and learning (due to agent population evolution).Thirdly, there are disputes over the agent heuristics, and policy makers (may) argue that 'in reality, agents do it differently'.Fourthly, it is argued that the ABM is not specific or generic enough for the case studied.
Arguably, part of the skepsis of policy makers can be obviated by involving them already in formulating model elements.Calibration of model parameters and initial system conditions (e.g.firm profiles, network topology, market concentration) may instill yet more confidence.However, that said, a possible lack of validity is (and should also be) a major concern to the modeler.The challenges of validation and calibration of ABMs for policy evaluation are discussed first.A comment heard occasionally is that the validity of simulation outcomes is hard to gauge given conceptual model choices, the operational definitions, possible faults in the code, and particularly the non-linear interactions of various software modules.We discuss several issues with validity3 and ways to improve it.This is followed by a discussion on the comment that 'real agents do it differently'.In this, the assumptions on and operationalizations of agent behavior ubiquitous in neoclassical economics and management science are pitted against the assumptions and operationalizations in the prevailing paradigm in the field of heterodox ABMs.Discussed last are the temporal disturbances in simulation outcomes and how to account for and prevent those.

Validation & calibration
To bolster external validity of the model and provide tailored, possibly quantitative policy advice, it is commendable to use empirical data to calibrate the ABMs.Not only can macro-level parameters be estimated from data, but so can the composition of the agent population, the properties of the individual agents and their relationships, as well as particularities of the environment (e.g.resources, infrastructure, geography).

Pitfalls of face validity, storytelling, curse of double dynamics, artifacts
Customarily, ABMs produce graphs or tables of some key variables over time.The ABM simulation is said to have face validity if a (panel of) theme expert(s) qualifies these simulation outcomes to be in check with the real-world phenomena studied.During ABM development, however, the modeler itself assesses face validity for 'reasonable' values for all parameters and experimental variables.Not uncommonly, trial simulation runs produce surprising results, e.g. a counter-intuitive or weak or rather strong effect of a particular parameter or experimental variable.It is then left to the developermodeler to assure that the outcomes are not caused by faulty code or inadequate operational definitions of concepts.In assessing face validity, an obvious pitfall is that of 'storytelling', i.e. fabricating a conceptual story that confirms the outcomes but does not properly reflect the operational chain of events in the model.Such a story may have the researcher erroneously certify face validity.While face validity and correctness of underlying code may be easily established for simple, linear systems, validity of simulation outcomes may become hard to gauge whenever there are non-linearly interacting components and double dynamics.Operationally and structurally simple models with interacting components may already give rise to intricate dynamic behavior of which it is hard to establish (face) validity, think e.g. of a double pendulum system.Given that actual causes of simulation output is sometimes hard to trace in the overwhelming complexity of non-linear interactions in ABMs, the danger of storytelling is lurking.Similarly, interesting results may be 'artifacts', i.e. outcomes of a programmatically correct, unsuspected error in the operationalization.Clearly, if the researcher-modeler engages in 'storytelling', artificial outcomes are less likely to be detected.

Internal validity
Unsurprisingly, face validity of simulation output is the first but also one of the weakest tests that ABMs should pass.A stronger validity concept is that of internal validity.In the context of ABM development, internal validity is taken here to be that there is no intellectual ambiguity of the causal chain through which input parameter settings cause the simulated output.Even if the outcomes are in line with expectations, the developer-modeler has the (scientific) responsibility to ensure correctness of the code and operational soundness of the model.We discuss three practices to assist in developing internally valid models.Firstly, to ensure sufficient intellectual scrutiny of the causal chain, developer-modelers are encouraged to provide functional design specifications (FDS) and a program technical description (PTD), just as professionals in the software industry are required to do.In the FDS, the developer lays down the sequences and interdependencies of operations, here, in the model of the economic system.Note that the software design and actual real-world system can and may differ.In fact, part of the design decisions may stem from what is considered 'good code practices' rather than conceptual model considerations.Moreover, developer-modelers should provide a PTD in the form of pseudocode or flow-charts, to communicate the operations internal to procedures.Ideally, both the FDS and PTD should be contained in reports presenting ABM results, or be available upon request.Creation and maintenance of these documents does not only force the developers themselves to think over interactions and effects of certain operations.It also convinces other researchers and policy makers of the causal chain and internal validity, and allows other developer-modelers to implement the model in a language of their choice, reproduce the research findings, and extend the model for different purposes.Secondly, the developer-modeler should assert bug-free code, which -as most developers in economics are not trained computer scientists-generally boils down to testing code to great lengths to make sure code does what it is supposed to do.In systems with a few variables, the developer can easily step through the code or study debugging logs.In case of complex systems with non-linearly interacting components, this task is daunting, although not impossible.We recommend to -at least initially-study the non-linear interactions in a controlled setting, e.g. a system with a limited number of agents, a system with agents that have some (interacting) heuristics disabled.Thirdly, when internal functioning of individual procedures/ modules has been approved, developermodelers should ascertain that there are no unintended temporal interferences across procedures/ modules.Many economic ABMs are discrete time simulations in which each agent gets to conduct its activities each period.Given that agent heuristics respond to and operate upon the same macro-level variables (e.g. the product market, product prices) and upon the states and activities of each other, there is feedforward of actions of agents into actions of agents executed later (e.g. a product launch by the first agent may affect the decision to launch products of other agents).An example of flawed intra-periodic interaction is discussed in Vermeulen (2015).

External validity
In the context of ABM development, external validity requires the ABM to reproduce a wide range of general features of the real-world system being modeled.Where face validity is a weak test that a model generates 'reasonable' outcomes for 'regular' parameter settings, external validity requires the software to produce outcomes that meet expert expectations, (stylized) facts, etc. under wildly varying settings.Ideally, the researcher can conduct cross-validation tests between different models, test similarity of an empirical dataset and the generated dataset, etc. Conveniently, external validity tests can also be conducted at the level of ABM modules.As these modules generally have one-to-one associations with real-world systems, modelers can conduct unit tests of separate modules.Concretely, developer-modelers can devise tests to assure that a module reproduces particular economic 'stylized facts'.For instance, consider an ABM in which firms sell substitutable products to consumers and adjust prices to increase profits.A unit test would be to make demand inelastic and see whether there is product price explosion when the number of firms is kept equal to one (i.e. it is a monopoly) or whether there is profit margin erosion when the number of firms is set and kept high (i.e.there is unfettered competition).Arguably, whenever thorough unit tests of a module indicate external validity, this is also an indication of the internal validity.

Iterative model development
The ultimate target for the programmer is to develop an ABM with 'hotspots' in the parameter landscape (for controlled variables) for which experimental variables produce simulation output that is (i) in check with (stylized) facts and is (ii) scientifically interesting or policy relevant (depending on the application).However, sometimes to the dismay of developer-modelers, a typical ABM development process is iterative.Even despite a proper design and technical specification stage, the first simulation runs may produce unexpected output.It is not uncommon that plots of certain key variables deviate from expectations, that the system responds in surprising ways to certain parameter changes, or that dynamics are extreme (e.g.exponential or rather zero growth, or featuring a lot of noise).In the exploratory runs of a simulation model, the researchers should look for such 'hotspots' in the parameter landscape for those parameters that are not calibrated or do not have clearly 'reasonable' values.Generally, this requires doing a coarse grained Monte Carlo study and then zooming in on possible hotspots to conduct a fine grained study to test the ABM simulation.If such hotspots cannot be found or simulation outcomes defy the stylized facts, it is then up to the developer to ascertain that the code is bug free.As explain before, this is done by analyzing the code, scrutinizing extensive event logs, and conducting unit tests.If code is bug free, the next step is to reconsider operationalizations in the ABM.It is not uncommon that limitations in the operationalization of realworld elements cause unexpected simulation outcomes (see the discussion on 'artifacts' above).Finally, if even the ABM's operationalization withstood scrutiny, the conceptual design forming the basis for the ABM may need revision.After each ABM redefinition and redevelopment, new parameter landscape searches and test rounds are needed, possibly leading to new redefinitions and redevelopments.Our experience is that, indeed, there are occasionally more iterations needed to adjust heuristics.In case of a high-dimensional ABM with double dynamics and second-order emergence, it may take an experienced modeler quite a number of working days to get acceptable and valid output.

Operationalization
Here, we discuss only those operationalization issues that directly relate to the uncertainty dimensions mentioned in this paper.For an in-depth discussion of all the building blocks of neo-Schumpeterian models, the reader is referred to Pyka & Fagiolo (2007).

Agent heuristics
As we have seen, real-world agents are boundedly rational and have to make decisions under various uncertainties, generally following rules-of-thumb that imperfectly processes limited information.In ABMs, agents follow 'heuristics', i.e. a set of if-then rules on state variables that specify the actions for an agent to take.In the operationalization of these heuristics, the researcher-modeler should distinguish the operational sophistication of heuristics as well as the extent to which the decision domain is understood and incorporated in these heuristics.Depending on the topic studied, heuristics may have different levels of sophistication and domain knowledge.In between the two extremes of rational processing of perfect information and completely random behavior, there are agent heuristics that cope in a more or less sophisticated way with a limited understanding of or limited information on the decision domain.Arguably, the researcher-modeler should include only those heuristics that are the main operations in the economic process under study.The researcher-modeler and the policy maker should discuss early on which actions to include and how to abstract rules-of-thumb into heuristics.

Technology search
Technology competition is central in neo-Schumpeterian models.Depending on the purpose of the model, technology exploration and exploitation are also operationalized.Exploration is generally modeled as 'search on a landscape', where the search heuristics reflect the limited information on and understanding of the 'state of nature'.Ideally, agents use past search findings to update the a priori probability of new search directions, but it is however common to simply take technology search to be a hill-climb or a random walk.Regretfully, despite the importance in neo-Schumpeterian models (particularly if they are used in policy evaluation), there is no definite landscape search model.Exploitation is modeled by having agents compete with the 'product' of their search activities on a consumer market.'Fundamental' models used for scientific inquiry commonly do not feature distinct operationalizations for knowledge on the one hand and products that are created with this knowledge on the other.In models on innovation networks, such as our own (cf.Gilbert et al., 2001;Vermeulen & Pyka, 2014a, 2014b), knowledge is exchanged between and (jointly) created by firm agents in one process, while this knowledge is used to create products for consumers in another process.Across the various innovation network models, though, the operationalization of 'knowledge', 'knowledge search', 'product, and 'production' differs, based on the application of the model, the researcher's conceptualizations, data available for calibration, etc.Given that, for computer implementations, knowledge needs to be encoded (in bytes), knowledge may well be treated as indivisible units that can be stored in a repository and transferred to other agents as if perfectly codified.In case the researcher-modeler focuses on industrial innovation, knowledge units may be technical concepts used in products, technological fields in which an agent is active, etc.In case of a focus on scientific discoveries, knowledge units may be papers, proved/ disproved hypotheses contained in it, etc. Knowledge search may be operationalized as a random walk or hill-climb from an existing unit of knowledge to another or as a procedure that takes a collection of knowledge units readily owned and returns a (possibly new) knowledge unit according to some probability distribution (cf.Gilbert et al., 2001) or fixed graph (cf. Taylor & Morone, 2010;Vermeulen & Pyka, 2014a, 2014b).Actual calibration of this knowledge graph can be done using patent data or historical invention trees, either creating hard-wired graphs or probability matrices.Researcher-modelers face a similar challenge when operationally defining and empirically calibrating the translation of 'knowledge' into 'products' and determining the features or qualities of the products.In case the researcher's focus is on industrial innovation, then empirical calibration may be done by historical analysis of inventions, underlying knowledge flows, and market value.In case the researcher's focus is on scientific discovery, the product could be papers and the knowledge units could be hypotheses/ concepts.

Emerging dynamics: temporal patterns, simulation onset, learning
Both in designing an ABM as well as interpreting simulation outcomes, the researcher-modeler and policy maker should bear in mind that the emerging temporal pattern generally takes only one of several possible forms, that there is onset dynamics, that evolutionary training of heuristics affects dynamics, and that these dynamic phenomena interact and thereby possibly render artifacts both in dynamics as well as emerging strategies.Firstly, researchers use ABMs to simulate a wide variety of processes, but in general the key output variables either converge to particular 'steady state' values, form temporal patterns (e.g.staircasing S-shapes, multi-dimensional cycles), or display continuous growth or decline.It may be challenging to find, tune and justify the model elements to reproduce or test the formation of temporal patterns, particularly in case of high-dimensional double dynamics.Secondly, there may be an onset phase in the simulation output that should not be mistaken for conceptually meaningful results.With the 'onset phase', we refer to the first periods during which the system dynamics is largely determined by the initial conditions (e.g.values of variables) rather than conditions that are common to or could emerge within the real-world system.An example is that right after the start, the consumer agents have yet to do their first purchase, firm agents have yet to find knowledge and produce products, differences in firm strategies have yet to reflect in products on the market and capital stock, network relationships have yet to form, (macro-level) exponentially smoothened variables driving agent behavior have yet to lose initial variability, etc. Ordinarily, the rule-of-thumb is to discard results before a certain period.When the initial ABM simulation state (in terms of capital stocks, knowledge bases, social relationships, etc.) can be calibrated to empirical data, there is no/ much less onset dynamics.Thirdly, there is population-level learning in the agents' strategies.With population-level learning, we refer to the evolutionary process driving improvement in the strategies and/or features of agents in the population.This evolutionary process is driven by (i) imitation (with slight involuntary mutations) by entrants of the strategies of successful agents, (ii) deselection of agents with strategies that are 'unsuccessful', e.g. in the form of bankruptcy.Fourthly, there are various interactions of these processes.First of all, researcher-modelers should be aware that the population of agent strategies needs time to converge.In the onset phase, there may be an abnormal diversity of strategies or there may still be 'inviable' strategies present, such that dynamics may be extraordinarily erratic.Second of all, there may be (coincidental) emergence of strategies that are merely artificial.A strategy that emerges in the onset phase as superior, may be 'nonsensical' in the post-onset phase but nonetheless persist, e.g. because variation is too limited to escape the basin of attraction of that strategy.
In deliberating with policy makers on modeling choices, researcher-modelers need to explain which ingredients are required for the ABM to reproduce particular dynamics (e.g.successive product life-cycles, societal transitions).Moreover, in contrast to the common understanding that forecasts become less reliable the more into the future, simulation results may become more reliable (or rather, stable) over time with the vanishing of variation caused by the onset dynamics and learning effects.It is important to be aware of and also explain to a policy maker exactly these intricacies in the interpretation of simulation results.On top of that, it should be taken as a warning to researchermodelers that one should favor empirical calibration, should be reluctant to introduce learning in a great number of strategy variables, and should strive for simple models with limited temporal patterns to reproduce.

Reflection
In this paper, we have discussed the type of challenges that policy makers face when they seek to enhance growth in a capitalist economy.We have seen that firms suffer uncertainties, inefficiencies and market failures, both in everyday operations management generating immediate value from the current products as well as in innovation management to generate new products for future value.We have made a case for agent-based modeling as a policy evaluation tool because it (i) allows modeling the (necessary) real-world complexity with agent heuristics reflecting the bounded rationality of firms and inherent uncertainties they seek to deal with, and (ii) allows experimentation with and ex ante evaluation of effects of (a mix of) policy instruments that closely match(es) those sought to use in reality.On top of the practical value of using ABMs in policy making, ABMs are particularly suited for abductive studies of possible mechanisms rendering empirical realities and thereby for economic theorizing.Given the potentially big impact of policies derived from policy evaluation exercises, we have given the challenges of validation and operationalization due attention.On top of that, we have highlighted intricacies in interpreting simulation outcomes (e.g.due to onset behavior, learning, and predefined temporal patterns).Reflecting on the qualities of present-day ABMs and the intended use, we stress that modeling of technology (and the search for knowledge and translating that knowledge into product) needs more attention.That said, if technology is treated as just another factor kept equal ('ceteris paribus'), we may very well still study the effect on economic growth of policy interventions in population dynamics network topology, institutional support, etc.Finally, we remark that the historical outlook on sequential decision making revealed that neoclassical scholars concur with neo-Schumpeterian scholars in that economic agents suffer from stochastic, information, and technological uncertainties, thus have to resort to (probabilistic) heuristics, and generally act non-optimally in the light of these uncertainties.These two schools may very well be considered complements due to the type of models sought to apply (neoclassical economics better suited to study operational, microeconomic decision problems and neo-Schumpeterian economics better suited to study complexity and dynamics in innovation processes) rather than head-on substitutes.If anything, assumptions seem to be matched to the method sought to apply and type of (normative) recommendations sought to give rather than to reflect a heartfelt perception of human capabilities.