From OpenVCE

Jump to: navigation, search

PMESII stands for "Political, Military, Economic, Social, Infrastructure, and Information". It's possible that this collection of domains has different connotations in different communities, but it's known in the National Security Modeling community as a comprehensive set of spheres of human behavior from which the elements considered during modeling should, and perhaps must, be drawn.

It may also be considered a useful list of factors to be considered during crisis and emergency response.

This area of the VCE Wiki will ultimately host a series of short descriptive and evaluative essays describing a non-exhaustive set of PMESII modeling tools (and similar IT PMESII tools that aren't quite true modeling in a few cases) which the author had the opportunity to research in 2008-9. The work was done while the author worked as an intern at National Defense University's CTNSP, but not published or officially presented at any point during that time. NDU has cleared this unclassified work for further distribution.

A Poster display and link to this Wiki page is available in the Expo Pavilion on the VCE Region in Second Life: [1]. See image at [2]


The author would like to thank Bryce Nicholson, Lauren Lee, Dr. Michael Baranick, Dr. Eunice Santos, Al Sciaretta, and Dr. Hans Binnendjik. The author also thanks a long list of model developers for their cooperation and corrections. Additionally, Prof. Austin Tate's assistance in posting these reviews to is much appreciated.

Introduction / Precautionary Note

The following document contains a series of evaluations and descriptive summaries of recent and current PMESII tools. The author relied on a list of PMESII tools from the Naval Research Laboratory. This list is not exhaustive, nor does it include current every high-profile tool, but it includes many of them. Although much time was spent on the evaluations, they should still be considered cursory, not exhaustive or defining. These reviews relied on documentation, literature review and interviews; many theoretical and practical issues may remain to be discovered in a more in-depth VV&A process. Evaluation of PMESII tools is a difficult task in ideal circumstances, and the current information environment is not ideal. The author hopes to complete a future paper on the limitations of current evaluation methods.


This page on is free for anyone to edit, and I welcome feedback and even additional or new content. However, I please ask that you do not modify or remove any material on the current PMESII tools here without my explicit permission. If you wish to dispute any information or add your own, please use a new section, marked by its own second-level headline, that can be placed directly under my own section on the tool in question. Removal or replacement or my content here will itself be removed.


AGILE - Advanced Global Intelligence and Leadership Experiment

Behavior Modeled: AGILE is a modeling tool intended to assist in simulating national behavior across a spectrum of governmental influence – Political, Economic and Military. AGILE is not a model per se; it expects the user to create a model for each behavioral domain, consisting of the available actions in the domain, and how each action will affect world or sub-domain variables. Developers recommend using just one agent per domain, representing the total of all entities that perform a national function, because each action type must be unique to one agent. The AGILE user must model each agent’s goals in terms of world variable state, values about all actions for his nation, cause-effect beliefs (about action results, and “judgment” (interpretation of world state, and triggers for action). All agents in a nation vote on actions based on these beliefs.

Methodology: The AGILE world model updates in turns, with each nation making decisions, followed by a world state update from the results, and then a new decision stage. Each agent uses his beliefs about the world state to decide upon the need for action, and his action values and cause-effect beliefs to select the appropriate action. The ‘leader’ agent orders the actions by agent ‘influence’ and allows all agents to vote on each action in sequence. Agents use their values about an action and beliefs about its effect to determine their vote. This includes consideration of the expected results of previously approved actions. AGILE has no built-in variable relationships, thus effectively no preset theory or calculations about the results of any action.

Strengths: AGILE allows users to program an iterative initialization series of multiple “runs” within which two variables vary parametrically, and/or with any set of manual variations to any variables. Wide solution spaces can be explored with minimal extra effort. AGILE extends across PMESII domains and allows user modeling of all kinds of cognitive error and bias in the action / decision process, through its adjustible and highly granular agent belief values. Agent behavior is highly customizable. AGILE’s built-in relationships are claimed to be completely traceable, which seems plausible given their simplicity, and user-created aspects (the models) are also traceable.

Limitations: Some of the bounds of AGILE’s framework constrain realism and utility. The forced association of a single type of action with exactly one agent creates a highly abstract behavioral system. The effects of actions on world state variables can not be made to vary: all Military Incidents, Embargoes or whatever else have the same output. The requirement for collective national decision-making contrasts with the wide variety of institutional freedom often found in real life. Along with the constraints, there are predictable corresponding costs to the open-ended nature of the framework: The user must define a large number of variable states and relationships per agent per simulation, (often relating to real-world values for which empirical data is not available), and each model instantiation may vary greatly in methodology and performance. AGILE does not consider resource constraints. Nor does it allow for dynamic cognition – agent preferences and beliefs are fixed through the scenario.

File:AGILE Table.png


Gwyneth Lee & Jordan Willcox

Behavior Modeled: APOLLO is a software application that enables an analyst to build probabilistic predictions of a subject’s decision options in a real or imagined scenario, as well as the reactions that possible decisions may elicit from other actors or their environment. In using APOLLO, analysts cooperate with the developers to produce a graphical representation of the predictions, with probabilities explicitly stated. Dependent and interdependent variables appear in APOLLO as nodes linked by lines that represent causality between the nodes. The analyst builds Bayesian networks that integrate situational information with the subject’s personality and culture to provide a probabilistic prediction of the hypothesized actions of a subject. APOLLO requires the client analysts to not only model a set of probabilities for decision choices, but also a full psychological profile for the actors whose decisions are being evaluated. The Bayesian network is integrated with a tool that sorts through incoming all-source reports pulled by the analysts’ favorite search engines, ranks their relative salience to specific variables, and captures sourcing (the evidence) for later review by the user or others.1 Additionally, APOLLO assists the analyst in performing basic sensitivity analysis of the relationships between any two variables in the prediction schema.

Methodology: Bayesian modeling is based on conditional probabilities: if variables A and B are not independent, then the belief in A given that B is known is the conditional probability P (A/B) = P (A, B) / P (B). In APOLLO, Bayesian modeling follows three steps: (1) developing a probability model that incorporates existing knowledge about event probabilities, (2) evaluating model in respect to data and sensitivity of the conclusions to the assumptions (3) updating the model by adjusting probability according to observed data, as it becomes available. In the case of APOLLO, users are also asked to provide psychological profiles prior to establishing behavioral probabilities. The profiles are created according to developer-provided format derived from prominent psychology literature. Specifically, the format combines the “five-factor” psychological paradigm with a series of personality trait categories from the political psychology literature. The user must also decide, in each model instance, exactly which nodes on the prediction schema are dependent on which nodes of the personality profile, and the extent and the correlation of the dependency. Modification of the causal maps involves consideration of identifying conditional independencies, discerning the underlying links between concepts, distinguishing between direct and indirect relationships, and eliminating circular relations (adjacency matrices).

Strengths: In comparison to other Bayesian systems, APOLLO forces its users to consider a baseline psychological profile for its key actors, which might fractionally assist in achieving predictive consistency. APOLLO’s attached tools make it easy for analysts to gather and consider external data, although it does not require any specific data for any network. APOLLO does not seem likely to have great hardware requirements, and should be completely transparent in its methodology during each scenario created. APOLLO, unlike other Bayesian networks, allows parent-child branching of outcomes beyond binary.

Limitations: APOLLO relies on user-generated, acyclical causal maps (DAGs) that express judgments that certain actions will lead to particular outcomes with specific probabilities. APOLLO does not bind the causal relationships to a specific theory or structure, leaving them completely up to the user. No two APOLLO simulations will correlate the same outcomes from the same variables. APOLLO’s accuracy is likely to vary unpredictably. APOLLO requires greater effort in model design as desired granularity increases: for example, if every soldier’s behavior in a company is to be considered separately, that requires 200 discrete sub-networks to be modeled.

File:APOLLO Table.png

ISSM - Interim Semi-Static Stability Model

Behavior Modeled: ISSM tries to model the network of conditions and entity characteristics most relevant to its predefined goal / final child node – “Civil Stability and Durable Peace Exists” – over the territory of a governance institution, most often of one nation-state. The model is coarse-grained, in that its data input is of outcomes rather than actions. The ISSM main model does not directly consider, for example, the number of suicide attacks occurring in a province in one week, but instead might score the extent to which ‘citizens feel secure’ or ‘violent events are common’. However, it is claimed that an ISSM Preprocessor allows the user to create custom logic to connect such metrics to the main model. Documentation of this function was not reviewed by the author. ISSM mostly avoids establishing a single defined outcome for any one action taken by an entity participating in the situation modeled. It instead requests data on the collective outcome of the behavior of groups of participants, usually obtained through expert observation, recorded as Likert scales.

Methodology: ISSM is built on a codified set of presumed relationships between real or perceived conditions within the area studied. These relationships form a network with two subsections – one section describing the territory of interest, and one cataloging the outcomes of various external intervention actions. The first network subsection is based on the “Doing Windows” book written by professors at the Naval War College, following their research on important goals to be accomplished during peacekeeping operations (and thus for all missions beyond conventional warfare).

Strengths: ISSM has many strengths common to of “light” tools and Bayesian Net tools: specifically, it can capture broad areas with relatively light data requirements; its output encompasses a wide suite problem domains; it neither requires extensive training nor advanced hardware, and its methodology is easy to understand and very transparent / traceable. ISSM has some other specific advantages. First, it is available for free to any interested person or organization. Second, its basic causal network is not subject to user customization – a disadvantage for some but an advantage in other ways. The consistency of the interrelationships may allow for meaningful comparisons between ISSM modeled scenarios. The ISSM has also been used as the capstone model for a set of simulations, converting their results into higher level situational awareness measures of merit.

Limitations: ISSM is not a simulation, and does not react internally to behavior changes internal to the observed territory. In order for ISSM to react, these behavior changes must affect a ‘measurement’ node with enough force for the expert observer to record a change in node values. ISSM is vulnerable to the same potential limitations as surveys – including samples whose distribution is not the same as that of the target population. ISSM’s node network may or may not capture all of the truly important determinants of its final output node, and no clear method is proposed to differentiate between error resulting from incorrect data values, vs. error from incorrect relationships. However, the model does allow for correction of incorrect inputs, if this can be determined. The end-user cannot customize the basic network to correct perceived missing relationships. However, the ISSM Postprocessor does support custom logic, defined by the user for this purpose. Lastly, the accuracy of the data inputs themselves are hard or impossible to verify, due to their abstract nature. The author suggests that the precise values are not as important as the changes in reported values over time, which ISSM should clearly record.

File:ISSM Table.png

MANA - Map Aware Non-uniform Automata

Behavior Modeled: MANA is an agent-based simulation inspired by ISAAC and using algorithms and behavior spaces apparently similar to Pythagoras. The range of modeled behavior for an agent includes movement, weapons fire, sensing & awareness, and communication. MANA uses a 2-D grid to represent the real world & physical distance. It also attempts to capture terrain features and their real world effects on agents. The communications module allows modeling of many process details, such as delay, transmit time, reliability, and so on.

Methodology: Agents can possess multiple goals (‘desires’) in relation to movement – often employing parameters about distance to other agents - and assigns each option a ‘score’. The scoring multiplies each goal achieved by the ‘weight’ of that goal, which is a user-defined parameter for each agent. The movement option is selected randomly from the set of scores falling within an acceptable parameter range, also user-defined. [Firing decisions probably use a simpler formula, as agents always prefer to fire at enemies meeting weapon range and sensor awareness criteria.] As mentioned above, agents communicate the location of enemy agents to each other according to user-defined parameters for latency, capacity, reliability, buffer, range, and recipients. Most of these parameters delay the arrival of communication or reduce its likelihood of success. Agent ‘personalities’ largely consists of the weights attached to various desires, many of which involve movement. Affiliation is as simple as possible – Agents can only be Friendly, Neutral, or Hostile, and values are static, except for trigger events. Trigger events allow the replacement of one static value with another value in most of the above parameters. This adds limited dynamism to the model, but only where the developer specifically makes room for it.

Strengths: MANA employs physics-based constraints on its weapon and movement behavior, adding important realism to that behavior set. MANA can be run very quickly in thousands of runs, and parameters can be set to iterate across a batch of runs, thus allowing exploration of full sets of possible values. MANA’s knowledge and training requirements are not onerous, and the tool seems to be government-owned. The communications model, though highly limited about type of information conveyed, employs realistic constraints in time and success quotient missing from more sophisticated models.

Limitations: As with Pythagoras, MANA relies on a large number of assumptions by the end user about the real-world exogenous variables that determine the number of agents, their armnament, movement rates, affiliation, and so on. MANA models physics-based constraints in lesser detail than Pythagoras and lacks even a crude system for incremental change in affiliation, making models IO and ‘hearts and minds’ operations very unrealistic. Behavioral options of agents are highly limited, and agents require extensive user specification, so total effort increases linearly with agent quantities. Data requirements & collection processes are left to the user. Computer memory needed also scales with scenario ‘size’ (number of agents), but MANA does not appear to be scalable. The model is heavily deterministic, with exceptions for movement and perhaps communication.

MIT State Stability Model

Behavior Modeled: The MIT State Stability Model consists of a conceptual flowchart that postulates certain results in a politico-military-system, given certain input parameters. Input paramaters consist of all ‘parent’ nodes, whose value is not influenced by other nodes in the chart. For MIT, these are Population Growth, Normal Probability of Recruitment, Economic Performance, Political Capacity, Social Capacity, Propensity to Protest, Propensity to Violence, Relative Strength of Violent Incidents, Message Effect Strength, and Desired Time to Remove Insurgents. Once the values of these parameters have been set, the system will calculate values for all nodes in the flowchart over any number of iterations. The calculations relating parent and children nodes are static.

Methodology: The MIT model consists of several subsets. The core of the model depicts “Population” members positively correlated to Dissidents, Insurgents, and finally Removed Insurgents. Each link in that chain is related to secondary factors. Recruitment, the node linking Population to Dissidents, is affected by Regime Opponents (quantity), and Propensity to be Recruited. Various parent nodes related to regime nonmilitary capacity impact a “Regime Resilence” node, which in turn impacts Propensity to be Recruited. Insurgents generate Incidents, which are positively correlated with Increasing Anti-regime Messages, which impacts recruitment nodes. Anti-regime messages is also correlated with other parent nodes. The specific calculations between any of these nodes (many nodes logically seem to transfer only a fraction of their value to ‘children’) are not known, but they do not appear to change over the course of a simulation.

Strengths: Concepts are intuitive and easy to understand. The tool promotes non-kinetic operations, by stipulating the feedback that comes from “Violent Incidents”.

Limitations: MIT State Stability model is a “System Dynamics” model. What is meant is that this model does not vary the relationship between any two nodes, based on re-evaluation, varying circumstances, poor execution, and so on. The model output is predictable, and there is no adapation or learning process between nodes to change any nodal relationships. Many parent nodes – the basic nodes from which the results are captured – are forced to quantify items such as “Political Capital”, where no objective data can exist. MIT suggests that the model’s parent nodes, or input parameters, need not be realistic, as the Tool’s purpose is to verify and teach the interconnections listed. No mechanism is described by which to verify the simulated results against real results. Due to the large number of unobservable variables under the end user’s direct control and the predictable nature of the calculations that follow, data integrity is low.

Note that this table was never completed, due to time constraints. File:MIT Table.png


Behavior Modeled: Nexus is a high-resolution agent-based simulation of the evolution of affiliation between organized groups in a society. Each agent in the simulation represents a social group whose decisions and cognitive processes are assumed as equivalent to all the group’s members. Each agent is linked to its own matrix of interdependent nodes representing subsets of attitude and affiliation towards every other agent in the simulation. Thus, each of Agents [B, C, D… N], Agent A has an “Opinion Set” consisting of the following nodes – a ‘support’ node, a ‘trust’ node, an ‘ideology’ and a series of ‘Evidence’ nodes linked to associated ‘blame’ nodes. ‘Support’ and ‘trust’ are recalculated every turn, ‘evidence’ for each agent is derived from actions taken by that agent, and ‘ideology’ – which represents a metric of ideological similarity to the reference agent – is held constant.

Methodology: Nexus relies on a calculation sequence for its support, trust, and blame nodes based on neural networks, in turn derived from scientific knowledge of association and cognition within the human brain. Each node in the Opinion Set for Agent A of Agent B is linked both to other nodes in the set, and similar nodes in agent [C, D…N]. Links can be positive or negative. In each turn after the initial, the activation (all nodes are binary: do / does not support, do / does not blame for Event A, etc) of all linked nodes is calculated as an overall probability between 0 and 1. The node then does or does not activate with a probability equal to that calculation. The result is that agents prefer to support those supported by others they support. After each round of calculation, agents take actions against other agents they do not support, and those actions are recorded as ‘evidence’ nodes2. Evidence nodes initially assign blame to the agent responsible for evidence, but the effect of other linkages can re-associate the blame to other agents over time. The initial node activations and linkage patterns are set based on SME opinion – SME vocabulary is translated to numeric values according to a methodology derived from Affective Control Theory.

Strengths: Nexus uses a model of simulation drawn from the activation of neurons in the human brain – activity patterns empirically observed. It does not follow that this pattern, applied in a different context, is demonstrated as accurate, but it does intuitively suggest the relevance of the paradigm. Unlike most models, Nexus’ model is dynamic backwards in time – the behavioral impact of past events persists and can recalculate over time, which seems to mirror human reinterpretation of past events. The associational, cumulative method of solving affiliation between two groups as a product of all other associations seems to be a promising and realistic concept. Nexus can be integrated with other models to provide greater granularity on behavioral areas it does not consider in detail.

Limitations: Nexus is a narrowly focused simulation that does not attempt to model the varying parameters of many aspects of human interaction. Specifically, “actions” vary in significance based available “evidence” nodes, but actions themselves are generalized so that variations in the intensity of a possible hostile action are not relevant to the effect of the action on support, trust, and other nodes.3 Resource possession is also not modeled, and thus is not factored into the inter-group support decisions, although social science suggests that groups tend to offer support based on prospective sharing of resources. Nexus does not offer guidance on differential behavior of individuals within groups. It assumes the complete, certain, accurate flow of information to each agent about the behavior of other agents, and seems to assume an omnidirectional information flow, whereas in real life agents acquire and disseminate information according to different patterns and in a chronological sequence.

File:NEXUS Table.png

ORA - Organizational Risk Analyzer

Behavior Modeled: ORA is an network analysis tool that can be used to assess and simulate the characteristics of organizational behavior and structure, rather than the specific organizational behavior. Given the real organization, ORA can simulate desired or expected internal organizational changes. At the heart of ORA is a “Meta-Network” that lists the key People, Knowledge, Resources, Tasks, Locations and so on of the organization, and examines how specific entities in each set are linked to each other, drawing conclusions from the cumulative linkage patterns. The Meta-Network can be filled out by the leadership of an organization, for use in self-assessments or inferred from email, or collected by HR. Alternatively, ORA can use various add-on tools such as Automap, to extract the Meta-Network from raw text data.

Methodology: Once the initial series of networks have been created, either manually by users or via Automap, ORA analyses the connectivity patterns, using a suite of over 60 metrics classified by entity class, risk category, and scale (whether the metric applies to individual nodes or to groupings). Metrics in risk category assess the exclusivity of links from knowledge, tasks, or resources to a single employee. Another set assesses whether employee/task, employee/resource, and task/resource linkages interrelate effectively. A third assesses the interdependency of tasks and association with specific employees. Another examines communication between employees with similar knowledge bases. ORA outputs are organized by risk category, and average results per category are visually represented. ORA contains some ability to suggest changes to an organization, and simulate the results of possible changes to the organization, in terms of the relevant metrics applied previously. Changes and results do not describe specific entity behavior choices & actions in relation to the real world.

Strengths: ORA looks at data internal to single organizations, thus studying rarely considered variances in organizational behavior. Its analytical ouput is thus focused on subtle organizational attributes, based on internal communications metadata, that are often overlooked or guessed at. ORA can store MetaNetworks for multiple organizations and/or time periods and make comparisons between them; it also contains an Optimization tool that suggests linkage changes that provide better results according to a customizable set of measures; and it contains facilities for detecting change over time in networks and for forecasting potential changes in these organizational networks. ORA also comes with a set of visualization tools, as well as grouping and pattern detection algorithms and a variety of ways to customize the views of MetaNetwork or their subnetworks. ORA requires only a Windows 2000/XP PC and is available for free. ORA scales well for large networks. If Automap or a similar option is used, the tool requires little user management beyond data collection. CASOS claims validation of the metrics: assuming validity is empirically accurate, and the underlying network data is accurate and representative – which may be difficult to determine – ORA’s information may be useful in understanding vulnerablities in both friendly and hostile organizations.

Limitations: ORA does not assess relationships between an organization and the outside world, so the effect of DIME actions (for example) could only be modeled via manual and estimated changes to network structures. ORA is of only indirect use in modeling change in world states in which multiple entities interact. It may not be easy for non-experts to understand the significance of the various metrics, especially when metrics in the same risk category disagree. ORA can simulate peformance of possible entity evolution, but does not predict evolution itself. This allows for “dynamic” analysis within a time-series of observations, but not dynamic simulation, meaning automated reiterations of feedback between entity behavior and external stimulus. Subjective creation of the MetaNetwork by analysts or organization members may not reflect real behavior, and only limited quality control procedures are currently in ORA. Meanwhile, the automated generators are vulnerable to incomplete or poorly bounded data sets (one Automap-generated network of Al-Qaeda, analyzed from open-source media, included Ariel Sharon as part of the network)

File:ORA Table.png

PMFSERV - Performance Moderator Function Server

Behavior Modeled: PMFServ is a computational methodology for simulating human behavior created by Dr. Silverman of the University of Pennsylvania. The methodology consists of a framework of various interdependent modules (called “performance moderator functions”, or PMFs) that attempt to implement various aspects of human psychology and resultant effects on behavior. The various modules interact to determine the behavior output of an agent in response to various events. The modules are based on series of “Best-Of-Breed” social science theories from the human behavior literature, as well as, at times, Dr. Silverman’s own heuristics for quantifying and interrelating these theories. PMFServ forms the core of several simulation software packages, including Athena’s Prism, FactionSim, and CountrySim. The available behavior differs in each simulation, and this briefing does not focus on behavior options available to agents, but instead on the process by which PMFServ translates world events into agent actions.

Methodology: PMFServ methodology is complex and not fully articulated here. However, the core of the cognitive framework that differentiates agent behavior in PMFServ is the agent’s GSP Tree (Goals, Standards, Preferences). GSP trees detail an agent’s inherent biases, both towards specific action choices on its own part and towards or against certain moral and process-oriented behavior patterns. Like most of the modules, GSP trees are dynamic, influenced both by other modules like the biological PMF’s and the Social Relationship Module; both of these influence how world events influence the GSP Tree. Ultimately, GSP trees interpret the significance of simulation events and pass the result to emotion modules. The extent and manner in which an external event conflicts or matches an agent’s GSP Tree determines the emotions they will experience. The emotions are calculated to further to add or subtract to the utility of various possible agent actions, which is otherwise derived from the action’s match with the GSP trees. The emotions are framed as positive/negative pairs; as world states trigger a negative emotion, actions that trigger an antithetical emotion will receive greater weight. The relationship between world state and emotional activation is also influenced by the social module, of which space prohibits extended discussion. Meanwhile, emotional activations also stimulate the biological PMF’s.

For an example of a biological PMF, the “Stress” accepts Gillis-Hursh’s three main categories of stress sources – effective fatigue, event stress and time pressure – and presents the results as numeric values. These numeric values set the agent’s “coping” style to one of five Janis-Mann “coping styles”, four of which are inefficient. The three stress sources here are in turn calculated from social-science-derived theory. For example, Effective Fatigue is determined according to the value of an “energy tank”, which is in derived from the values of nutrition, sleep, injury and temperature tanks. These values can currently be set by the user – it is not clear that they are dynamically simulated. (In other words, agents do not have an “eat food” action). In sum, computed stress controls coping style, and coping style affects Utility Calculations, after these have been determined by other modules. The Utility Calculations serve as the final driver of agent decision choices.

Strengths: PMFServ’s methodology stands alone among observed models in complexity; this briefing has no space to overview, even generally, more than a fraction of it. The simulation incorporates an unusually broad array of social science theory; the theories chosen must meet criteria for robustness and peer acceptance. Almost all agent traits are dynamic and influenced by simulation events. PMFServ is coded so that individual modules can be removed or substituted for other modules. Most of the simulations informed by PMFServ can be run on a desktop PC. The most recent simulation (CountrySim) is claimed to have high traceability with medium ease of use. The qualitative aspect of building any given model requires about 12 hours on average, quantitative data requirements do not seem to be high. The complexity of the system and the close interaction with U. Penn required inhibits curve-fitting, i.e. manipulation of input to correlate model output with real-world observations.

Limitations: While simplicity can serve as a constraint on predictive ability, an increase in complexity does not guarantee a corresponding increase in accuracy. The robustness of PMFServ as a whole in response to inaccurate modeling in any sub-modules is unclear, and information on the predictive record of PMFServ is not available.4 Importantly, the complexity of the cognition models is not matched in modeling physical behavior and variation in action outcomes based on agent capabilities, which threatens to misinform the world states reported to the model. PMFServ requires area specialists to code many variables whose values are not empirically certain, such as the GSP trees of various leaders. This process is subject to error. PMFServ’s minimum training time is 2 days, but genuine confidence may require up to a month of experimentation, with guidance from U. Penn. PMFServ appears to assume total information by agents about world states and events.

File:PMFSERV Table.png

POFED - Politics of Fertility, Economics, and Development) Model

Jordan Willcox & Bryce Nicholson

Behavior Modeled: POFED bears certain similarities to parsimonious or “thin” models such as SENTURION, in that it applies a relatively small, (and in this case, public) set of equations homogenously to a relatively narrow dataset, consisting of both empirical and subjective measures, and returns predicted future values of these metrics. From there, POFED draws conclusions about the state’s condition and the predicted utility of various foreign interventions. Using key relationships between its data values, POFED predicts aggregate behavior along a few specific axes. POFED requires measurements over time of certain economic & political conditions, such as tax revenue, GDP, various categories of government spending, as so on, as well as certain estimated variables,. It proceeds to model the interplay of these political, economic & demographic indicators to anticipate their change over time, as well as impact of interventions in the nation-state. It can thus be used to identify & anticipate direct policy levers to mitigate state fragility. In addition to identifying interagency actions that impact structural conditions to decrease state fragility, links to provincial level analysis can provide detailed tactical leverage points.

Methodology: The POFED model has five major components that capture the vast majority of state fragility and the effects of potential intervention: Income (y), Fertility (b), Human Capital (h), Instability (S), and Relative Political Capacity (X). POFED is a dynamic general equilibrium model based on the intersection of political and economic maximization functions in which individuals seek to maximize their lifetime utility by choosing how much to consume, save, and how many children to have while policy-makers choose the tax rate, the amount of public investment, and military spending to maximize their chances of remaining in power. Ultimately, the model purports that higher fertility rates impede economic growth, through diminishing rates of generational human capital transfer, diminishing capital-labor ratios, and other mechanisms. This is commonly suggested in development literature; but POFED also specifies a relationship between low levels of Political Freedom, RPC, and Instability values, and high fertility rates. This can create a negative feedback loop known as the “poverty trap”, which is one of several possible equilibria in the political-economic market dynamic. POFED’s model suggests that nations in this equilibria will not successfully employ economic aid.

Strengths: Evidence of successful predictions is claimed - as a forecasting tool POFED correctly anticipated recovery from exogenous shocks in Asia (such as the tsunami & financial crises). POFED has been applied by several US agencies to a variety of nations, mostly in Africa. POFED’s architecture, consisting only of public equations, is completely transparent and has very small requirements of data storage. Some of POFED’s posited relationships are relatively well-known and accepted in social science, although this does not necessarily extend to the specific parameters of POFED’s equations. The POFED approach has been cross-nationally validated for 78 countries across various indicators, and at the provincial or sub-national level for 6 countries. “Validated” here means that statistical tests of national data over time have shown the expected correlation between the variables. For the given real changes in the values of the various economic and political indicators, fertility rates have changed in the expected direction to a statistically significant extent. There are only a few variables in the POFED model that seem to be subjective, rather than representative of empirical data – notably the following variables: preference for children, and the ‘patience parameter’ specifying preference for consumption in adulthood vs. old age.

Limitations: POFED models trends in behavior such as births per household, but does not simulate the unique responses of individualized agents to their environment. Its domain largely excludes military operations and their effects on the other factors (except in the most general sense, such as the “Instability” variable, which seems to exclude war with external powers regardless). Its results may relate to anticipation of conflict, but does not simulate or predict the results of conflict, except for its effects on fertility rates. Several of POFED’s key variables do not directly correspond to empirically definite phoenomena, and must be estimated. It is not clear that the methods used for this process, by Sentia or other agencies, take precautions against error here. Similarly, this model is affected by general questions about the reliability of governmental economic data. As one of POFED’s variables is RPC (which is a ratio of expected tax revenue to actual tax revenue), the model cannot be applied effectively to states which have experienced total governmental collapse (as tax revenues will be zero).

File:POFED Table.png

PSOM - Peace Support Operation Model

Behavior Modeled: Not a true model, PSOM is a multi-sided interactive war game which pits a unified stabilizing force against either a unified or fractious insurgent force. Red (insurgent) and Blue (counterinsurgent) forces are distributed across a national territory and proceed, under human guidance, to fight for their opposing objectives. Each force can choose from a limited menu of options that will change the environmental values of their current territory, and/or else attack enemy forces in that territory. Blue’s forces include “Yellow” forces representing humanitarian NGO’s.

Methodology: The game’s iterations (“{turns”) occur across a series of uniformly shaped tiles which can be adjusted to reflect terrain and population density. PSOM further postulates that enduring success for a Stability Operation rests on achieving three critical conditions: 1) sufficient consent of the indigenous population, 2) sufficient security to enable normal societal activity, 3) sufficient stability (national functionality) for sustained reconstruction. Across multiple iterations, PSOM generates an indexed score for each of the three aforementioned conditions. The score is affected by both Red and Blue actions, and it in turn affects the value of these actions. For example, Red units regenerate more quickly in territories with a low “consent” value.

Strengths: Some randomness is incorporated into PSOM results. Very few data requirements are specified for the determination of initial environmental and population values within the scenario hexes, allowing PSOM to be played with little setup time. The rules that quantify the effects of actions by the entities are consistent across games, allowing for the possibility of probability distributions across multiple game iterations. PSOM does attempt to consider non-kinetic factors and negative relationships between kinetic operations and non-kinetic factors, in its calculations.

Limitations: PSOM is neither predictive, nor a simulation. Its scenarios do not appear to require resources available to “Red” or “Blue” to be derived from data, nor does it provide an estimation methodology for determining those resources. The capabilities of different types of units are, similarly, plausible but subjective. Furthermore, PSOM assumes identical performances from all instances of the same unit class; capabilities determine performance. In the real world, Blue and Red performance varies according to a number of circumstantial factors (number of translators, time delay in air support response, ideological fervor, etc). Also not considered are various psychological, bureaucratic, and other circumstantial influences on the choices made by actual military, civilian, and insurgent units, to which game players are not subject. There seems to be very unit little individuality or dynamic evolution involved in the system. The available information does not clarify the specific calculations by which unit statistics affect the environmental indexes of the hex. Human-in-the-loop games are subject to results influenced by the background of the individuals playing the games. No validation of the internal logic structures used to relate actions to results is discussed.

File:PSOM Table 1.png File:PSOM Table 2.png

PSTK - Power Structure Toolkit

Behavior Modeled: PSTK is an agent-based modeling tool that allows a single type of interaction between its agents: the transfer of ‘capital’, representing resources possessed by agents. Agents are motivated by one or more simple goals related to ownership of capital. (A literal quote of an example goal: “Actor A wants to have more economic power5 than Actor B”). Based on these goals, actors can transfer capital in a positive manner to agents, mirroring cooperation and aid, or in a negative manner, representing the spectrum of real-world hostile actions.

Methodology: “Agents” are a system primitive representing real-world entities of any scope – individuals, organizations, etc. The dynamics of the prior paragraph represent much of PSTK’s methodology. Another important aspect is “Power”, assigned by the end user to agents and representing (as a scalar) a constraint on the agent’s ability to use its capital. It appears that ‘power’ represents a fixed constraint and can not be affected by other agents, although the documentation contradicts itself on this question. Another constraint on agent behavior involves the user-defined “lines of influence” (LOI) that link the various agents. An agent can only transfer capital to another agent if they are connected by lines of influence, and these are also set initially by the user.

Strengths: PSTK appears to have low hardware requirements, and can run quickly. Its creators suggest that it can be easily incorporated into multi-resolution-models, i.e simulation environments incorporating multiple modeling tools. PSTK can model at any scope and the abstract nature of the interactions allows it to model various interaction domains (political, economic, military). Traceability is high.

Limitations: Certain functions important to creating a valid model are left up to the subjective judgment of the end user. For example, the model asks the end user to enter initial allocations of “power” and “capital”, specific to unspecified subdomains, and to specify the fungibility of power across domains; but the model does not provide a validated methodology for converting real-world observable data to “power” units, or for determining the fungibility of power. These freedoms, common to modeling “toolkits”, ensure that the predictive utility of the model varies by user and by individual model instances. However, the potential advantages of this choice are hindered by other limitations - PSTK is not a completely “thin” tool, as it does encode theory into the model framework. An advantage of toolkits is to allow user experimentation towards a superior model, but PTSK poses severe constraints on user design that may obstruct this evolution to a high extent. For example, agent ‘goals’ are pre-set into a hierarchy of importance and cannot dynamically adjust during the scenario. LOI paths are also predefined and cannot change. The fact that every agent has only two actions – positive and negative transfer of capital – is an extreme simplification of real-world action options. PSTK actions of a given capital cost seem to vary only by the ‘power’ constraint, which provides only an upper bound. PSTK’s reliance on “power” as a sole, unvarying constraint may render it unable to forecast actor effectiveness that varies widely in a given domain, discontinuously with capital expenditure. By default, agents use perfect rationality to achieve goals, limited in scope to the capital possessed by various actors, with no ethical, legal, or logistical constraints upon action. Some available ‘belief’ algorithms represent a step forward in agent cognition, but the fact that different systems can be used by different agents in the same scenario, without an underlying real-world rationale for the differentiation, is likely to impair realistic outcomes. Lastly, while ‘power’ varies across domains, the model appears to assume a universal stock of ‘capital’ that is equally expendable in any domain.

File:PSTK Table.png


Behavior Modeled: Pythagoras is an agent-based simulation whose algorithms provide rules for how agents move, when they take violent actions and to whom, and to whom they consider themselves as either friendly or hostile. Pythagoras is among a small subset of ABMs that pay careful and detailed attention to physics-based constraints on individual agent capabilities. Agents act within a specific and artificial physical environment, which considers terrain, weapon lethality, physical distance to targets, and other agents, LoS, climate, and so on. This fine-grained detail may be in part responsible for a soft constraint on the number of agents per scenario, typically held to the low hundreds.

Methodology: A key concept used repeatedly in Pythagoras’ decision rules is ‘fuzzy logic’, a modeling technique that incorporates both deterministic and stochastic elements. Determinism is present in that agent acts generate numerically consistent changes to other agents’ internal variable values. Stochasticity is added through the use of decision thresholds that vary randomly for each agent within a bounded range. The result is that while being shot at or given food might internally affect variable values to a constant magnitude, identical final variable values may provoke, in the boundary areas, different agent responses. This technique accounts for the fact that human beings assign differing significance to the same verbal and physical events. Decision ranges and other agent attributes have limited dynamism – they can change in reaction to predefined event triggers created by the developers. Leader agents can give orders, and agent obedience can vary more or less according to the above system. Another factor inducing variance in agent response to stimuli is Pythagoras’ allowance for multiple desires and method of resolving conflicts in desires. Agents can consult a list of desires when deciding on a specific action, such as movement. It can require the sum of desire inputs to pass a collective threshold to initiate action, and compare the strengths of competing desires to determine the nature (direction, intensity) of the action. Other available options in decision logic include random selection and averages. A final noteworthy feature is the complex system to determine effectiveness of certain actions, like weapon fire, which considers Kill ranges, terrain, and range, using realistic physics models.

Strengths: When the strategic assumptions are appropriate, Pythagoras uses robust, insightful, complex algorithms to determine agent core behaviors. Pythagoras runs on a single modern PC, and also allows for batch processing by processor clusters that allow scenarios to be run in thousands of iterations in short times. Pythagoras has been used often, and its results are very traceable. The consequences of movement and weapon fire are modeled with much more precision than many broad-scope simulations. As a precise and complex combat simulation with a foundation added of non-kinetic actions and effects, Pythagoras demonstrates methodological approaches that could be important and useful to future models.

Limitations: The most obvious limitation is that agents in Pythagoras have a limited number of actions – not much more than move, shoot lethal weapons or “influence beams” (nonlethal propaganda equivalents using the weapon modality), obey or give orders to move or shoot, and decide affiliation of other agents. Political, economic, and strategic factors that would determine how many red, blue, and neutral agents should be included in a scenario, how they would be armed, and so on can not be modeled, and must be assumed. These assumptions may not be valid and may render real-world operations unlike Pythagoras models. The user input can be a significant time investment (libraries / tools exist to expedite it). Although Pythagoras can run on a single PC, required user input and computing resources increase with scale. Pythagoras does not attempt to codify the data collection process, or broaden that process beyond the end-user and protect it from bias. Non-lethal behavior, while better than nonexistent, is simplistic, with generic “propaganda fire” actions that do not mimic real information flows well, nor the conditional effectiveness of such actions. Leader control needs to be less deterministic – not only allowing obedience to vary randomly in a controlled range, but to allow the range itself to change dynamically and conditionally.

File:PYTHAG Table.png

SEAS - Synthetic Environment for Analysis & Simulation

Behavior Modeled: SEAS tries to comprehensively model the social, cultural, psychological processes, and the physical actions influencing a virtual population or territory, focusing political and military actions. Depending on entity type, agents can communicate to each other, change each other’s attitudes and wellbeing both through explicit actions and automatic comparative & calibration functions, join and leave organizations, riot, demonstrate, participate in violence, and take constructive actions. Infrastructure is modeled as nation-specific organization-class agents who communicate information on their status.

Methodology: There are four basic types of agents – Individuals, Organizations, Institutions, and Infrastructure Sectors. The simulation requires a set of Individual-class agents proportional in demography to the real nation or territory modeled. Individuals have static traits assigned: race, ethnicity, income, education, religion, gender, and nationalism. They also have dynamic traits, ‘orientations’ within political, social, religious, and ‘violence’ spectrums. Individuals have a dynamic well-being value across eight categories of needs, and also receive information about world events from other individuals, institutions, and media organizations. All of these inputs determine agent actions (from within various lists).7 Wellbeing motivates individuals to join or leave organizations offering (or failing to offer) appropriate well-being benefits. Soldiers and leaders are other Individual-class agent types with special traits (example: “will to fight”). Leaders can act to change both the wellbeing and the attitudes of their members, using certain ‘action’ types. Individual-class and other agents have sensors from which they receive information about other agents’ well-being and actions. Meanwhile, Institutions and Organizations are modeled as single agents with membership linkages to Individual-class agents. Organizational perceptions and reactions seem to be shaped by those of its leaders, and shape affiliated individuals in turn. The available actions for organizations and institutions differ. Media organizations are a subtype of the Organization-class agent, focused on communicating information. The effect of the information is influenced by the media agent’s ideological position, which also helps determine communication patterns to agents. Organizations and institutions use resources to influence member wellbeing. Resources come from infrastructure agents, subject to attack (or, for media, purchase).

Strengths: SEAS contains interesting, potentially useful instantiations of theory in several areas, including: Associational / Social influences on desires/weights, leader influence on desires, resource calculations, information flow tracking & agent reconciliation of multiple sources, media affiliation, and agent sensing frequencies. SEAS goes to great lengths to render its controlling variables dynamic. Very large simulations are possible. SEAS is also a full-spectrum model, broad enough to be potentially useful in a wide variety of military and non-military operations. The SEAS developers are continuously collecting data on 81 countries to facilitate the initial data collection process. As a whole, SEAS demonstrates above-average ambition, complexity, and comprehensiveness in its simulated universe, although this does not guarantee superior accuracy.

Limitations: Some traits open to dynamic change lack stochasticity in initial settings. Individual starting ‘desires’, ‘wellbeing’, and ‘action thresholds’ are initially assigned according to demographic & other static traits, rather than incorporating variance. The net result seems to risks excessive homogeneity and self-reinforcing feedback loops. Leader ‘types’ – static traits influencing choices of action – are dynamically affected by aggregate preferences of their organizations, but this may exclude external influences on leader decision tendencies. Well-being categories of SEAS are broad, but do not include negatory well-being – such as aversive and antithetical motivations. Leader ability to influence “weight” modifiers to desires as well as wellbeing potentially overstates leader control. There are limitations to the action possibilities for both individuals and organizations, most acutely in the area of “constructive” actions for organizations (institutions, but not organizations, can undertake rebuilding actions). Infrastructure interdependencies are static, as are infrastructure agent outputs. Associational models of information spread are important but seem over-determined (some stochastic, counter-intuitive interaction with hostile-source information is needed). SEAS requires large quantities of data and increasing computing resources with simulation size. SEAS must be linked to other simulations to provide realistic combat modeling. In general, resource availability seems to be overly static.

File:SEAS Table.png


Behavior Modeled: Senturion models the change over time of the stated positions of a user-defined set of stakeholders seen as influential to the outcome of a particular political issue or disagreement. From these simulated changes in stated positions (or stated preferences), the analyst can draw inferences to expected stakeholder bargaining behavior on the issue. When groups of stakeholders with decision making authority behave in particular ways (e.g. reaching agreement, forming opposing coalitions, polarizing in their positions), then the analyst can interpret the expected outcome as indicated by the Senturion simulation. The model does not explicitly model actions taken by stakeholders to achieve a world state compliant with their preferences, nor any other physical actions.

Methodology: Senturion uses algorithms derived from game theory, decision theory, spatial bargaining, and micro-economics. Data are collected from structured interviews with SMEs to define the bargaining space (the issue), and identify stakeholders and their positions for the defined issue. Stakeholders are classified according to a matrix of power and motivation, and their positions are collected and evaluated against the aggregated issue positions to create a ‘risk profile’ for the stakeholder. Negotiation among stakeholders, who may form coalitions and groups, is then simulated through agent-based modeling, and agent reactions are shaped by their characteristics – characteristics based in part on extrinsic data. The algorithms consider the relative strengths of the issue coalitions to determine what proposals will be made and the reaction to them. Senturion attempts to predict change in issue position of agents/stakeholders based not only on their interests, but their knowledge of median position and expectations of the final result – so initial changes are factored into later changes.

Strengths: The model has been used extensively used, and the owners attest to several documented cases of accurate predictions. Data needs are manageably light, and sources are consistent across time and thus less subject to random fluctuation than data collection methods used for other models. Output is intuitive and easy to understand. The theory base seems sophisticated and plausibly ‘bounded realism’ (i.e., describes accurately a piece or aspect of a complex dynamic). The narrowly focused output makes it easier to understand the results and create an empirical record of model performance. Senturion’s core theory is important and relevant to other models. The risk of proprietary code and algorithms is mitigated somewhat by the Joint Warfare Analysis Center’s review of the Senturion software code and algorithms, which is available to other U.S. government entities. Some of Senturion’s “limitations” are nevertheless superior to other model methods. For example, to procedurally specify that a group of SME’s are to input subjective variables creates protection against nonrandom bias that is limited, but seems superior to leaving similar variable settings up to the end user.

Limitations: Stakeholder power and motivation, expect as influenced by coalition formation, is held fixed within a Senturion simulation and does not change endogenously. Real dynamics may change over the observation period, possibly from external ‘shocks’, as Senturion does not independently incorporate new physical events. Sensitivity analysis may (or may not) be performed to help assess the extent of these vulnerabilities. The classification system and stakeholder selection relies on SME opinion, vulnerable to nonrandom bias. Senturion analysts are trained to select a representative sample of SME opinion and perform sensitivity analysis on divergent SME judgments, but effectiveness depends on the user. Even a diverse group of SMEs may provide less reliable classifications about stakeholders who refuse communication and seek to limit observation by others. The proprietary format severely limits review and evaluation of the algorithms, and traceability of the results is not complete. The methodology does not account for meaningful asymmetry in stakeholder perceptions or available information, or for deliberate misinformation. The risk quotient of a stakeholder position seems vulnerable to influence by factors external to the Senturion methodology. For example, a ‘risky’ position, defined as such by Senturion because of unusual variance from that of other stakeholders, might be subsidized against risk for an actor by external powers, or a ‘moderate’ position might indicate benefits specific to that position and stakeholder, rather than risk aversion. The model’s limited scope of forecasting – of political positions on pre-established issues – constrains the utility and flexibility of the projections, even as it simplifies data collection and possibly facilitates accuracy.

File:SENTURION Table.png

SIAM - Situational Influence Assessment Module

Behavior Modeled: SIAM software comes with very little built-in variable definitions or relationship, and is thus flexible enough to model almost any scenario or behavioral environment. Note that SIAM is not a simulation, and the outcome of all described behavioral processes are user-determined. The entities and events can be at any scale, but larger numbers of discrete entities (be they persons, brigades or hurricanes) or behaviors will require more time to model.

Methodology: SIAM is built on the concept of Bayesian Influence Nets, a common system in PMESII modeling, based on probability theory. The visual display is of a series of nodes, linked by lines or arrows. A node can represent an entity or physical object, or a perception held by an entity. Parent nodes ‘cause’ child nodes according to an analyst-defined probability decimal between 1 and -1. Algebraic equations calculate the probability of the child node, given the aggregate of positive and negative-probability parent nodes. Nodes are aggregated according to the rules of probability calculations, not as scalar values (for example, 0.5 + 0.5 = 0.75, not 1). A ‘baseline’ probability represents the influence of all non-modeled parent nodes (the non-modeled universe). Note that no methodology is presented to guide the analyst’s assignment of probabilities or the creation of nodes needed to describe causal paths of an event or environment.

Strengths: Calculations are very quick. Various options are available to specify probabilities of node subsets, perform sensitivity analyses of nodes. The tool is easy to understand and use. The results are completely traceable. Data collection requirements are up to the user, starting at none whatsoever.

Limitations: SIAM does not truly simulate and contains no persistent analytic content. It allows users to visually display their self-created, completely customizable network of causal relationships, and it calculates aggregate probabilities, based on the user-defined estimates of causal significance between nodes. The accuracy of any given relationship is unknown, and may or may not be knowable, as a poorly designed network may have few intermediate observable events. There is no requirement that nodes correspond to empirical or observable events. The model utility is completely dependent on the analyst design of individual models, requiring VV&A for each model created. The feasibility of VV&A will vary with model design. SIAM is static, and requires the user to update the network over time as indeterminate parent nodes representing uncertain events either do or do not occur, or change in significance. The freedom presented by SIAM makes it entirely possible to create nets that do not display or consider important phenomena. Furthermore, every unique detail of the model desired for inclusion requires user definition and probability descriptions. SIAM requires greater effort in model design as desired granularity increases: for example, if every soldier’s behavior in a company is to be considered separately, that requires 200 discrete sub-networks.

SIAM Table.png

SOCRATES - Simulation of Cooperative Realistic Autonomous Transportation Entities

Behavior Modeled: SOCRATES is an agent-based model whose agents make decisions regarding the following behaviors: movement, weapons fire, and communication. Agents may represent individual humans or groups at the modeler’s discretion. Agents also possess an identical set of goals, varied by the weights assigned to various goals (weights appear to be static) that influence their decision reactions to various stimuli, such as friendly or enemy communications. Another source of variation among agents includes various special capabilities – two examples are agents that are hard to sense and agents that intercept enemy communications. SOCRATES is based on the pilot decision logic in the Air Force Brawler air-to-air combat simulation. The “chessboard” spatial environment in Socrates considers distance and time increments.

Methodology: Agent decisions are driven by their “Sentiment” values towards a set of possible ‘colors’ of agent affiliation. Agent behavior is also affected by their personality variables, some listed below. “Introversion” sets the frequency of communication with friendly agents. “Agreeableness” sets agent sensitivity to “sentiment” influence via communications. “Emotional Stability” modifies the likelihood of random variance in other decision variables, and appears to also vary by agent environment (for example, according to local force ratios). Stability is also dampened by the “Conscientiousness Variable”.. and so on. The last component of decisions is modified by the ‘value’ weights assigned to various goals, such as survival, obedience, and threat attrition. The action selected is the one that provides the highest score of likely success per goal multiplied by goal weight. A good heuristic is that Goal Weights & Success perceptions determine what should be done, Sentiment determines who to do it to, and Personality variables indirectly affect the values of the other two categories. Communication pathways are user-configurable.

Strengths: SOCRATES takes some physics-based variables into account during combat modeling. Functionality for modeling information flow goes beyond command hierarchy and explicit orders, which is helpful for realism in this area. The personality variables, unfortunately static themselves, introduce variance in reactions to world events between agents.

Limitations: Most of the above variables are static, while theory and evidence suggest that human beings have dynamic goals, values, and sentiment, influenced by external events. The ‘emotional stability’ variable introduces dynamism, but not enough, and without bounds or nonrandom variation. Combat modeling is overly deterministic, without sufficient peformance variance in human capabilities & dynamic environmental influences. Actions are limited to movement, weapon fire, communication, and propaganda events. User-configurated values introduce a high risk of subjective or self-fulfilling initializations, not mitigated here with data integrity safeguards. The function of the module to determine agent expectations of success is not clear. Pythagoras is similar to SOCRATES, and seems to mostly improve upon it.

File:SOCRATES Table.png

Appendix A: Taxonomy & Context of Vital Statistics:

These statistics have been developed for a target audience of analysts, auditors, and modeling & simulation performers. The primary purpose is to create a set of short answers or easily memorable associations that describe the model roughly.

Developer: The organization responsible for creating the tool. If multiple organizations are involved, their current responsibilities and roles will be mentioned. Initial Release Date: Month and Year of the first ‘public’ (involving a third party as client, audience, etc) use of the modeling tool.

Key Theory Tags: Short names of a few theories or areas/branches of theory dubbed fundamental to the model. (Max: 3)

Limitation Tags: Short phrases listing some of the model’s most important or obvious weaknesses, risks, or limitations. Limitations will be generalized where possible to archetypes that occur over multiple models. Common limitations may be left out in favor of unique ones; list is not exhaustive. A glossary of Limitation Tags is found in Appendix B.

Strength Tags: Notable features of the model, tasks it performs well, weaknesses common to similar models that have been addressed or mitigated. Also generalized to archetypes where possible. A glossary of Strength tags is found in Appendix A.

Utilization Record: How many times the model has been used, with a few words about by whom and the circumstances, if possible.

Forecasting Capacity: The first response is coded as “Yes” if the model’s output includes a projected outcome of specific future or potential events, and “No” if it does not. The developer’s qualifications regarding the proper use / expected accuracy of these projections are noted. Second, specifications of the forecast parameters: “Point” if a single specific outcome derives from an event or event set, “Probability” if the model provides a set of possible outcomes (2+) with a numerical probability for each outcome. Third, the scope of the forecast will be described (exactly what is forecasted? Empirical Single Events? Frequencies of Multiple Events? Relationships? End State/Status of Conditions / Entities?)

Traceability: Estimates the extent to which a change in the value of any variable can be traced back to other inputs. The categories are: Low, Moderate, High, and Complete. Some things that affect traceability include the openness of equations and algorithms to external and user review; the ability to review (‘trace’) agent behavior for each increment or step of the simulation; and the ability to observe or examine changes to agent internal and world / environment variables and their origin.

Data Collection Source/Method: Describes where data required for the model is obtained, and how it is obtained.

Data Volume: Describes the relative quantity of data required. Terms are low, medium and high. If available, a brief description of the dataset required will be provided. An example of “High” data volume would be a system requiring, for example, six months of high-granularity real-world information on violent events in a conflict zone, perhaps from a government database. “Background” data inherent to the tool may be excluded from consideration.

Data Integrity Issues: A one-word estimate: Response categories are: Low, Moderate, Significant, and High. Some things that affect data integrity include:

  • Written documentation and training, or the lack thereof, for coding data in a consistent manner.
  • Frequency and significance of use of subjective (not directly observable, or requiring interpretation) data.
  • Who inputs the data, and how this input is supervised, or checked against other sources or estimates. A system that allows model end users to input their own data without scrutiny suffers in data integrity.
  • If the data is retrieved from other collections, integrity is influenced by how many collections, who created these collections, how accessible the collectors are and how susceptible the data is to cross-checking.
  • This is followed by a brief description of the validation issues concurrent to the data volume, source, and collection methods.

Minimum Use Requirements: Discusses the necessary preconditions for model requirements. The following, where possible, will be discussed: the data collection process in terms of its time required and difficulty; the training time for end-users, and the number of support staff, including any special skills (software developer) or affiliations (i.e., from the vendor) needed. Also, how often support staff involvement is required. Lastly, the hardware requirements will be mentioned.

Scope: Attempts to characterize the entities or agents whose interaction is modeled. Examples: Individual, Group, Single Organization, Multiple Organizations, Nation, and any combination of the above.

Domain: The PMESII categories of behavior modeled.

Operation Matches: Lists the types of military operations for which this simulation would appear to add value. Typical or at least demonstrated uses included, theoretical / uncommon uses often excluded.

Ownership Type: States the legal and institutional arrangements related to ownership, with implicit limitations on access to the tool, use of the tool, and exploration of the tool’s internal functions.

Personal tools