Appendix D. Objective Tests

Note: This is an excerpt from a draft of my thesis, A Computer Model of National Behavior. The introduction and table of contents are also available

Appendix D. Objective Tests

D.1 Objective Test Descriptions

The simulation was checked against three objective tests. They measured the nation displacement, state displacement, and internal validity. Nation displacement is a measure of the degree to which the predominant nation in different places changed. State displacement is a measure of the degree to which the predominant state in different places changed. Internal validity demonstrates the difference between runs, and with that how consistent the output is.

Four nations were considered for every test. These included the British, German, Italian, and Polish nations. These nations were predetermined before the simulation code was written.

The displacement tests were considered successful if the simulation results matched known history. The results for all tests are interesting. Three of the national displacement tests were successful. In matrix form:

  Nation Displacement Test

State Displacement Test
British Pass Pass
German Pass Fail
Italian Pass Fail
Polish Marginally Pass Fail

For the national displacement test for the British, German, and Italian nations, no variance between reality and the simulation develop during the run.

For the state displacement test for German, Italian, and Polish, there was a great deal of variance. Specifically, total variation for those nations occurred by 1962, 1962, and 1961 respectively.

Regarding the national displacement test, it is argued that all nations pass the test, three completely and one marginally, since the model accurately captures the behavior of nations. The simulation was specifically designed to be a simulation of nations. An explanation of the special case of the Polish nation is on the following pages.

Regarding the state displacement test, it is clear that the model does not accurately model the behavior of states. That is, it does not correctly show how state behaviors emerge from national behaviors. The sole example is the British nation, whose behavior is completely correct throughout the simulation. The report then discusses this surprising result.

The report also investigates internal validity. Internal validity measures how much the simulation agrees with itself. A simulation that gives very different results each time would have little internal validity. The simulation exhibits internal validity, though issues with two specific nations, Italian and Polish, are troublesome. They pass by an absolute measure, but are much less internally valid than the British and German nations. The report then analyzes the ways changing how the data is interpreted effects the internal validity results.

Finally, the report mentions two unanticipated objective findings. Both point to areas of improvement. One relates to incomplete data in 5.7% of places, and the other relates to incomplete data reporting. The consequences of both are quickly discussed and it is recommended that these issues be resolved before future research is attempted.

Computer Science Thesis Index

Leave a Reply

Your email address will not be published. Required fields are marked *