Transcript Complexity
Complexity Bruce Kogut October 2006 We are entering the epoch of the digitalization of knowledge: past, present, and future Sciences bring to this new epoch: – More powerful statistical methods (some which date back 50 years in physics) – Tools to manage and mine large datasets. – Methods, such as imaging, to make inferences from ‘damaged’ and ‘missing’ data. – Visualization technologies that make our standard powerpoint slides look very sad What Complexity Seems to Mean In Practice Interdisciplinary sharing of knowledge and creation of a larger community of scholarship. Appreciation of the ‘non-linear’ view of the world where critical events and self-organization matter. Analyzing the statistical properties of large datasets (100,000+) Understanding local interactions by micro-rules whose effects depend on topology (structure) Identification of common patterns by re-scaling and normalization (e.g. power laws) to seek more general explanations. Example from Venture Capital deals • Over 200,000 transactions over 40 years in the US alone. • Several thousand VC investors, targets • Let’s start by posing a simple question: – Can we find rules by which VC companies do deals? – Do these deals explain aggregate patterns? Examples of Rule-based Formation • We choose 18 firms that are connected from the actual data. Actual links are green lines. • We simulate for 60 iterations, with each click representing implementation of the stochastic rule. Rules are analyzed one at a time. • Red lines show outcome. • We collect network statistics at the end. Simulations with Four Rules Random Propensity Preferential Transitivity Simulation Strategy: Estimate from Data, Simulate Forward to the Future The common hypothesis is that Venture Capital Deals Favor the Big Players: Rich get Richer (sometimes called ‘preferential attachment’) 378.742 Freq75 Freq80 167.387 1 1 1 1 61 140 K K 3416.19 Frequency90 Frequency 1527.65 .223825 .694947 1 1 623 1104 K K Inference by adduction: the dog did not bark, the graph does not show linear slopes, there are no power laws in degree, hence the culprit of rich get richer is innocent and released. So much for complexity, big science, and robust patterns. Power Laws in Complex Weighted Graphs: Incumbents like to rely upon trusted partners • Most Deals are Incumbent to Incumbent 10000 • Hence we find power laws in repeated ties. Frequency 1000 100 10 1 1 • Trusted expertise based on experience, 10 100 1000 10000 Strength s These modest results can be shown to refute the leading hypothesis on venture capital: VC partnerships are not clustered by regions, they span regions even in the early history of the industry. 100000 Complexity is Already Percolating, and We Need to Take Notice • Organizational studies: claims of self-organizing teams and innovations. • Finance: clustered volatility, whereby there is ‘memory’ and hence inefficiency in markets. • Marketing: Data mining is the principal example. • Economics: Re-scaling shows remarkable commonalities in size and growth distributions. • Macro-sociology: Dynamics result in scale-free networks. • Statistics: Search for new methods applicable to big networks can sort out whether you smoke because you are born that way, or because you hang out with the (wrong/right?) friends. • Operations Research: once again, we can do math. Today’s Panel • Nigel Gilbert: sociologist and founding editor of the Journal of Artificial Societies and Simulation, shows how ideas from complexity have implications for how we do social science and management research. • Roberto Serra: Physicist, former manager, academic, now engaged in promoting complexity education. • Ralph Dum: Physicist, EU Commission, Catalyst for Complexity Research, Wants to See a European Santa Fe Institute built. • Jeff Johnson, Researcher in Design,Believer that Complexity requires math and math can be learned even by Ph.D. graduates.