Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Big Data and the Soviet Ghosts (mempko.wordpress.com)
62 points by mempko on Sept 8, 2013 | hide | past | favorite | 15 comments



There's a lot of interesting ping-ponging back and forth in some of these technocratic / data-driven visions. Some parts of Soviet central planning were initially lifted almost directly from Western success stories of massively integrated industrial conglomerates, which had a data-driven central-planning ideology. Some of the late 19th-century / early 20th century American industrialists explicitly thought that the era of markets was over, because the messy, imperfect information transmission of the medieval bazaar would be finally replaced with scientific and statistical optimization of production, management, supply chains, etc., and the more centralized, the more scientific and efficient this process could be. This was a major argument against anti-trust legislation, that by breaking up the large trusts, America was only going to be sending itself backwards into an era of decentralized, uncoordinated, inefficient industrial production.

The Soviets pretty much agreed with the large trust owners on all these points, disagreeing only on some minor details, like ownership. There was even a vision among a certain kind of early 20th century communist (nowadays called "orthodox Marxists", somewhat derisively) that capitalism was a necessary phase whose virtue lay in developing and centralizing industrial production. The communist could just wait until the capitalists have succeeded in unifying the entire industrial sector into one large conglomerate, and then the only thing left to do for a communist revolution is knock off this one remaining capitalist, who at this point owns a giant company controlling all industrial production. Then you install the party at the top of this company, and voila, a socialist, central-planned, scientific industrial sector!

Then from the 1930s American management started looking at Soviet management techniques to see if they could borrow anything back in the other direction. There were even some Soviet experiments in what today we would call "gamification", which were very interesting to Western observers, and copied in various ways [1]. One way of reconciling things was to consider a model of companies being Soviet-style internally but capitalist externally: use hierarchies, internal non-monetary rewards, central optimization, etc., up to the boundaries of the company (which would ideally be very expansive boundaries), then compete past that. And I suppose now we're at the point where using data to optimize society, another old Soviet idea, is seeming intriguing to the West...

[1] http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2115483


> One way of reconciling things was to consider a model of companies being Soviet-style internally but capitalist externally: use hierarchies, internal non-monetary rewards, central optimization, etc.,

I've always pointed this out in relation to large companies. They are not much different than a dictatorship or a single party totalitarian system. It comes full on with indoctrination into company culture, team building events.

Especially small towns where big factories used to pretty much own the towns, provide trash collection services, etc. If the company fired you, you had to pack up and leave. With countries, if you break rules they send you to a labor camp (prison in US). You criticize the company on Twitter you get punished.

In theory one could say well if you don't like the company you just move. In current economy not many have the option of giving their company the finger and saying "fuck you" and I switching jobs. People are desperate are willing to put up with extra hours without overtime, abuse and just to have a job.


> Especially small towns where big factories used to pretty much own the towns

Don't forget the practice of providing pay in company scrip that could only be spent in company provided stores.

https://en.wikipedia.org/wiki/Company_scrip

In countries with very wide reaching conglomerates, like Japan and South Korea, I can almost predict a cyberpunk like future where you become a citizen of a company and cradle to grave live out your life in that company. Eat feed grown on company farms (farmed with company equipment), live in a company apartment, drive company cars, wear company clothes, watch entertainment on company TVs etc.

Samsung is an example of a company that I think has almost that kind of reach. And to put it into perspective, Samsung has revenue about the same as the GDP of Finland, Israel or Chile. And exceeds countries like Egypt, Pakistan, the Philippines, and Vietnam.


You're too late to predict that. Employment is on the downslide, companies care less about securing workforce supply and more about attracting talent and cross-pollinating.

You could expect that 20 years ago, and to some extent it happened, but I doubt it's the future.


The central planning, linear programming-driven Soviet dream was explored in the excellent book "Red Plenty" by Francis Spufford. I would heavily recommend it.

http://www.amazon.com/Red-Plenty-Francis-Spufford/dp/B00B9ZD...


Excellent and insightful reply. Most people don't realize how much soviet planners admired American industry. Stalin admired Ford for example. Lenin said communism is socialism with electricity. Stalin's planners literally copied Gary Indiana to push the Russian Steal industry and hired many American engineers such as grandpa Koch to develop the oil industry.


Everything about this is wrong. How can you compare early computer analysis by central planners in the Soviet Union to data mining employed by advertisers?

This paragraph takes the cake:

>Just as the centralized Soviet economy and computer systems produced gross absurdities in Soviet society, the centralization of information has produced vast obscurities in the lives of this generation. As demonstrated by luke warm reaction to the recent revelations about the NSA, to the Great Recession and the computerization of trade, to the perverse incentives Facebook and Twitter provide for exhibitionist behaviours, to the death march of consumerism despite the hard evidence that global warming will lead to a miserable existence for hundreds of millions of people within decades.

None of these sentence fragments have anything to do with each other. The author is so vague a reader can draw pretty much any conclusions they want from this article.


I think there's a valid comparison there, although I'm not sure the author captured it. We're being collectively manipulated by all sorts of marketing and propaganda campaigns.

Those manipulations cause us to make decisions not necessarily in our long-term interest.


On the side of computer controlled economy, like what GosPlan had, there is also the Chilean computer planned economy system -- Cybersyn

http://blog.fabric.ch/index.php?/archives/1886-Cybersyn-a-re...

In general, assuming we've already given up our privacy and it is gone forever, Google+Government can read our thoughts, email, shopping patterns, knows our positions at all time, would it be possible to even create an optimized planned economy.


> would it be possible

Apparently computer controlled central planning is performed by linear programming, so if you fix the information problem of knowing exactly what products there are, and the inputs and outputs of all practical ways of making them, the complexity is O(n^3.5) where n is the number of distinct goods.

I don't know if the economy is creating goods faster or slower than society's capability to solve LP problems is increasing.

http://crookedtimber.org/2012/05/30/in-soviet-union-optimiza...


Apparently, the Soviets were too, working on something like Cybersyn, but didn't complete it -- http://bit.ly/15br831 (ru.wikipedia.org)


The East German dissident Wolf Biermann characterized late Soviet practices as Stalinism with computers - while under Stalin you would get a bullet hole in your head, in later Socialism you would get a hole in your punch card (to mark you as a politically unreliable)

http://www.spiegel.de/spiegel/print/d-41843129.html

Now we all have our punch cards back ;-) I still wonder how it all came about: is it just as a preparation for possible future crisis ? are the rulers feeling really that insecure ? is it a real or an imaginary threat ? Isn't it that any power is always trying to extend its reach ? Many questions here.


Here is a British/US conspiracy version from the 1970's: http://www.syti.net/GB/SilentWeaponsGB.html

Caveat: The website in general has pretty "out there" ideas, and that I concluded from just the English versions (it is mostly French).


The author is on to something, but is too pessimistic. Big data is not a reprise of Gosplan. It is much better. It's the difference between Stalinism and the ability to optimize-out both the state and the 0.1%, and use all that capital and productivity in actual production. A kind of modern Bakunin-style economy.


(Automated) Gosplan was after Stalinism actually.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: