NOW AVAILABLE The draft of my book on Organizational Intelligence is now available on LeanPub http://leanpub.com/orgintelligence. Please support this development by subscribing and commenting. Thanks.

Saturday, February 4, 2017

The Art of the New Deal - Trump and Intelligence

In his 1967 book on Organizational Intelligence, Harold Wilensky praised President Roosevelt for maintaining a state of creative tension in the US administration. Wilensky reckoned that this enabled FDR to get a more accurate and rounded account of what was going on, and gave him some protection against the self-delusion of each department.

(In FDR's time, of course, it was considered entirely normal for an administration to be staffed by a bunch of white men with similar education. And yet even they managed to achieve some diversity of perspective.)

Early reports of Donald Trump's administration suggest an unconscious echo of the FDR style. Or perhaps a much earlier pattern.

At the center of it all has been a cast of characters jockeying for Trump’s ear, creating a struggle for power that has manifested in a mix of chaos, leaks and uncertainty. The Trump White House already bears more resemblance to the court of a Renaissance king than to most prior administrations as favorites come and go, counselors quarrel over favor and policy decisions are often made by whim or without consultation. (Guardian, 4 Feb 2017)

But it is difficult to see this as "creative tension" resulting in an "accurate and rounded" view.
“Trump thinks he’s invincible,” says Hemmings, who doubts whether his advisors will ever question or criticise him. “Usually leaders choose the people around them to keep them in check, and Trump needs people to temper his hotheadedness and aggression. Instead, he’s picked advisors who worship him.” (Independent, 2 Feb 2017)

Wilensky's book also discusses the dangers of a doctrine of secrecy.

Secrets belong to a small assortment of individuals, and inevitably become hostage to private agendas. As Harold Wilensky wrote “The more secrecy, the smaller the intelligent audience, the less systematic the distribution and indexing of research, the greater the anonymity of authorship, and the more intolerant the attitude toward deviant views.” (Gladwell 2010)

And secrecy seems to a key element of the Trump-Bannon modus operandi.
“These executive orders were very rushed and drafted by a very tight-knit group of individuals who did not run it by the people who have to execute the policy. And because that’s the case, they probably didn’t think of or care about how this would be executed in the real world,” said another congressional source familiar with the situation. “No one was given a heads-up and no one had a chance to weigh in on it.” (Politico 30 Jan 2017)


But perhaps in reaction to the Bannonite doctrine of secrecy, there has been a flood of leaks from inside the administration. Chris Cillizza suggests two possible explanations - either these leaks are intended to influence Trump himself (because he doesn't take anything seriously unless he hears it from his favourite media channels) or conversely they are intended as a kind of whistle-blowing.


Marx thought that history repeated itself. (Alarmingly, Trump's Counselor Steve Bannon adheres to the same view.) So are we into tragedy or farce here?




Rachael Bade, Jake Sherman and Josh Dawsey, Hill staffers secretly worked on Trump's immigration order (Politico, 30 Jan 2017)

Chris Cillizza, The leaks coming out of the Trump White House cast the president as a clueless child (Washington Post, 26 January 2017), The leaks coming out of the Trump White House right now are totally bananas (Washington Post, 2 Feb 2017)

Malcolm Gladwell, Pandora's Briefcase (New Yorker, 10 May 2010)

Rachel Hosie, The deeper reason we should be worried Donald Trump hung up on Australia PM Malcolm Turnbull (Independent, 2 Feb 2017)

Linette Lopez, Steve Bannon's obsession with a dark theory of history should be worrisome (Business Insider, 2 Feb 2017) HT @BryanAppleyard

Carmen Medina, What is your Stupification Point? (6 May 2010)

Joseph Rago, History Repeats as Farce, Then as 2016 (Wall Street Journal, 4 November 2016) paywall

Sabrina Siddiqui and Ben Jacobs, Trump's courtiers bring chaotic and capricious style to White House (Guardian, 4 February 2017)



Related posts

Puzzles and Mysteries (January 2010)
Enemies of Intelligence (May 2010)
Delusion and Diversity (October 2012)

Thursday, November 3, 2016

Pay as you Share

Announced and rapidly withdrawn, Admiral's proposed collaboration with Facebook was supposed to give drivers a discount on their car insurance premiums if their Facebook posts indicated the right kind of personality. According to some reports, the idea was that people who were reckless with punctuation (too many exclamation marks, not enough full stops) might also be reckless in their driving habits.

The punctuation example is probably a red herring. The analysis of personality will undoubtedly be based on much richer aspects than mere punctuation: Facebook is capable of much more sophisticated analysis, as well as selling data to other organizations for the same purpose.

For example, a Korean study in 2013 found that Facebook activities had predictive power in distinguishing depressed and nondepressed individuals. However, Facebook may not wish to draw too much public attention to such capabilities. (There are some important ethical issues in the use of algorithms to predict mental health issues, for example in recruitment screening, discussed at length by Cathy O'Neil.)

Meanwhile, insurance companies will wish to use any information and insight they can get their hands on, to try and calculate risk more accurately. People may consent to sharing their data if they feel they will benefit personally, or if they are unaware of the possible data uses and implications, but that could just result in discrimination against the people who refuse to share their data. So privacy campaigners may not be reassured by the fact that this particular collaboration has been withdrawn.



Cathy O'Neil, How algorithms rule our working lives (Guardian, 1 Sept 2016)

Sungkyu Park et al, Activities on Facebook Reveal the Depressive State of Users (J Med Internet Res. 2013 Oct; 15(10): e217)

Graham Ruddick, Admiral to price car insurance based on Facebook posts (Guardian, 2 November 2016, 00.01 GMT)

Graham Ruddick, Facebook forces Admiral to pull plan to price car insurance based on posts (Guardian, 2 November 2016, 18.41 GMT)


Related posts
Weapons of Math Destruction (Oct 2016)

Monday, October 31, 2016

The Transparency of Algorithms

Algorithms have been getting a bad press lately, what with Cathy O'Neil's book and Zeynap Tufekci's TED talk. Now the German Chancellor, Angela Merkel, has weighed into the debate, calling for major Internet firms (Facebook, Google and others) to make their algorithms more transparent.

There are two main areas of political concern. The first (raised by Mrs Merkel) is the control of the news agenda. Politicians often worry about the role of the media in the political system when people only pick up the news that fits their own point of view, but this is hardly a new phenomenon. Even in the days before the Internet, few people used to read more than one newspaper, and most people would prefer to read the newspapers that confirm their own prejudices. Furthermore, there have been recent studies that show that even when you give different people exactly the same information, they will interpret it differently, in ways that reinforce their previous beliefs. So you can't blame the whole Filter Bubble thing on Facebook and Google.

But they undoubtedly contribute further to the distortion. People get a huge amount of information via Facebook, and Facebook systematically edits out the uncomfortable stuff. It aroused particular controversy recently when its algorithms decided to censor a classic news photograph from the Vietnam war.

Update: Further criticism from Tufekci and others immediately following the 2016 US Election


The second area of concern has to do with the use of algorithms to make critical decisions about people's lives. The EU regards this as (among other things) a data protection issue, and privacy activists are hoping for provisions within the new General Data Protection Regulation (GDPR) that will confer a "right to an explanation" upon data subjects. In other words, when people are sent to prison based on an algorithm, or denied a job or health insurance, it seems reasonable to allow them to know what criteria these algorithmic decisions were based on.

Reasonable but not necessarily easy. Many of these algorithms are not coded in the old-fashioned way, but developed using machine learning. So the data scientists and programmers responsible for creating the algorithm may not themselves know exactly what the criteria are. Machine learning is basically a form of inductive reasoning, using data about the past to predict the future. As Hume put it, this assumes that “instances of which we have had no experience resemble those of which we have had experience”.

In a Vanity Fair panel discussion entitled “What Are They Thinking? Man Meets Machine,” a young black woman tried unsuccessfully to explain the problem of induction and biased reasoning to Sebastian Thrun, formerly head of Google X.
At the end of the panel on artificial intelligence, a young black woman asked Thrun whether bias in machine learning “could perpetuate structural inequality at a velocity much greater than perhaps humans can.” She offered the example of criminal justice, where “you have a machine learning tool that can identify criminals, and criminals may disproportionately be black because of other issues that have nothing to do with the intrinsic nature of these people, so the machine learns that black people are criminals, and that’s not necessarily the outcome that I think we want.”
In his reply, Thrun made it sound like her concern was one about political correctness, not unconscious bias. “Statistically what the machines do pick up are patterns and sometimes we don’t like these patterns. Sometimes they’re not politically correct,” Thrun said. “When we apply machine learning methods sometimes the truth we learn really surprises us, to be honest, and I think it’s good to have a dialogue about this.”

In other words, Thrun assumed that whatever the machine spoke was Truth, and he wasn't willing to acknowledge the possibility that the machine might latch onto false patterns. Even if the algorithm is correct, it doesn't take away the need for transparency; but if there is the slightest possibility that the algorithm might be wrong, the need for transparency is all the greater. And evidence is that some of the algorithms are grossly wrong.


In this post, I've talked about two of the main concerns about algorithms - firstly the news agenda filter bubble, and secondly the critical decisions affecting individuals. In both cases, people are easily misled by the apparent objectivity of the algorithm, and are often willing to act as if the algorithm is somehow above human error and human criticism. Of course algorithms and machine learning are useful tools, but an illusion of infallibility is dangerous and ethically problematic.



Rory Cellan-Jones, Was it Facebook 'wot won it'? (BBC News, 10 November 2016)

Ethan Chiel, EU citizens might get a ‘right to explanation’ about the decisions algorithms make (5 July 2016)

Kate Connolly, Angela Merkel: internet search engines are 'distorting perception' (Guardian, 27 October 2016)

Bryce Goodman, Seth Flaxman, European Union regulations on algorithmic decision-making and a "right to explanation" (presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY)

Mike Masnick, Activists Cheer On EU's 'Right To An Explanation' For Algorithmic Decisions, But How Will It Work When There's Nothing To Explain? (Tech Dirt, 8 July 2016)

Fabian Reinbold, Warum Merkel an die Algorithmen will (Spiegel, 26 October 2016)

Nitasha Tiku, At Vanity Fair’s Festival, Tech Can’t Stop Talking About Trump (BuzzFeed, 24 October 2016) HT @noahmccormack

Julia Carrie Wong, Mark Zuckerberg accused of abusing power after Facebook deletes 'napalm girl' post (Guardian, 9 September 2016)

New MIT technique reveals the basis for machine-learning systems’ hidden decisions (Kutzweil News, 31 October 2016) HT @jhagel

Video: When Man Meets Machine (Vanity Fair, 19 October 2016)

See Also
The Problem of Induction (Stanford Encyclopedia of Philosophy, Wikipedia)


Related Posts
The Shelf-Life of Algorithms (October 2016)
Weapons of Math Destruction (October 2016)

Updated 10 November 2016

Monday, October 24, 2016

Learning at the Speed of Learning

According to a recent survey by McKinsey,  "the great majority of our respondents expect corporate learning to change significantly within the next three years".

It seems that whatever the topic of the survey, middle managers and management consultants always expect significant change within the next three years, because this is what justifies their existence.

In this case, the topic is corporate learning, which McKinsey recommends should be done "at the speed of business", whatever that means. (I am not a fan of the "at the speed of" cliche.)

But what kind of change is McKinsey talking about here? The article concentrates on digital delivery of learning material - disseminating existing "best practice" knowledge to a broader base. It doesn't really say anything about organizational learning, let alone a more radical transformation of the nature of learning in organizations. I have long argued that the real disruption is not in replacing classrooms with cheaper and faster equivalents, useful though that might be, but in digital organizational intelligence -- using increasing quantities of data to develop and test new hypotheses about customer behaviour, market opportunities, environmental constraints, and so on -- developing not "best practice" but "next practice".



Richard Benson-Armer, Arne Gast, and Nick van Dam, Learning at the speed of business (McKinsey Quarterly, May 2016). HT @annherrmann

Chris Argyris and Donald Schön, Organizational Learning: A Theory of Action Perspective. Reading, MA, Addison-Wesley, 1978.

Monday, August 29, 2016

The Judgment of whole Kingdoms and Nations

@Cybersal @kirstymhall @UKParliament #Brexit #VoxPopuli


A radical Whig tract was published in 1709 under the title Vox Populi, Vox Dei. The following year, an extended version was published under the title The Judgment of whole Kingdoms and Nations. I want to use these two phrases as the starting point for my submission to the UK Parliament Public Administration and Constitutional Affairs Committee, which has launched an inquiry into the lessons that can be learned for future referendums.

The first thing I want to mention is the rushed timescale. The inquiry was announced on July 14th, with a deadline for submissions of September 5th. I shall argue that this rushed timescale is symptomatic of the referendum itself, in which people were asked to make a complex decision with inadequate information and analysis.

(For the sake of comparison, an inquiry on the future of public parks was announced on July 11th, with a deadline of September 30th. So we are given more time to analyse the physical swings and roundabouts of council-run playgrounds than the metaphorical swings and roundabouts of parliamentary sovereignty and media oversight.)

To be fair, most parliamentary inquiries only give you weeks rather than months to compose a submission. This effectively limits submissions to people who have already formed an opinion, and already have the evidence to support this opinion. In other words, experts.

But then most parliamentary inquiries are about issues that people have been concerned about for a much longer time: Bus Services, Employment Opportunities for Young People, Food Waste. There is an existing body of knowledge relating to each of these topics, and it is not unreasonable to ask people to base a submission on their existing knowledge.

In contrast, nobody knew precisely how this referendum was going to be mismanaged until it actually happened. Although many people (including some Brexiteers as well as many Remainiacs) predicted that it would end in tears, and can now say "we told you so".

I can read you like a magazine ... Don't say I didn't say I didn't warn you (Taylor Swift)

No doubt the Select Committee can expect to receive a number of submissions that fall under the heading of what the Dictionary of Business Bullshit calls "Pathologist's Interest".



But told-you-so is not a good starting point for a proper analysis, because it concentrates on confirming one's previous expectations, rather than discovering new patterns. So the Select Committee might not get much well-grounded analysis. Partly because there isn't time to do it properly, and partly because many of the potential "experts" are affiliated to UK universities, which are currently on summer vacation. Looks like the Select Committee is falling in line with Michael Gove's idea that "the people in this country have had enough of experts".

("The Voice of Gove is the Voice of Government". I wonder what that would look like in Latin?)

The official announcement sidesteps from "the lessons that can be learned" to "lessons learned", which is not the same thing at all. The former suggests an open exploration, while the latter suggests merely rattling through a project postmortem for form's sake. The timescale does not seem particularly conducive to the former. So is this apparent haste triggered by thoughts of a second referendum, or it is just intended to curtail criticism of Parliament for its earlier folly?

As I have argued elsewhere (including my book on Organizational Intelligence) complex sense-making and decision-making cannot go straight from the Instant of Seeing to the Moment of Concluding, but require what Lacan calls Time for Understanding. In this respect, the inquiry repeats one of the errors of the referendum itself.

The timescale and debating rules for the Brexit referendum were modelled on a General Election campaign. But a General Election has three important characteristics that were absent from Brexit. Firstly the electorate is generally familiar with the main parties: Labour and Conservative were around before any of us were born, and the Lib Dems also have long-established roots. Secondly, there is some rough notion of symmetry between the two main parties. Thirdly the parties make promises to which they will be held accountable in the event of victory. In other words, the General Election campaign can be compressed into a matter of weeks precisely because the rules of engagement are broadly understood, and there is very little new material for the electorate to process.

In comparison, as Kirsty Hall argues, the referendum for Scottish Independence was given a lengthy period of debate and analysis, because of the perceived complexity of the issues that needed to be considered. This would have been a much better model for the Brexit referendum.

Finally, let me return to the phrase "the judgement of whole kingdoms and nations", which of course raises the prickly subject of sovereignty. Although we supposedly have a system of parliamentary sovereignty in this country, parliament occasionally permits the voice of the people to be heard. As the Latin phrase has it, The Voice of the People is the Voice of God; and as the Establishment has discovered, the People can be a vengeful God. Parliament is still learning to listen to this vengeful voice. But who will teach what these lessons mean, and in what timescale? Or will the Establishment just adopt a Brechtian solution?



Update: Since I wrote this post, the Electoral Reform Society has published a critical report on the Brexit referendum, which makes the same unfavourable comparison with the Scottish Independence referendum that Kirsty made back in June. The Society has confirmed that it will be making a submission to the Parliamentary Inquiry.



Will Brett, Doing Referendums Differently (Electoral Reform Society, 1 September 2016)

Kirsty Hall, Brexit was a Con (28 June 2016) HT @cybersal @MerrickBadger @Koann

UK Parliament: Future of Public Parks, Lessons Learned from the EU Referendum

Wikipedia: Vox Populi, Vox Dei 


Wednesday, August 3, 2016

Political parties and organizational intelligence 3

According to Wikipedia, a party leader is the most powerful official within a political party. I think this statement is debatable. Party leaders in recent history have had varying degrees of power and influence over their own party members, let alone the wider political system.

Writing in The Atlantic during the election campaign, @jon_rauch expressed strong opposition to the conventional view of party leadership.
The very term party leaders has become an anachronism. ... There no longer is any such thing as a party leader. There are only individual actors, pursuing their own political interests and ideological missions willy-nilly, like excited gas molecules in an overheated balloon. ...
This is not only a problem of leadership and individual agency, but also a question of the nature of the political party as a viable system with collective agency and intelligence. Rauch continues
The political parties no longer have either intelligible boundaries or enforceable norms.

The relationship between the politician and the party has always been problematic - consider Winston Churchill who changed party allegiance twice before becoming party leader. But the root cause of this problem is unclear.






Jonathan Rauch, How American Politics Went Insane (The Atlantic, July 2016)

Wikipedia: Party Leader (retrieved 4 Feb 2017)


Related posts

Political parties and organizational intelligence 1 (May 2012)
Political parties and organizational intelligence 2 (June 2015)

Friday, July 15, 2016

Boughing to the Inevitable

What is the best time to plant a tree?

A popular answer to this question is that the best time to plant a tree is twenty years ago, and the second-best time is now.

This is often claimed to be an ancient Chinese proverb. Or an African proverb. It is unlikely to be either of these.

And obviously we are not supposed to take this proverb literally. Because if the best time was twenty years ago, the second-best time would be nineteen years ago.

But instead of interpreting this logically, we are presumably supposed to interpret it as a motivational statement. Don't waste time regretting that you didn't plant a tree twenty years ago, act now to make sure you don't have similar regrets in twenty years' time. (Do real Chinese proverbs do motivational statements? I suspect not.)

In his new book, The Inevitable, Kevin Kelly talks about the opportunities for internet entrepreneurs thirty years ago. "Can you imagine how awesome it would have been to be an ambitious entrepreneur back in 1985 at the dawn of the internet?"

He then looks forward to the middle of the century. "If we could climb into a time machine, journey 30 years into the future, and from that vantage look back to today, we’d realize that most of the greatest products running the lives of citizens in 2050 were not invented until after 2016."

In other words, for an internet start-up the second-best time is now.



By the way, I'm not the first person to use the pun about 'boughing' to the inevitable. For example, @rcolvile used it in the context of ash dieback. "Half the trees in the country were going to be torn down. He’d already had to veto a particularly insensitive press release describing him as 'ashen-faced' about the situation, but 'boughing to the inevitable'. Meanwhile, Google is asking me if I meant 'coughing to the inevitable'. Thanks Google, it's always useful to spot something you haven't yet mastered.


KK.org, The Inevitable
Kevin Kelly, The Internet Is Still at the Beginning of Its Beginning (Huffington Post, 6 June 2016)

On The Best Time to Plant a Tree (Reddit)

Robert Colvile, Friends: The One with the Guy in a Yellow Tie (Telegraph, 3 November 2012)