Last week I was interviewed about the Noisy Idiot Dilemma by Jerry Brito of George Mason University’s Mercatus Center for his podcast Surprisingly Free.  Listen here… Thank you @jerrybrito and @tatemwatkins!

From the Surprisingly Free website:

“Catherine White, graduate student at New York University’s Interactive Telecommunications Program, where she is researching productive participatory discussion, talks about her thesis on the Noisy Idiot Dilemma.  White explains the dilemma — how to foster productive online conversation when certain speakers exhibit noisy, unproductive, or unhelpful behavior — and discusses her research on various online forums, weblog comments, effects of humor, anonymity, and empathy online, and characteristics of elastic, oily conversation.”

I spoke on the Noisy Idiot Dilemma at Ignite NYC recently during Internet Week.  It was huge fun, slightly daunting: my not-very-eloquent first words as you’ll hear are “my god, there’s a lot of you…’  Thanks to all the Ignite NYC crew – I had a great time.

I’ve been redrafting and tweaking my thesis after handing in the initial draft for grading in May at ITP.  Here are the first two chapters – more to come soon. Below are the background and introduction.  Methodology, results and conclusions will be up here over the next few weeks.  I’d be really interested to hear your comments and feedback.  Thanks, Catherine.

The Noisy Idiot Dilemma:

How To Make Participatory Online Conversation Easier

Catherine White

Interactive Telecommunications Program

Tisch School of the Arts

New York University

catherine.white[at]nyu.edu

May 3, 2010

Abstract

How do we improve participatory online group collaboration? More specifically, how do we work together online in groups when there are noisy, dominant and often unhelpful people in the group? Especially when these people are not necessarily breaking forum rules, instead they often just make group collaboration and decision making difficult, if not impossible. This paper provides a high level overview of this disruptive behavior through the study of various online forums and proposes optimization methods for online group collaboration.

Keywords

online collaboration, decision making, participation, groups

1. Background

In March 2009, I wrote a mid-term paper [1] on this topic for the ITP Social Facts class. The paper was centered around the Debate Europe Forum. [2] The European Commission set up a forum in the attempt to engage European citizens in debate over several issues including climate change and energy, women and politics and the future of Europe. The main driver for setting up the site was a lack of interest in the EU elections in 2009. It turned out more British people voted in the UK reality TV show Big Brother than in the 1999 European elections. [3] As of February 28, 2010 the site has closed. [4] I spent time looking at a specific conversation within the Debate Europe forum, where people with loud voices on single issues had filtered into the discussion, the result being that the group was often sidetracked by this noisy behavior. A member of the forum attempted to engage the group in finding a solution to this issue, such as ignoring these people. However, this person was accused of stifling debate, and issues of free speech were called into play. In the end, the person who raised the concerns about this single issue, dominant, Noisy Idiot behavior left the group as a result.

The conclusion of the study into the attempts by the group as a whole to deal with Noisy Idiot behavior by members of the forum lead to a dissatisfactory conclusion: that democracy sucks. Why have open conversation and debate if this is the result? On further reflection, I realized that ‘Noisy Idiots’ and their impact on democratic decision-making is a huge dilemma. And yet so very important. [1] The idea that a group as a whole can find it hard, if not impossible to deal with this kind of behavior and for their discussion be completely derailed as a result is truly concerning. Especially given that online forums are the method my generation are increasingly turning to as a conversational arena.

In June 2009 I lead a seminar on the topic of the initial paper on Debate Europe at the NYC Participation Camp and received some great feedback, it sparked a interesting and enthusiastic debate, which continued online after the conference. In July 2009 Craig Newmark used the term ‘Noisy Idiots’ at the Personal Democracy Forum 2009 conference, this again sparked more debate and confirmed this was an issue that really bothered people. That’s fun. During the summer of 2009 I worked for a short while on the concept of creating online conversations and community as part of the team on the HerdictWeb project at the Berkman Center for Internet and Society. [5] We wanted to investigate if the level of participation in an online group was affected by remuneration of the participants in the group. This confirmed my interest in this type of subject matter and that I wanted to expand my study of conversation within online groups further.

Dealing with difficult people, yet not shutting them out is a really juicy issue: hard and tough, with concepts of the right to free speech and not silencing minority viewpoints coming to the front of the discussion. This project looks at existing theories in the field and analyzes current online conversation. It should be noted that this issue will never be completely fixed; there is no solution to a ‘problem’ here.  We can merely try to optimize.  This fact adds to the challenge to find ways to really help people to work together given that this dilemma will always be present due to the very nature of group conversation online.  One final note, this thesis analyzes the concept of ‘Noisy Idiot behavior’ – not ‘Noisy Idiots.’  There is a difference.  The term is meant to be provocative and memorable but in no way labels people, rather – behavioral traits.

2. Introduction

Online conversation is increasing. Fast. The rise in group discussion and collaborative decision making is evident in most online environments. These include attempts by corporations to build community around their brand or a product, [6] by nonprofits to raise awareness of causes, and in classrooms through the use of class wikis. The list is long and varied. However, compared to how long we’ve been conversing offline, quite simply – we haven’t been talking in groups online for very long. Our Roberts Rules of Order for the management of these online conversations is in its elementary stage compared to the hundreds if not thousands of years of experience of governing real-world conversations. And the ‘discussion around the dining table’ is different online, the parameters are different, and hence a different approach is needed.

2.1 Order Order, And Please Pass Me The Salt

The dynamics of conversation are fascinating: the ebb and flow of a discussion or debate, volume rising to reach a crescendo, peaking and then falling back as members reach a conclusion, the personalities in a conversation: the steerers, the silent listeners, the rowdy ones, the peacemakers, the controversy courters. The variety of conversational forums is breathtaking. Before even venturing into the online world, it is worth looking at this variety and diversity in the sort of places conversation and debate occurs in the physical world, in order to frame the further discussion in this paper:

i) Geography

The uninitiated Brit would likely keel over at the exuberance of some of the social interactions found in various Manhattan restaurants. At the dining table in London, politics, religion and money are strictly off limits for some, but go to dinner in any New York restaurant and you’ll hear the Mayor, Albany, Seder dinner and the tax season debated and dissected. However, there is an order to these discussions: people take social cues from others, and while the volume and temperature of the debate may rise; lively conversation and debate usually occur in a good natured and constructive manner.

ii) Age

The F bomb. If you’re under 30, chances are you’ll drop it in your social group without the need to peel someone’s eyebrows off the ceiling. Start using it in grandma’s knitting circle and chances are you’ll be told in no uncertain terms this is not an acceptable expression of feelings in that particular forum. Different age groups have different acceptable social practices.

iii) Voting Mechanisms

The corporate shareholder meeting is a classic example of decision making following informed discussion within large groups of people. All sorts of techniques have been honed for this over years of corporate jostling. Complex approaches to classes of shares, proxy systems, methods of dealing with vexatious shareholders have lead to a sophisticated decision making system dealing often with many different interests and agendas.

iv) Government

One of the best known forums for hearty, loud, challenging debate is the UK House of Commons. Prime Minister’s Question Time is noted throughout the world, not only as a place where the people (via their elected representatives) speak directly to power, but also – a place where often loud, noisy jeers are somehow parsed into constructive debate.

v) The Town Hall

Another example of people speaking directly to power is in the traditional town hall meeting, seen most recently during the 2008-2009 election campaign in the US.

vi) The Board Meeting

Hand in hand with the shareholder’s meeting, this is a type of group conversation that results in decision making, with a variety of rules and traditions governing how the meeting is structured.

vii) The Grad School Seminar

Through the management of a professor, lively classroom debate is kept on topic and time managed with the aim of letting everyone be heard – not just the noisy few.

Common to the above conversational forums are various techniques for effective group discussion to ensure orderly debate occurs, and without the discussion being derailed, for example by being overwhelmed by a noisy majority or minority; these must haves include -

i) An effective moderator;

ii) the ‘Collective Eye Roll;’

iii) an established and known format; and

iv) personal accountability.

2.2 The Variety Of Conversation Online

As seen in the above examples, our methods of keeping order and ensuring fair debate in real-world conversational sphere are pretty established. In addition to actual strategies for keeping order in group conversations, there are a variety of behavioral factors influencing the motivations of members in groups. For example, we are drawn to that which is similar: ‘birds of a feather flock together.’ In looking to join groups with those of similar sociodemographic, behavioral and intrapersonal characteristics, we seed groups in this manner. [7] The more you have these kinds of similarities in a group, the more likely it is the conversation will be less confrontational.

Another important concept to consider when looking at how we interact in groups is that of Axelrod’s Shadow of the Future. [8] According to the Prisoner’s Dilemma, if we interact just once, parties get the most reward by defection. However, if interacting more than once, there is a memory of past behaviors and it is of more benefit to the parties to behave in a cooperative manner. In groups, individuals gain most by playing fairly, it pays to care about your future interactions. [ibid.] The above concepts are factors to consider in online as well as offline group interactions, with specific issues in the application of these theories to online groups.

In beginning to look at the online conversational sphere, a veritable Pandora’s Box of separate issues is opened. Specifically, there is a disconnect, or a lagging behind between the development of techniques used to moderate discussion in the real world vs. conversations taking place online. This disconnect, can in some cases render participatory, collaborative group discussion (and in some cases, collaborative decision making) arduous, unproductive and inevitably, not fun. There is a need though to deal with this disconnect – because, as shown in the section below, we are increasingly collaborating online.

2.3 Why Is Online Conversation So Very Important?

With the rise in usage of social media tools by pretty much every type of organization: academia, corporations, nonprofits – to name a few, people expect to have a voice in a variety of arenas. This section outlines some success stories.

Corporations are tweeting [9] as a method of conversing with their customers and trouble shooting problems quickly. Blogging has become a natural partner to an corporate website; a place where news can be disseminated quickly, in perhaps a more informal and conversational manner. [10]

Facebook pages are appearing in the most unexpected places as a tool for engagement with an organization and the public. The US Department of State has been a real pioneer of this approach. Speaking in March 2010 at the Policy Making in the the Digital Age conference at Columbia University’s School of International and Public Affairs school (SIPA), [11] Kimberly Harrington of the State Department’s Office of Innovative Engagement spoke of the State Department’s successful use of Facebook in engaging with communities in other countries. As of March 2010, the State Department had over 180 Facebook pages for various embassies and consulates, 50 twitter accounts, 20 Flickr sites, 15 blogs and 45 YouTube sites. One specific example of conversation between an embassy and people using a Facebook site was by the US Embassy in Cairo. They started a Facebook group to engage with Egyptian students wanting to study in the US, specifically post 9/11. They set up a group called Study USA Egypt. It was deemed a good medium to reach people as Facebook usage is popular in Egypt (2 million Egyptians use it). Every day the embassy posts information and has committed to responding to queries within 24 hours of receiving them. Also posts are public so reach a wide audience. The page has over 8000 fans and the embassy reports an increase in applications for students to study in the US as a result of this. In addition the quality of applications has also increased due to students being better informed about the process.

These are great examples of organizations opening up a dialog, not just broadcasting to their audience.  These days, there is more of an expectation of this kind of conversation: to be able to engage with a company, government, museum – the list goes on.  Hence the importance of looking into how we can converse together online more effectively.

Footnotes

[1] Modifying an existing online group: European Union – Debate Europe or What to do about Noisy Idiots, White, C., March 2009

[2] Debate Europe, http://ec.europa.eu/archives/debateeurope/index_en.html, accessed April 18, 2010

[3] BBC Online, The EU’s Democratic Challenge, November 21, 2003, http://bbc.news.co.uk/2/hi/europe/3224666.stm, accessed April 18, 2010

[4] Debate Europe, http://ec.europa.eu/archives/debateeurope/index_en.htm, accessed April 18, 2010

[5] HerdictWeb, http://herdictweb.org, accessed June 6, 2010

[6] The Barbarian Group were hired by Kellogg to build an online community around their cereal brand Kashi, http://www.barbariangroup.com/portfolio/kashi_com, accessed April 11, 2010

[7] McPherson, M. et al., Annu. Rev. Sociol. 2001. 27:415-44

[8] Axelrod, Robert; Hamilton, William D. ‘The Evolution of Cooperation‘, Science 211: 1390–96

[9] Jetblue Engages in Real Conversation on Twitter, Socialized Blog, March 18, 2009, http://bit.ly/1k1mVj, accessed April 10, 2010

[10] Edelman and Intelliseek, Talking From The Inside Out – The Rise of Employee Bloggers, Fall 2005, http://bit.ly/dzZvmS, accessed March 20, 2010

[11] The Morningside Post, Blip TV channel (Kimberly Harrington), http://themorningsidepost.blip.tv/, accessed March 20, 2010

I’ll be speaking on my ITP thesis “The Noisy Idiot Dilemma” at Ignite NYC IX on Wednesday as part of NYC’s Internet Week.  There is such an exciting and vibrant tech community here in NYC at the moment, its great to be part of it.

It is my first speaking gig on my thesis since I presented it at ITP (video here).  I’ll be speaking alongside some people I really admire including the Director of Expert Labs – Anil Dash and Smith Magazine’s Larry Smith.  The magic of Ignite is in its format: 5 minutes per speaker, 20 slides.  That’s one way to really summarize your year-in-the-making-thesis…

I’ll be speaking about how we can make online group discussion easier, specifically how we can deal with noisy, disruptive behavior in the online conversation sphere.  The results surprised me, I’ll be really interested to hear your thoughts.

Details:

Ignite NYC IX – Internet Week

RSVP here

Doors open at 7 pm, talks begin at 8 pm

Internet Week HQ, The Metropolitan Pavilion, 125 W. 18th Street at 6th Avenue.

More on Ignite NYC

More on NYC Internet Week

More on The Noisy Idiot Dilemma

Last week was ITP’s annual Thesis Week.  Every member of  the Class of 2010 presented their thesis projects with a Q&A session.  The faculty sat through five days of presentations, from midday to sometimes as late as 9 pm.  Its a real ITP rite of passage – the culmination of 2 years of exhausting late nights and long, fun, collaborative days.

I presented The Noisy Idiot Dilemma.  The paper is currently being graded and will be posted soon.

In the meantime, here is the video of me presenting the Noisy Idiot Dilemma at ITP:

A catch-up on recent Noisy Idiot Dilemma progress.

Week 7:

Midterm presentations this week to outside reviewers Lauren Marrus (Dempsey & Carroll)  and Eleanor Powers (Powers Media & Entertainment Consulting, LLC).  The task was to pitch to someone who aside from a couple of sentences in an introductory email hadn’t heard of your project.  Feedback was surprising: “this is such a huge, huge problem, I just can’t even begin to see how you’re going to solve it.’  Takeaway from that: super glad to hear it’s thought of as a huge problem, I’ve been told the same thing by many others, but good to hear it again.  As for the second part – it’s impossible to solve, but by studying these groups, I hope to at least propose some steps to optimize parameters to enable less disruptive online discussions leading to more effective group decision making.

This week was also hugely useful to see people’s reactions to your work for the first time – need to work on better slides.

Week 8:

Tackled the ‘better slides’ issue.  Samples here:

The two slides above show the current color scheme – feedback was positive in class, as ever – I very much appreciate all feedback on this.  Any suggestions gratefully received.

I used the rest of my presentation time to show the class the ten groups I am focusing my research on:

• Debate Europe, European Union, http://europa.eu/debateeurope/index_en.htm (archived February 28, 2010)

• Urban Baby, http://urbanbaby.com

• World of Warcraft Universe Guide (WoWWiki), http://www.wowwiki.com

• The Coffee Party USA, http://www.coffeepartyusa.com

• Gossip Rocks, http://www.gossiprocks.com/

• Wall Street Journal Community, http://www.wsj.com/community

• The Student Room, http://www.thestudentroom.co.uk

• Linux Questions, http://linuxquestions.org

• Topix Pasadena Star-News Forum, http://www.topix.com/forum/source/pasadena-star-news
  

I included some screen shots of some of the online conversations to show the class the kinds of interactions I’m studying.  Collective Storytelling class last year really hit home with the point that when you are doing a long term project (eg a semester in length) reporting back to the same group of people each week, its important to bring them with you, to keep them appraised, and to make it interesting for them.  That way the class interactions and discussions are more fulfilling and useful.  Its a challenge when you are so absorbed in something to explain it each week in an innovative way – but its a good skill to work on.

Week 9

Spending time this week working on the 5-page mandatory thesis report that each of us have to submit.  Sent first draft to Katherine for comment.

In addition, useful feedback from Thomas last week prompted me to really consider the scope of what Noisy Idiot behavior comprises.  Its not trollish behavior, rather – its behavior outside a social norm, that while still within the bounds of group behavior makes others uncomfortable and is disruptive.  Usually personal attacks are considered trollish, so what about attacks to people’s arguments. Whitehouse2.org makes the difference between the two, the former is not ok, the latter is.  From their rules page (http://www.whitehouse2.org/about/rules) :

‘What is a personal attack?

This is okay: “That’s a dumb idea.”

This is not: “You’re an idiot for suggesting such a dumb idea.”

See the difference? You can say an idea or proposal is a bad idea (hopefully more eloquently than simply “that’s dumb”), but it’s not okay to attack a person.’

So often though, there is a blurry line between the two.  When I observed the Noisy Idiot behavior in the Debate Europe forum last year, it was impossible to uncouple the two.  This research therefore, will not be based on a black and white set of criteria of what constitutes Noisy Idiot behavior and not.  Instead it will be a loose fitting trending and analysis of some basic Noisy Idiot behavior characteristics:

• loud

• upsets others

• not so egregious though to be kicked out as a troll

• quick to attack

• single issue focused

Hope this clarifies matters.

To conclude this week’s round-up, huge chunks of work at the moment is simply detailed conversation analysis, which isn’t ready to be posted yet – suffice to say, am really enjoying being completely absorbed in these online groups.

Finally – the Noisy Idiots Ning group has been hit by uber huge amounts of spammy comments, I haven’t had time to clear, so please excuse and I’ll let you know when its back on form.

Thanks.  See you next week. Catherine

Seeking Advice…

As part of preparations for our upcoming midterm presentations on Monday March 8, we were instructed to speak to three people who were knowledgeable in our subject area.  I had already spoken to Clay a few weeks ago in detail to discuss the outline of the project.  Last weekend I ran Tom Glaiyser at the Digital Media and Policy conference at Columbia’s School of Public Policy.  Tom is a Knight Media Policy Fellow at the New America Foundation with expertise in media policy reform.  We met when I first presented Noisy Idiots at NYU’s Participation Camp at ITP last summer.

I asked him about:

a) people/papers on analysis of group conversations; and

b) possible research methodologies I should be implementing to investigate Noisy Idiot behavior on the 10 websites I am studying.

Tom came back with a raft of great resources to look at including:

• Marc Smith (newsgroup analysis) – specifically, I’m getting hold of his Communities in Cyberspace book.
• Warren Sack (known for his work on the analysis of very large scale conversations (VLCS), I’m going to begin with his paper on discussions in an Open Source Software community)
• Martin Wattenberg (Visual Communication Lab at IBM Research) and Fernanda Viegas’ work on Wikipedia

In addition Tom gave some helpful suggestions on how to analyze behavior in online conversations, he also mentioned already existing tools such as DISQUS and INTENSE DEBATE, and that the slashdot/DailyKos style of moderation is more widely available.

Thanks Tom!

Places to look for discussion

Thank you thesis class for the suggestions today in class for where to look for lively discussion.  This week I’ll be looking at:

• DMCworld.com – discussions around DJ-ing and the World DJ Championships
• World of Warcraft forums – worldofwarcraft.com
• Additionally – World of Warcraft wikis: wowwiki.com – information on WoW
• Urban Baby and Park Slope Parents
• 4Chan – does anonymity result in Noisy Idiot behavior or lead to trolling?
• Chowhound.com for foodie discussion
• Stuy Town/Peter Cooper Village/Atlantic Yards – sites discussing proposed real estate development issues/residents committees
• Streetsblog.org
• Victims of Bernie Madoff groups
• Wall Street bonuses discussion groups.

There’s a lot to be looking at there, and am now confident the 10 groups I’ll be studying will be suitably diverse in their subject matter.  The main issue I had last week was finding groups with lively discussions that were not just groups about political issues.  In addition there is enough variety in the level of moderation of these site for my studies.  Final list coming and a research plan coming soon….

Week 3 – Discussion with @cshirky

This week I discussed with @cshirky my concept to attempt to undertake some actual experiments and testing for this project.  I realized having spoken to him, that this was quite an ambitious idea to execute in just 12 weeks.  It takes longer than this for behavior to change, and I need time to research tweaks, ideas, concepts to integrate into already-existing group rules, before I even reach the testing stage.

Instead, he suggested picking 10 diverse discussion groups/forums.  Observe them, find trends, and look for patterns.  Having done that, I’d be in a position to present a detailed study of behavior patterns within online groups.  This idea appeals to me, specifically choosing 10 diverse groups, my aim is present research that people from many different sectors would find useful.  Also – from a interest point of view, it makes it fascinating for me to research.  Feedback from the class indicated this was a good change in tack, with the additional suggestion of finding a ‘control group’ where there is lively debate, discussion, decision making that takes place successfully.

Week 4

I took the NY State Bar Exam this week – sadly no Noisy Idiots time, just torts, evidence, crim pro etc.

Week 5

The selection of ten groups to study is well underway and will be presenting them in class tomorrow. The list is subject to change.  I want to select groups from a variety of subject areas such as politics, music and the arts.  I also attempted to pick groups with a variety of control mechanisms – some requiring registration, some heavily moderated – others not.  Any suggestions for other groups to study would be very much appreciated.

Tea Party Patriots

This is a fascinating group to study, the Tea Party is in the press a lot recently following Sarah Palin’s appearance at National Tea Party Convention in early February.  According to an article in the Economist on February 11, where “Much of the Nashville event was devoted to teaching the fired-up newbies practical skills, such as using Facebook and Twitter to spread the word, raise money and get out the vote … “  I anticipate lots of group discussions aimed at working together, to make decisions and reach consensus from grass roots Tea Party Groups.

Debate Europe

I want to revisit this group I studied initially for the first Noisy Idiots Dilemma paper.  Specifically because it is an example of a heavily moderated forum, with strict user registration requirements, with a substantial amount of single-issue Noisy Idiot behavior.  I’d also like to see if anything has changed there since this time last year.

Music Discussion

An example of where people seem to get along – a harmonious community, how do they do it?  Is it because subject matter is really not contentious?  I’d like to explore further.

White House 2

Jim Gilliam’s site, with very novel ways of optimizing group discussion, an example of people attempting to reach consensus on issues, with very effective moderation techniques – having spoken to Jim he’s a knowledgeable resource on dealing with Noisy Idiots.

Not Yet Ascertained Open Source Program Forum

I’m currently reading Steven Weber’s The Success of Open Source, and very much want to study an open source programing forum, however I need some advice from some techie-er members of the class than me,  @thomasrobertson, will be speaking to you tomorrow!  Open source program communities are quite unique in how they work together to build something and refine it – and they evolve in ways that are fascinating in terms of group interaction online.  They’ve also been around for a while.  This linux forum is an example.

Final thoughts:

In the next week, I anticipate finalizing the ten groups and complete the proposed analysis model to use.

This week we were tasked with reviewing the thesis proposals of two other people in the class.

My “Two for Review” were Fiona’s concept for interactive fiction and Thomas’ Banyan Speak, a system to allow comments to traverse between applications.  At this stage, being in a class with such variety of ideas doesn’t fox me, but it does make me sit up for a second or two and wonder how this will affect our concept development?  ITP is such a truly interdisciplinary environment.  I’ve written a thesis before – using a program that models the dehydration of rocks.  It was such a great experience working in a lab.  Seems years ago. I was in a Chemistry department, with physical chemists.  We were all chemists or programmers though, all doing the same sort of work.

This thesis is different, and I wonder if a large part of this will be due to just how our ideas will be affected by each other?  Fascinates me.