The largest issue, to me, is accessibility. Obviously the first step to making data accessible is making it available – getting it out of government hands and into the public’s, whether through open data initiatives, FOIA requests, or asking very nicely. But how that information is made available very much impacts who will access it:

Not all obstacles are the result of malevolence, but they diminish accessibility just the same. Mike Morisy of Muckrock often talks about how they’ll request digitized information – databases, emails – and receive it printed on thousands of pages of paper. PDFs aren’t much better. At our last open government meetup we spent an hour debating the best text recognition program for searching meeting minutes and proposed regulations only available in PDF format. Non-programmers may face the opposite problem. I know plenty of people who’d be confounded by a CSV file, nevermind the raw contents of an Access or mySQL database. They’d be happy to get their information on printed paper or in PDF files. What works best for one person with one set of skills will be a constant frustration for another.

Once you’ve got the data in your preferred format, you need to have the the training to manipulate it, to have/know software programs like Excel or Calc or SPSS or JMP or scripting languages like R or Matlab or Octave. And you’ll need to understand at least some statistics – no simple feat when people who do analysis for a living often fall prey to common mistakes. Not to mention the healthy amount of civic literacy necessary to understand the meaning behind the numbers: how bill amendment works, or how federal contracts are awarded, or how the FDA’s clinical trial system works.

Accessibility isn’t simple. But it isn’t something we can ignore either, not if we want to be truthful when we say that we’re advocating for better data access for all. As transparency bloggers have talked about before, efforts to increase transparency can have unexpectedly oppressive effects:

A very interesting and well-documented example of this empowering of the empowered can be found in the work of Solly Benjamin and his colleagues looking at the impact of the digitization of land records in Bangalore. Their findings were that newly available access to land ownership and title information in Bangalore was primarily being put to use by middle and upper income people and by corporations to gain ownership of land from the marginalized and the poor. The newly digitized and openly accessible data allowed the well to do to take the information provided and use that as the basis for instructions to land surveyors and lawyers and others to challenge titles, exploit gaps in title, take advantage of mistakes in documentation, identify opportunities and targets for bribery, among others. They were able to directly translate their enhanced access to the information along with their already available access to capital and professional skills into unequal contests around land titles, court actions, offers of purchase and so on for self-benefit and to further marginalize those already marginalized.

The digital divide exists within the United States as well. Last year Boston unveiled a new initiative called Street Bump, the goal of which was to map pot holes by collecting and analyzing accelerometer data from smartphones. My friends and I were eager to join in the effort, until a friend pointed out that smartphone users were likely to live in – and therefore travel over the pot holes of – mostly well off neighborhoods. And in a session at TCamp, an activist (whose name unfortunately I didn’t catch) pointed out how increased access to crime rate data and the creation of apps like CrimeReports has the potential to stigmatize and further disadvantage poorer neighborhoods.

Which brings me to another issue I’ve been wrestling with – how can you build a narrative while maintaining accuracy? Data is, by itself, meaningless. Take this table:

Group A	0	0	1	0
Group B	1	1	0	1

What does it mean? Is it a count of men and women at a particular event? Is it a record of coin flips? Is it the political affiliation of people being canvassed at a sporting event? Even if we label our variables – say, assigning “Group A” as men and “Group B” as women, we’re still creating a narrative. We’re implying that men and women are the only two possible categories. We’re assuming that our sample is representative. We’re asserting that sex ratios at this event are a topic worth considering, if only for a moment. And this with a set of only four data points!

More concretely, let’s look at a semi-random dataset from Data.gov, labeled EPA Toxics Release Inventory Program. It’s a dataset with a few dozens columns and nearly two million rows – there’s no way a human mind could understand this holistically. We have to group the data together some way, maybe by location, or by the parent company, or maybe by whether the parent company was military, or whether the released toxic was a carcinogen. And as we organize, stories emerge. Maybe we see that toxics are disproportionately released in southern states, or that the vast majority of toxics are released by the military (or vice versa – I have not actually analyzed this data.) These are good and useful stories but they come at a price: lost nuance. A quick skim of this EPA page suggests that facilities do not have to report toxics release if it is under a certain amount per year. So we can’t necessarily say release is greater in southern states – only releasing by larger facilities. What are the potencies of the various carcinogens? If we say that some areas or companies or industries release “more carcinogens” than others, we may be misleading, if others are releasing small amounts of much more hazardous materials.

This might seem like nit-picking. In many ways it is. But the language that some transparency advocates use worries me. From the Data Journalism Handbook, which was released freely and very recently – on the first day of TCamp, as it happens:

Data analysis can reveal “a story’s shape” (Sarah Cohen), or provides us with a “new camera” (David McCandless). Using data the job of journalists shifts its main focus from being the first ones to report to being the ones telling us what a certain development might actually mean. The range of topics can be far and wide. The next financial crisis that is in the making. The economics behind the products we use. The misuse of funds or political blunders, presented in a compelling data visualization that leaves little room to argue with it.

I haven’t had the chance to read the book “cover to cover”, though I have skimmed it. I see a lot of quotes like the above, and not much about how to interrogate data or avoid common statistical mistakes. (Although to be fair, there is some discussion.) If we want to set ourselves up as storytellers, if we want to turn data into something meaningful, then we have a responsibility to make sure that what we’re saying is, well, true. Or as close to true as we can get it, with qualifications and caveats as our epilogue.

One last issue which I found myself talking about a lot was, ironically, communication. This came up first in an early session with the creator of Purple Binder, an online directory of Chicago social services. There is apparently a wealth of information being stored in the paper directories of social workers around the country – information that is duplicated and deprecated a little more every day. An online directory seems like an obvious solution, but it’s hard to get social workers to buy in to the process – they’re already stressed to the limit with a heavy workload, and entering data online is more cost (transferring information, adapting to new formats, dealing with bugs) than benefit to early adopters. And there have been efforts like this before, efforts which have failed due to disagreements about how to organize information, and power struggles over who gets to play the gatekeeper.

There were also debates about data standards. If we can agree on taxonomies and formats, we can combine and share data more easily, making it more accessible, more meaningful, more powerful. I’m not going to talk much more about this, because it’s not an area of expertise for me (although one talk at TCamp about RDF organization has me bookmarking pages to learn more.)

Finally, there’s a question of community. TCamp was, as most conferences are, primarily a networking opportunity. I met so many people working on a variety of projects, most of whom I’d never heard about before. I exchanged email addresses and project URLs with people who were doing work very similar to mine, and left the camp each day pleased to have all these new resources. But where are the tools for building a community beyond TCamp, and for those who couldn’t get there? The Sunlight Foundation has a google group, which is well-trafficked but difficult to search through, and an IRC channel, which is pretty quiet. Open Congress has a wiki, but I haven’t seen it promoted much and consequently, it stores only a fraction of the community’s knowledge.

My very last Tcamp session was with a dozen other local open government organizers from around the country. During the session, we did an exercise where we wrote down what we need to be successful. When we compared notes, we realized that we were all starved for communication. We needed to talk to our communities and find out what they wanted from the open government movement. We needed to talk to our local officials and figure out if they were willing to work with us, and how. We needed to talk to experts who could give us legal and technical advice. And most of all, we needed to talk to each other. To share resources and insights, to keep ourselves from needlessly duplicating others’ hard work, and, perhaps most importantly, to build a community. Because organizing can be hard, frustrating work, and it’s good to do it with friends. And you don’t really need a better to reason to do something than that.

1. Participatory Budgeting
Developed in Porto Alegre, Brazil, this process allows community members to decide how to spend a portion of the budget. Since 2009, Chicago Alderman Joe Moore has been doing this with his ward, and as of just a week ago, Vallejo, California will be doing participatory budgeting as well.

2. Opening Up Atlanta
Matthew Cardinale told the story of how he sued the city of Atlanta for violation of the state’s open meeting law. Warned by every lawyer he approached that his activism was futile, he argued as a pro se litigant all the way up to the state supreme court, where he won. In their decision, they made clear that the default state of government in Georgia is openness.

3. Superfastmatch
An API for a new algorithm which finds overlap between large blocks of text, well, super fast. Why is this good for transparency advocates? It can help detect when journalists are regurgitating press releases, for one. It can also uncover when legislators use “model bills” given to them by think tanks, as was the case with some of the recent ultrasound bills and the ALEC-backed Stand Your Ground laws.

4. The Hacker Bus
A project of Transparencia Hacker, a Brazilian open government group, this bus travels around hosting transparency awareness events and arranging meetings between local hackers and government authorities.

5. FollowTheMoney.org
Where you can find some truly disheartening reports, such as: “The top five recipients of $3.7 billion in federal corporate tax breaks paid $0 in 2009 federal taxes and enjoyed a combined profit of $77.16 billion in 2010. This report reveals that these corporations also gave $78.7 million to state political campaigns and $45.3 million to federal campaigns in the last decade.”

6. Lots of funny t-shirts
This one was my favorite:

7. The Plain Writing Act of 2010
A perfect example of the difference between availability and accessibility. If you’re not already convinced we needed this, take a look at the example on the wikipedia page linked above.

8. France’s Literal Underground of Hacker-Artists
I forget what this has to do with transparency, but it’s still pretty cool. From the liked article: “This stealthy undertaking was not an act of robbery or espionage but rather a crucial operation in what would become an association called UX, for “Urban eXperiment.” UX is sort of like an artist’s collective, but far from being avant-garde—confronting audiences by pushing the boundaries of the new—its only audience is itself. More surprising still, its work is often radically conservative, intemperate in its devotion to the old. Through meticulous infiltration, UX members have carried out shocking acts of cultural preservation and repair.”

9. Politwoops
An API of tweets deleted by politicians.

10. Programming Metaphors
Unit-testing and system-testing legislation. That is, documenting the stated purpose of legislation on a modular and holistic level and evaluating whether it “passes”. Related, Gitlaw: using git (or git-like) version control systems to help track/comprehend incremental changes to legislation. (I feel like maybe this would only be helpful to programmers, and just confuse everyone else more – but I’d like to see it done.)

11. Citizen Science
Okay, I already knew about most of the projects mentioned in this session but I was still glad to talk about them! I see a lot of overlap in the open science and open government movement, and not just in the names. And I did learn about a couple new projects: Be a Martian and Leaf Snap.

That’s not nearly all, but I’ll stop there for now.

Over the last week, I’ve come across a couple new ‘science experiments’ – that is, experiments in improving science by encouraging the involvement of the online public. The first one, Petri Dish, is basically a Kickstarter for science projects:

Right now, it’s skewed pretty heavily towards ecology and animal behavior. Which makes me curious about how different scientific fields have embraced online innovation in different ways. For instance, there’s arXiv, which provides open access to articles in a handful of fields, including nearly all articles published in most subfields of mathematics and physics. However many other fields hide the bulk of their research away in closed access journals which require $10-20 per article.

Then there’s crowdsourced science projects like FoldIt and GalaxyZoo, which allow the public to participate in large scale protein synthesis and astronomical classification experiments, respectively. These may just be pioneers at the forefront of a wave of crowdsourced experiments in all fields, but so far it doesn’t look like it. So why these projects, in these fields?

My own field of psychology is in some ways ideal for small scale, independent research. It can be quite cheap to do a study, doesn’t require years of training or elaborate equipment just to get started, and understanding the published work in the field can be fairly intuitive compared to, say, physics. We’ve all got minds of some kind, after all. But I haven’t seen many efforts to involve the public in psychology as scientists. The field has embraced the internet as a source of subjects (see PRO, Project Implicit, the Moral Sense Test and of course Mechanical Turk for just a few examples) but I don’t recall ever seeing an effort to get the public involved in the research.

Until a few days ago. A friend linked me to the Reproducibility Project, an effort by the Open Science Framework, which is in turn the brainchild of University of Virginia psychologist Brian Nosek. The project is an effort to replicate all articles published in three major psychology journals in the year 2008. This seems largely in response to growing concern over the accuracy of published psychology findings (see here, here here and here for again just a few examples), although Nosek is quick to point out that a failure to replicate does not mean that the original article was fraudulent or even incorrect.

Although it appears that the actual replications are being done entirely within academic institutions, the meta-analysis is being conducted openly on the web, with the public able to review the full introduction and methods sections for each replications, as well as the “bare bones” result, participate in discussions of the project as a whole on the group mailing list, help with administrative work such as coding, and possibly much more.

I am planning on getting involved with the project, so I will report back. While the researcher in me would love to be able to run a replication on my own, independently, I understand the ethical problems (though they could be solved by a public IRB! I’m just sayin’) and I appreciate just how open the rest of this project promises to be.

At the same time, though, each of the projects are a far cry from, say, the Public Lab, which as far as I know does not require institutional affiliation for any part of the research process. They work primarily with aerial mapping. Is it possible to achieve that level of independence in other fields? If not, what’s stopping us?

There are so many roles that people can play in the research process, even without being academics. They can be subjects, funders, reviewers, data collectors, analysts – and, I am sure, visionaries. But I’m not sure how we get there.

I turned to the other people in the room and said, “Look it’s a brain! A braaaaaaain!”

I was thinking specifically of the cerebellum which, when cut sagittally, looks like this:

Of course, what both a cauliflower and a cerebellum look like, when cut right, is this:

The term for this type of structure, dendritic, comes from the greek word for tree, dendron. Anyone passingly familiar with neuroscience will recognize the term dendrite:

But the most gorgeous natural dendritic structure I’ve seen recently are these rivers in California:

Any dendrites I’ve missed?

Gefilte fish cupcakes with horseradish whipped cream. Gefilte fish cupcakes.

That said, this site is actually pretty cool, and I do want to try out a bunch of her recipes (once passover is over.) On the list are:

Tomato Cupcakes with Basil Filling
Margarita Cupcakes
Parmesan Sour Cream Cupcakes with Whipped Raspberry Frosting
Olive Oil Cupcakes with Lemon & Thyme Frosting
Wine and Cheese Cupcakes

If you want to make them with me (or just taste test the results!) let me know.

We decorated beer bottles:

And made silly hats:

Mako requested a puzzle egg hunt, so I obliged.

Success!

Turns out sea slugs (and related sea creatures) are frequently quite gorgeous, and a few of them have some pretty neat skills. For instance, there’s a couple species of slug that can eat sunlight. And there’s a sea slug which feeds on Portuguese man-o-wars. Not only is it immune to their nematocysts (stingers), it collects them in sacs, concentrating the poison until it’s even more dangerous to the touch than its prey.

My favorite creature I’ve discovered in the course of making this cake is not actually a sea slug – it’s a “sea squirt”, a fairly simple organism that starts off life with a rudimentary brain (really a ganglion), uses it to find a home, and then says, “Well, I guess I don’t need this any more” and digests it.

Victoria once said she was “a child without a childhood”. Born Victoria California Claflin, the seventh of ten children, she had only a few years in school before dropping out to support the family by begging and helping to raise her younger sisters. Her father, an abuser and a drunk, bought an old mill and burned it down for the insurance money, an act which got the family kicked out of town. In their new home, Victoria and her beloved younger sister Tennessee (Tennie) began working as clairvoyants. When she fell ill, 28 year old doctor Canning Woodhull treated her. Even sick, she must have been charming – he fell in love, and within a year they were married. Victoria was 15.

Married life was not easy for Victoria. Her first child, named Byron after the poet, was born developmentally disabled. Her husband, unsuccessful as a doctor, spent much of what he earned on alcohol, so Victoria began to take odd jobs to support the family, working first as a dressmaker, and then as an actress. When her husband delivered their second child, left both mother and newborn bleeding, and didn’t return for three days, Victoria decided she was done with him. She spent the next several years traveling with her birth family. She posed alongside Tennie first as mediums, then as healers who could cure cancer. At one point the family was charged with nine counts of fraud, but they left town before they could be apprehended.

Eventually Victoria and her sister Tennie decided they wanted a fresh start. They moved, along with Victoria’s children and new husband Colonel James Blood, to New York City, where they billed themselves as simple clairvoyants. One of their clients was multimillionaire Cornelius Vanderbilt, who fell in love with Tennie and who encouraged the sisters to go into business together. Their business received a boost on Black Friday. As others frantically sold socks, Victoria stood on the steps of the stock exchange – women weren’t allowed in – and sent couriers inside to buy. The risk paid off, and Victoria earned a tidy sum, which she and Tennie used to open up a brokerage business. They were the first American women to ever do so, and the novelty of it catapulted them to fame. The experienced Vanderbilt helped manage the firm, and Colonel Blood was his wife’s personal secretary. She and Tennie used profits from the brokerage to start up a weekly newspaper.

Victoria’s life up until this point had certainly been unconventonial, however the archives of the newspaper make clear how radical she truly was. The paper exposed financial fraud, published the first English-language version of the Communist Manifesto, and made impassioned arguments on social issues which challenged the morals of the period.

After the death of a young woman from an abortion gone wrong, which caused most people to call for stricter penalties against abortion, Woodull published The Social Volcano (Sept 16th, 1871 edition of the paper), writing:

You have erected a social system which you teach from childhood to manhood, which you inculcate at home, at school, at church, and in all your social relations. Yet the victims are made daily in spite of the inhuman punishment you inflict for a non-compliance with your standard. Now in dealing with your system, we say here are a hundred victims; if they have no means of escape, at least ten will commit suicide, at least fifty will be driven from home into the streets, and the remainder will have the finger of scorn pointed at them so long as they live, while ninety children will be brought into the world under a ban of shame, which the better men they become, the harder will that ban be to bear, the deeper will it sink into their heart, although they had no control over the circumstances that make them degraded for life.

The Rosenzweigs step in and say, practically, “I will spare you nine deaths out of the en (for only one in a hundred dies under my hands). I will send fifty more out of the hundred victims home, still virtuous according to your rules, because no one knows of the so-called transgression, and the remainder shall have their error known only to their parents or relatives, and the outside world and future husbands shall remain in happy and blissful ignorance of the truth. All this I will do for one hundred dollars each.” You find the money, you have the private interview with Rosenzweig (not that you are interested; oh, no, it is all out of pure friendship; you didn’t do it; it was some other man.) You cheat society’s rules of its victims, until another victim dies. Then you howl at their depravity and Rosenzweig’s villainy.

They published a tract about free love on August 26th, 1871:

In this country; and in this age, we have, in one sphere of social affairs, a successful and triumphant practical illustration of the theory that the recognition of the rights of the individual is the talisman of order and harmony in society. . . . Not only is he permitted “to worship God according to the dictates of his own conscience,” but, equally, to neglect or refuse to worship Him altogether; and the result is peace and fraternity; in the place of the inquisition, the burning fagot and war.

For one, I reject and repudiate the interference of the State in my morals, precisely as I do the interference of the church to prescribe my religious deportment or believe. The outrage on human rights is in my view no less in kind to assume to determine whom men and women may love, and what manifestation they may make of that sentiment, than it is to burn them at Geneva or Smithfield for heretical practice or faith.

(Woodhull herself was an ardent proponent of free love, saying in a speech a few months later, “Yes, I am a Free Lover. I have an inalienable, constitutional and natural right to love whom I may, to love as long or as short a period as I can; to change that love every day if I please, and with that right neither you nor any law you can frame have any right to interfere. And I have the further right to demand a free and unrestricted exercise of that right, and it is your duty not only to accord it, but, as a community, to see that I am protected in it. I trust that I am fully understood, for I mean just that, and nothing less!”)

Then there’s this gem on the failure of the credit system after the gold speculation crisis which caused the 1869 crash, which sounds sadly familiar to us now in 2012:

The sufferers by the credit system are generally those who leas suspect themselves to be victims of the robbery. They know that many people fail in business, and that their creditors are supposed to lose large sums of money; but, as long as none of these debts are due to them, they do not feel concerned. They know, also, that they are obliged to pay unreasonably high prices for many things; but they do not dream that there is any connection between these prices and the bad debts and bankruptcy resulting from this abuse of credits. But there is such connection, and the mass of consumers have to pay roundly for the folly and fraud of men who undertake to do an immense business with little or no capital, and who for a while pride themselves on the excellence of their credit…..

I also greatly enjoy the “Women Items” that they ran occasionally:

Marion V. Churchill very sensibly says: “I would rather see a policewoman, club in hand, marching an offender to punishment than to see that same woman marched to punishment, by a policeman for lack of money which would have been her honorable salary in the police service. There are women in the world better fitted for police than for parlor duty. Let them do it.” (February 3rd, 1872)

Horace Greeley’s advice to women to learn how to cook a steak, and not to mind suffrage, is good under circumstances. But without the steak the knowledge to cook it is valueless. This is where the suffrage will come in. (September 9, 1871)

The paper published a great deal more, although sadly only a small portion of the paper is online. (It’s all on microform, though! If Victoria or Tennie were here, they’d prophecy a trip to the library in my future.)

Woodhull was very much involved in the women’s suffrage movement. She was the second woman to testify before congress, and like the first, Elizabeth Cady Stanton, she petitioned for the vote:

“Women constitute a majority of the people of this country – they hold vast portions of the nation’s wealth and pay a proportionate share of the taxes. They are intrusted with the most holy duties and the most vital responsibilities of society; they bear, rear, and educate men; they train and mould their characters… they are the secret counsellors, the best advisers, the most devoted aids in the most trying period’s of men’s lives, and yet men shrink from trusting them in the common questions of ordinary politics. Men trust women in the market, in the shop, on the highway and the railroad, and in all other public places and assemblies, but when they propose to carry a slip of paper with a name upon it to the polls, they fear them.”

In 1872, Woodhull ran for President of the United States. (She asked Frederick Douglas to be her running mate, in what appears to be an attempt to mend fences after the fight over the 15th amendment, which granted suffrage to all races, but not to women. Douglas declined the offer.) Victoria’s name was kept off the ballot in many places, and there’s evidence that votes for her were discarded. Then, two days before the election, Victoria, her husband and Tennie were all arrested for “publishing an obscene newspaper.” The man responsible for the arrest, moralist Anthony Comstock, was a pretty vile man: he destroyed thousands of books and millions of pictures he deemed inappropriate, clashed with activists Emma Goldman and Margaret Sanger, and allegedly boasted that he’d driven fifteen people to suicide. Victoria, Tennie, and Blood spent six months in prison before being released on a technicality. Victoria never got a chance to try and vote for herself.

The last forty years of Victoria’s life were not as sensational as her first forty, but she continued to advocate for women’s suffrage and to publish radical writings before retiring at the turn of the century. She and her sister moved to England, where she passed away in June of 1927, seven years after the United States finally granted women the right to vote.

Many of her contemporaries called Victoria insane, a prostitute, a devil. The famous cartoonist Thomas Nast called her Mrs. Satan. Even other suffragists at times found her too radical. But with a hundred years’ hindsight, it’s clear that Victoria Woodhull was a brave, resourceful, charismatic, brilliant woman overflowing with the capacity to love and possessed of a sharp sense of justice. I’d vote for that.

Sources:
Victoria In Her Own Words (source for speeches)
WC Weekly archives
Victoria Woodhull: Fearless Feminist by Kate Havelin
Legal Contender: Victoria C. Woodhull, first woman to run for US President – Feminist Geek
and, as always, Wikipedia

Black Hole Life Preserver:

Is there an invention a prospective [black hole] diver can use to resist spaghettification? Gott and his colleague Deborah Freedman Woods calculated that a giant ring might do. If this “life preserver” encircles your waist as you fall, its gravity counteracts the black hole’s, pulling your sides apart while pulling your head and feet together…

[S]uch a buoy would have a mass of more than 12,800 trillion tonnes, about two-millionths the mass of Earth, roughly equal to an asteroid 100 miles wide. “That’s somewhat beyond the current NASA budget,” Gott says.

…“Hero of Alexandria invented a steam engine before 100 AD, and people back then looked at this and said, ‘Good job, that’s fun, isn’t that nice,’ and nobody looked at it and said ‘Wait, this can change the world.’ We waited about 1,700 years for the Industrial Revolution,” Gott notes. “You can never tell — concepts for inventions that might seem like toys now might have unrealized promise.”

The sense of meaning in dreams:

By investigating why dreams feel profound, one might learn how events get imbued with this sense of meaning — perhaps the same one felt during revelations. Stickgold notes that during REM sleep, when dreaming typically occurs, the release of the neurotransmitter serotonin is shut off in the brain. The only other time that happens is because of LSD, “when people seem to have these totally uninteresting experiences they describe as profoundly meaningful called ‘acid insights.’”

… The solution? Experiments with drugs that suppress or boost serotonin levels could explore any connections between the neurotransmitter and the feeling of meaning. “You could give people such compounds or a placebo and get them to rate how deep or meaningful specific movie clips seems to them,” Stickgold suggests.

(Also: Raising animals to human intelligence and Printing people and Testing if 10,000 hours makes you an expert.)

Of course, reading this series made me think about what experiments I would run if I had no practical or ethical limits. I came up with this:

For over a decade now I’ve been obsessed with genocide. What fascinates me more than anything else is the moral complexity of the phenomenon, the number of people capable of goodness and kindness who somehow turn instead to fear, anger, and violence. A serial killer or a mass murder is just as “evil” as a person taking part in a genocide, likely more so. But I guess I’m not interested in evil so much as our collective failure to be good. It’s not the psychopaths that disturb me but the bystanders, the folks who say it isn’t that bad as the community slides into hell and never again! as it struggles to find its way out. You know, the people who could very well be me.

So my questions are, What makes a society attempt genocide? and What environmental and personality variables distinguish people who foment genocide, people who go along for the ride, and people who try to stop it?

Probably the “ideal” way to answer these questions would involve a really convincing virtual reality where you could simulate a society turning genocidal around an individual subject, tweaking aspects of the their experience and comparing the actions of different subjects who experience the same thing. Even better would be if we developed the technology to suppress people’s memories, so that you could run different simulations on the same individual. With such resources, it would be trivial to measure/fiddle with physiological reactivity to experiences as they happened, to see how a surge of adrenaline or testosterone, an anxiety attack, or a mild sedative influences moral decision-making.

Of course, the technological limitations here are secondary to the moral ones. Even if we could do all of this, we shouldn’t. We’d be no better than that which we seek to stop – which, when you’re trying to stop genocide, really says something.

(Although if we did have memory suppressants in our virtual world, I would volunteer to be a subject.)

Anyway, that’s the dream that can never be. What impossible experiment would you do, if you could?

It deals with what the authors call “researcher degrees of freedom” – a series of somewhat arbitrary decisions about experimental design and data analysis that researchers have to make over the course of their study. Because researchers are a) out of a job if they don’t find enough significant results and b) human, this ambiguity can frequently, unconsciously, perniciously be used to hone in on the most significant results, causing an abundance of false positives in the field as a whole.

The coolest part of the article is a set of simulations they ran where they looked at the effect of four common decisions, or degrees of freedom, on results. The decisions were:

A) If you have two dependent variables, which one should you conduct your analysis on – or should you use an average of the two?
B) If you’ve tested a moderate number of subjects and found no results, should you try to improve your power by adding more subjects?
C) How do you deal with covariants? For example, should you see if there is a main effect of gender on your study, or an interaction between gender and condition?
D) If you have multiple conditions, which should be compared and which should be combined?

The researchers simulated results 15,000 times by drawing randomly from a normal distribution. By definition, this means that they should get “significant” (p < .05) results 5% of the time, "highly significant" (p < .01) results 1% of the time, and "marginal" (p < .1) results 10% of the time. However, when they simulated the above decisions - for example, with B, by testing after 20 observations and only adding 10 more subjects if they failed to reach significance - they had many more false positives. When you combine all four degrees of freedom, the simulation was more likely than not to find a “significant” result!

Also cute: the researchers did a study on whether listening to the Beatles makes you age backwards. This paragraph below shows the difference between how they could, according to current reporting standards, describe the study (in bold) and how they recommend all studies be reported:

The authors use six reporting guidelines (and four corresponding reviewer recommendations) to tackle this problem. Although it’s not my preferred method I’m just grateful to see people with standing in the field continue to talk about these issues. I agree with pretty much everything in this paper, especially their conclusion, which I think packs a nice punch.

Our goal as scientists is not to publish as many articles as
we can, but to discover and disseminate truth. Many of us—
and this includes the three authors of this article—often lose
sight of this goal, yielding to the pressure to do whatever is
justifiable to compile a set of studies that we can publish.
This is not driven by a willingness to deceive but by the
self-serving interpretation of ambiguity, which enables us to
convince ourselves that whichever decisions produced the
most publishable outcome must have also been the most
appropriate. This article advocates a set of disclosure require-
ments that imposes minimal costs on authors, readers, and
reviewers. These solutions will not rid researchers of publica-
tion pressures, but they will limit what authors are able to jus-
tify as acceptable to others and to themselves. We should
embrace these disclosure requirements as if the credibility of
our profession depended on them. Because it does.