Algorithm-generated nonsense reviews fool actual human peer reviewers one in four times

[vellavu]

Algorithm-generated nonsense reviews fool actual human peer reviewers one in four times | ExtremeTech

Every so often, scientists need a little picking on, too. Especially when the well-respected peer reviewing process goes horribly wrong.

About a decade ago, some grad students from MIT came up with a project that started as a shot over the bow at computer scientists and their ever-increasing use of buzzwords and jargon. They made a program called SCIgen that uses a Markov chain to stochastically (randomly) sample out clauses from a body of text called a “corpus.” It then arranges the words in a syntactically correct way, which nevertheless reads like a name-dropping version of The Jabberwocky, crossed with Mad Libs. “One useful purpose for such a program,” they suggest, “is to auto-generate submissions to conferences that you suspect might have very low submission standards.”

I think it can fool them four in four times if it focuses on postmodernism social science research.

 

[Ethan Reigns]

There was an article three decades ago in Byte magazine about what they called a "Travesty Generator" that did the same thing. A colleague at one place where I worked had gone through a university with a strong Trotskyite communist organization, so he and his friends got together and generated a handout that they distributed that used similar travesty-generated text. The real communists were not amused.

 

[Blaze]

The researchers made randomly generated reviews of actual published articles using their iteration of a Markov chain, and then gave them to reviewers, slipped into the stack like ringers between actual reviews.

So they were actually making reviews? If a computer program can do this well enough, I can see how that would get through. It wouldn't be so easy to do that with new findings. (Although falsified results is another big issue when it comes to original articles). But as far as reviews, every scientist knows to read those as a general overview of a field, but you always double check the references if you actually want to use any of the info. Still, this made me giggle, oh dear these people must have had a blast fooling those reviewers.

 

[ILLYRIAN]

vellavu, how could you post that? How could you post something that implies scientists need to be picked on? How utterly disgraceful of you and I think it only right that you apologise profusely to Black Thorn's resident scientist, Blaze.

 

[November]

Hahaha. Yet another thing to prove that peer reviewed does not mean correct.

I remember reading something last year - about a peer reviewed study confirming the existence of god.


There was another that blamed language problems when a whole bunch got through that suggested God was the reason hands work.

 

[vellavu]

vellavu, how could you post that? How could you post something that implies scientists need to be picked on? How utterly disgraceful of you and I think it only right that you apologise profusely to Black Thorn's resident scientist, Blaze.

Our proud Black Thorn scientist Blaze needs no mortal peer approval. Thank you.

 

[thrasherpix]

I once knew a professional editor (at least that had been one of the many things she dabbled in, and she'd gotten credit for it in a few published books) but my memory is hazy on the details as she explained this to me back in 2000 about how she had once been an editor of journals while she was getting her masters (I'm not sure if this was more academic for the college circuit or actual professional journals), probably in the early 1970s, and there was a high turnover rate for editors.

She got frustrated because the articles submitted to her for editing weren't making sense to her (which made it hard for her to edit), but she assumed the problem was with her rather than the writing. So she took to diagramming the sentences, but it still didn't make sense so she then got the other editors to help her, and they all finally agreed it was a bunch of gobbledygook. As a result they gave up in trying to clarify ideas that were intentionally made murky but it was no problem as all the scientists (or science students) and others were lavishing high praise on each other, pretending the published articles were something other than nonsense. 😀

 

[Blaze]

I once knew a professional editor (at least that had been one of the many things she dabbled in, and she'd gotten credit for it in a few published books) but my memory is hazy on the details as she explained this to me back in 2000 about how she had once been an editor of journals while she was getting her masters (I'm not sure if this was more academic for the college circuit or actual professional journals), probably in the early 1970s, and there was a high turnover rate for editors.

She got frustrated because the articles submitted to her for editing weren't making sense to her (which made it hard for her to edit), but she assumed the problem was with her rather than the writing. So she took to diagramming the sentences, but it still didn't make sense so she then got the other editors to help her, and they all finally agreed it was a bunch of gobbledygook. As a result they gave up in trying to clarify ideas that were intentionally made murky but it was no problem as all the scientists (or science students) and others were lavishing high praise on each other, pretending the published articles were something other than nonsense. 😀

Well to be fair if you're not a scientist, most paper article will seem like nonsense. Heck, even when you are a scientist, they may seem as though they are nonsense if you're not in that field, but the principles and findings might, and often are, accurate. I've had family members and friends read my research and they couldn't make any sense out of it, but it doesn't mean that the research isn't sound. A lot of the stuff I do will only be understood by people who've studied the field for a few years. And even having done research for a few years now, if I look at a different field, chances are I won't understand.

Hahaha. Yet another thing to prove that peer reviewed does not mean correct.

I remember reading something last year - about a peer reviewed study confirming the existence of god.


There was another that blamed language problems when a whole bunch got through that suggested God was the reason hands work.

One of the major thing is there's a big difference between reviews, which is what they submitted in this research, vs. original findings. So this study doesn't disprove the entire process of peer review, just one aspect of it.

I'd be super curious to read that study confirming the existence of god though 😀 that sounds like it'd be a blast to check. Funny enough, I know a professor who published an article against the idea of god. Or to be more accurate, proving that otherworldy experiences (often associated with god, or the afterlife) are created by the brain.

 

[ILLYRIAN]

I never guessed that my post would have caused such a response, and that it delivered a wonderful response from vellavu that was totally effective.

OccaQuokka, I was a bit confused by what you wrote, something about confirming the existence of God but the blaming different languages and when a whole (hole?) bunch got through and proved that hands work because of God. Are you referring to a God who loves Doughnuts, and has God made the hand work when the pastry is above the hand that waves to the atheist?

The last paragraph Blaze was it a statement or a question, if it was a statement or even if it was a question does it confirm the existence of God?

 

[November]

Hands are the "proper design by the Creator," PLOS ONE paper suggests - Retraction Watch

Hope that link works! Haha. I love reading through some of the things caught out.

I have a lot of time on my hands... I read so much that makes me doubt the whole lot!

I think its gods plan, bring it all down ya know 😉 Dollhouse Style.

(Im joking dont go mental on me.)