Comments on: We need a more negative culture http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/ Daniel Lemire's blog is about life in academia, research in Computer Science, wondering how we can reconcile fast databases and algorithms with the informal and asemantic nature of the world around us. It is broadcasted from Montreal (Canada). Tue, 02 Dec 2008 19:37:01 +0000 http://wordpress.org/?v=2.6.5 By: Heather http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50087 Heather Tue, 12 Aug 2008 18:24:06 +0000 http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50087 Nice cartoon on <a href="http://www.vadlo.com/Daily_Research_Cartoon.html" rel="nofollow">Negative results</a> at Vadlo website. Nice cartoon on Negative results at Vadlo website.

]]> By: Mike Stiber http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50082 Mike Stiber Tue, 05 Aug 2008 18:17:49 +0000 http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50082 There is a once well-known paper by Drew McDermott from the ACM SIGART Bulletin (Issue 57 (April 1976), Pages: 4 - 9) entitled "Artificial Intelligence Meets Natural Stupidity" that concludes with: "...AI as a field is starving for a few carefully documented failures. Anyone can think of several theses that could be improved stylistically and substantively by being rephrased as reports on failures. I can learn more by just being told why a technique won't work than by being made to read between the lines." There is a once well-known paper by Drew McDermott from the ACM SIGART Bulletin (Issue 57 (April 1976), Pages: 4 - 9) entitled “Artificial Intelligence Meets Natural Stupidity” that concludes with:

“…AI as a field is starving for a few carefully documented failures. Anyone can think of several theses that could be improved stylistically and substantively by being rephrased as reports on failures. I can learn more by just being told why a technique won’t work than by being made to read between the lines.”

]]> By: Sylvie http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50031 Sylvie Tue, 22 Jul 2008 11:38:14 +0000 http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50031 This is also a problem in psychology, where the same mindset prevails: once an experiment has "proven" something, it is rarely redone (although it does happen). In biomedicine, they have a Journal of Negative Results (http://www.jnrbm.com/). We need more of those types of journals. This is also a problem in psychology, where the same mindset prevails: once an experiment has “proven” something, it is rarely redone (although it does happen).

In biomedicine, they have a Journal of Negative Results (http://www.jnrbm.com/). We need more of those types of journals.

]]> By: Anonymous http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50030 Anonymous Tue, 22 Jul 2008 09:48:51 +0000 http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50030 In complete agreement but would go further to argue that finding out what doesn't work and why is a lot more informative than the majority of positive results. The work of John Ioannidis in other fields such as medince and epidemiology http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0020124 must have implications in computing science. In complete agreement but would go further to argue that finding out what doesn’t work and why is a lot more informative than the majority of positive results.

The work of John Ioannidis in other fields such as medince and epidemiology http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0020124
must have implications in computing science.

]]> By: Mark Reid http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50029 Mark Reid Mon, 21 Jul 2008 22:55:55 +0000 http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50029 This topic came up at an ICML workshop on <a href="http://www.site.uottawa.ca/ICML08WS/" rel="nofollow">evaluation methods in machine learning</a> I recently attended. I've written up <a href="http://conflate.net/inductio/2008/07/evaluation-methods-for-machine-learning/" rel="nofollow">a summary</a> of what I though were the highlights. Janez Demsar made the argument that there is too much emphasis on positive results and, consequently, as a reviewer it is difficult to prefer a paper that shows big improvements over existing techniques over one that does but puts forward an otherwise interesting idea. This topic came up at an ICML workshop on evaluation methods in machine learning I recently attended.

I’ve written up a summary of what I though were the highlights. Janez Demsar made the argument that there is too much emphasis on positive results and, consequently, as a reviewer it is difficult to prefer a paper that shows big improvements over existing techniques over one that does but puts forward an otherwise interesting idea.

]]> By: Sérgio Nunes http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50027 Sérgio Nunes Mon, 21 Jul 2008 18:31:10 +0000 http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50027 Publishing bad results is also good as an alert sign for future researchers. If bad results are unpublishable then we are faced with a situation where future generations are condemn to repeat intuitions that have been proved wrong previously. Refuting a thesis should be as valid as proving it. And it is. Publishing bad results is also good as an alert sign for future researchers. If bad results are unpublishable then we are faced with a situation where future generations are condemn to repeat intuitions that have been proved wrong previously.

Refuting a thesis should be as valid as proving it. And it is.

]]> By: Peter Turney http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50026 Peter Turney Mon, 21 Jul 2008 13:50:15 +0000 http://www.daniel-lemire.com/blog/archives/2008/07/21/we-need-a-more-negative-culture/#comment-50026 Related: Journal of Interesting Negative Results http://jinr.site.uottawa.ca/ Journal of Negative Results in Biomedicine http://www.jnrbm.com/ Journal of Negative Results: Ecology and Evolutionary Biology http://www.jnr-eeb.org/ How to Maximize Citations (7. Positivity) http://tinyurl.com/5laylg Related:

Journal of Interesting Negative Results
http://jinr.site.uottawa.ca/

Journal of Negative Results in Biomedicine
http://www.jnrbm.com/

Journal of Negative Results: Ecology and Evolutionary Biology
http://www.jnr-eeb.org/

How to Maximize Citations
(7. Positivity)
http://tinyurl.com/5laylg

]]>