- Current conventions in computer science mandate that each time I write a paper, I should state my result four times: once in the abstract, once in the introduction, once in the body, and once in the conclusion.
- The introduction section is, to a large extent, a waste of everybody's time. Nearly every paper about a given topic has the same introduction: "Topic X is important because A, B, C."
- Here's a simple formula that seems to work for (and is overwhelmingly used in) computer science papers:
Abstract: We solve problem Y.
Introduction: Problem X is so important, that if we solve it, the world will be a much better place. Y is an approximation (or subproblem) to problem X. We solve Y.
Body: Here's a convoluted explanation of how to solve Y.
Conclusion: We did Y. Doing X would be awesome. - Once a paper is "published," it is set on stone and cannot really be changed, even if you find a much better way to convey the results or if you find that the data is better explained by a different hypothesis. The reason for this restriction is that, 30 years ago, papers were published using physical paper. Such a restriction makes no sense today.
- As an academic community, it sometimes feels that the final goal of doing research is publishing papers. The goal of doing research should be, well, doing research. I understand that communicating the results of our work is important, but surely there is a better method than one that was invented before computers were around.
- Given the number of people working in computer science and the fact that publishing papers is considered the goal of our work, there is an insane number of papers written every year, the vast majority of which contribute very little (or not at all) to our collective knowledge. This is basically spam. In fact, for many papers (including some of my own), the actual idea of the paper could be stated in one paragraph, but somehow people manage to write 10 pages of it.
Can a combination of a wiki, karma, and a voting method like reddit or digg substitute the current system of academic publication?
I think there's a vast amount of potential being wasted holding on to the paper medium of the past.
ReplyDeleteSimple systems like GPeerReview could take care of providing review and allow for tenure credit (e.g. a signed hash value of a good review of your paper on your CV by someone that's respected). I think the peer review would still be helpful to prevent a "pop culture" mentality that digg might cause.
I wouldn't stop at writing, I would love it if people with new ideas would learn Presentation Zen skills and convey their results in a YouTube like presentation. Pecha Kucha style would be preferred.
Perhaps a TED for Computer Science?
If you go with the traditional writing route, how about some school have a academic golf challenge of who can get the best idea down in the shortest, but easy to read, fashion? Feel free to use color and design skills.
What's stopping you from leading this?
Fore!
Voting method like Digg sounds awesome. Seems like the best way to get your research reviewed without having to go through this massive paper waste/nonsense.
ReplyDeleteFor many people research is publishing papers only. The world start and ends with it. Something should be done in this regard, but what are the alternatives????
ReplyDelete"Research = Papers" is mostly a cultural problem. One thing that is important to understand (especially for PhD students) is that the number of published papers is not a measure that is widely used when evaluating you. When hiring an assistant professor, for example, I'd say the impact of your top three projects is looked at a lot more than your total number of papers. Widespread understanding of this may help people stop thinking that "research = papers."
ReplyDeleteJust curious: how do you assess impact?
ReplyDeleteThere is no perfect way to evaluate impact, but some questions that are typically asked are: Have we heard of the work? Has it gotten awards? Does it affect a lot of people? Has it gotten press? How many other papers cite it? How impactful are the other papers that cite it (think PageRank)?
ReplyDeleteThat sounds like a reasonable heuristic. It seems that "all" that is needed to make your 2.0 system work is the right core group agreeing to use it. It'll bootstrap your PageRank philosophy. If the new approach already gets the first half of what you mentioned, it shouldn't be impossible.
ReplyDeleteFor me, additional paths include hearing about something in Communications of the ACM or a Google Tech Talk is important; this is how I learned about your work.
Don't settle for 1.0. Keep pushing for this; CS needs this.
Thanks for the response!
Bblogged at AcademicProductivity.com.
ReplyDeleteCool!
ReplyDeleteI think ideally you'd want to have two streams of review:
ReplyDelete- reviewing of content (theories, models and research), such as "professor Y's theory of Z",
- reviewing/rating/etc. of a given presentation of that model.
The linking of content and presentation is just a structural artifact of the paper system, and should be severed. I think it's also the only way to incorporate something like Moser's points on multimedia presentations, while avoiding the vapid popularity issues of rating systems.
thanks for this post! some 'researchers' in computer science have more than 15-20 papers a year, i.e. two weeks for research, one week of writing, next paper,... i am really tired about this kind of doing research and i am therefore not sure whether i should not leave to industry on the long term. and finally you need a given amount of papers for your tenure track....
ReplyDeletein fact i have another point to make: you write 'How many other papers cite it?' as a measure of impact. that is another difficult problem. the more papers we all write, the less one of these will be cited very often. moreover this is a measure for senior researchers not for phd students and post-docs. moreover such measure depends very much on your community within computer science. there are some areas with only few people working on the topic (but the topic can still be relevant). moreover we already see citing-networks of people citing each other work again and again to improve their h-index (and google scholar (i hate it) only lists like 50% of them and updates 2-3 times a year... not very helpful on post-doc level again. anyway thanks for your post!
I too can't understand, why the research community is behind in switching to the new medium.
ReplyDelete[:O]
We guys are supposed to be changing the the society with our new and cool ideas, not getting stuck with ancient techniques.
Hmmm. Luis, I think your statement about looking at impact is not true. If it were, I think we would see more professors without PhDs, or professors who came back to teaching after being in industry for a while (the only prof i know who did this is Hank Levy). Several people have made humongous impacts in industry.
ReplyDeleteGranted, there are many caveats here. First, universities look mostly for "academic" impact. I think this is not necessarily good, but that's the world we live in.
ReplyDeleteMy statement was meant to more to say that number of papers is less important than the quality of the papers -- which is *certainly* true when we hire in CS here at CMU.
As you point out, top schools and research labs already consider quality / impact over quantity. Academic impact may be a bit incestuous, but I don't see that centuries' (millenia?) old tradition changing any time soon.
ReplyDeleteBesides, I thought grants counted at least as much as publications.
Although research !- papers, the act of focusing on a paper often helps focus on a specific problem. This is particularly useful for new students I think, who have a hard time focusing on something and spending enough time on it.
ReplyDeleteWhat I tell students is that a paper is a tangible marker, a goal post if you wish, that helps them focus on making progress with research. Often, the marker will move as they conduct further research, but that's fine: the important point is the forward gradient.
A wiki edit can also be a tangible marker :) I still dislike papers and the fascination academics have with them.
ReplyDeleteWe are not shifting away from traditional publication models because the existing alternative publication models are not providing significantly greater benefits.
ReplyDeleteFor example, the existing conference and journal systems provide the benefits of recognition and impact. Any alternative model will have to match the strengths of the existing models in addition to addressing their weaknesses in order to be adopted by the academic community.
With regards to recognition and impact, tools that measure the impact of an online publication could be one way to encourage the use of an alternative. I'd be delighted to see an environment that has the benefits of the existing models and provides additional unique strengths, but it could take some effort.
I think we're not shifting cuz we're stuck on our ways.
ReplyDeleteI think we have to separate out a number of different factors.
ReplyDeleteI don't think papers by themselves are a bad thing. What you seem to object to are (1) the need to write 10 pages when 4 would suffice; and (2) the proliferation of "bad" papers.
(1) is easy to fix: when reviewing papers, don't penalize them for being short! (Though I find I more often wish a paper were longer than wish it were shorter; maybe papers are just, in general, badly written. This should also be fixed.)
As for (2), I would be all in favor of cutting down on the number of conferences. That would help, but still wouldn't solve the problem. As I see it, there are two contributing factors to this problem: (2a) the need to publish as much as everyone else (for tenure, especially), and (2b) the need to publish for each grant you are involved in. (2a) could be fixed if people generally published less; reducing the number of conferences would force this to some extent, and as a community we could decide to move in this direction. As for (2b), I don't see any way to fix this in the system we currently have (where professors need to hold several grants in order to get a full-year salary and support graduate students). Perhaps, in the long term, this system can change.
Hi Jonathan :)
ReplyDeleteI would also add to my list of objections: (3) The static nature of papers and (4) the fact that papers seem to have become our main goal in life.
(3) Is quite easy to fix by simply making the "authoritative" version of each paper the one that's posted on your site, or making papers be wikis that only the author can edit.
(4) is probably my biggest complaint of academia and fixing it would require a major culture shift. For some areas such as theory/math, I think it's less of a problem since the end results are mostly theorems and theorems are approximately equivalent to papers (although not identical, so there is some room to improve things). But for more applied areas, I think papers should really be an afterthought. Currently papers are the end goal of our research so a good strategy for academics is to do enough work just to produce a paper (i.e. test your system with 10 users, build only a mock-up, solve a problem that is of no consequence but enough for a paper, etc.)
I find it hard to motivate my students to work on a project past what's necessary for a paper, and I think that's a big problem.
I believe that a paper is never finished, and there are always things to improve it. By this reason, I consider it good that one has no chance to modify it, and can move on to other things.
ReplyDeleteWe could also apply the same idea to films: the director would keep changing slightly the movie.
I'm all in favor of making papers "evolve" in time. I've seen several papers about almost the same subject being published with just a few add-ons, that showed the research evolution.
ReplyDeleteThen you have the position paper, the mid-research paper and then the final results paper. You have these several, typical, milestones (which, I recognize, are good to plan your work) that you go through, but that are hard to follow for someone trying to get into the state-of-the-art of your research domain.
You think: "Ok, these guys are working on this..but...what paper do I cite? Only the last one?"
That is why the paradigm should shift, at least, to a more flexible, evolving media. The focus would be to define a "place" or "holder" of the continuous work of a certain topic or problem domain.
We are doing "Deadline-Driven Development", and, although it might have benefits, it just seems old-fashioned and lacking purpose...
Nice article highlighting some truths many people avoid to admit.
ReplyDeleteHere are some useful related links:
Using PageRank to assess scientific importance:
http://arstechnica.com/science/news/2009/01/using-pagerank-to-assess-scientific-importance.ars
ranking and mapping scientific journals:
http://www.eigenfactor.org/methods.htm
For even more results:
http://friendfeed.com/search?q=pagerank+scientific
Responding to Luis's point 4) - To play devil's advocate, I could argue that it isn't the part of the academic to develop an idea past the proof of concept stage. Because of the serious amounts of time, teamwork, and other capital required to develop an idea beyond the "5 research subject" stage, this development is best left to start-ups and industry. The proper role of an academic, in this conception, is to perform a breadth-first search over the space of ideas to find something that merits a deeper investment, by taking on "blue sky" directions that can't justify industrial or start-up investment.
ReplyDeleteI don't actually believe this, but I do hear it from time to time.
People who say this should go away.
ReplyDeleteOf course, nobody will argue that pushing papers just to push papers is acceptable, but writing a lot can help you think better. See this blog post of mine:
ReplyDeletehttp://www.daniel-lemire.com/blog/archives/2008/07/11/do-you-think-because-you-write-or-write-because-you-think/
I disagree that, in the past, results were set on stone because they were published on paper. Authors published several editions of their books, thoroughly revised most of the time. And, for articles, they revisited, for the best of them, the content several years after the initial publication, by making additions and corrections. Anyway publications are just a photographic picture of one's thought at a point in time.
ReplyDeleteI enjoyed this post (the incipit is fantastic), and the discussion in the comments. I believe that:(i) we (the research community) *must* be aware that the Web, Web 2.0, etc. do provide new tools, and (ii) these tools might change the way we disseminate science (or maybe even the way we do science), and we are the right persons to understand if, when, and how.
ReplyDeleteI also add a link to:
http://etuttounmagnamagna.blogspot.com/2009/03/whats-wrong-with-scholarly-publication.html
S.
Great Post! I just found this through a cross posting to the valve...
ReplyDeleteIts funny how these things vary by discipline. In history the measure of research is the monograph: you are only somebody if you've published a book! (Preferably lots of them). Some schools are catching on and switching to articles but its still measured in book equivalents(4 articles = a monograph).
I am in the middle of (procrastinating) grading my students essays. I wish there was another way for them to demonstrate their understanding of history: like maybe putting on a Mystery Play or writing a haiku about the Industrial Revolution (with footnotes).
Thanks again for the great post!
i agree with the post owner, academic writing takes lots of time and research.
ReplyDeletehttp://academic-essay-writing.blogspot.com
it's really quite interesting post, i enjoyed the comments also.
ReplyDeleteI am into academic paper writing and I love my work so much. In fact, I am also a part time term paper help tutor to students who have a hard time writing. This is a nice post. Keep posting
ReplyDeleteLuis, you and your readers may be interested in this paper I wrote that describes something similar to what you're suggesting, and it references this blog article!
ReplyDeletegrate too see informative
ReplyDeleteThis is a good post which poses an excellent question. The advent of Web 2.0 has certainly given the paper based publications a run for their money. Although these may be publishers which may be considered as authorities on various subjects, the fact remains that the general consensus created by the users on Web 2.0 properties may drive recognition online. Therefore a two-pronged approach where Digg, Wiki, etc. along with conventional paper based publications will create the biggest buzz for any paper.
ReplyDeleteGreat article keep on posting this type of fantastic articles, thanks you guys
ReplyDeleteGet your required Term Paper here....
ReplyDeleteTo draw a parallel from law, the law is created from lots of separate bills that are short pieces of information like papers, most of which just amend existing laws. Thus the law is contained in a set of living documents that are refined through amendments. Each amendment is a well-formed document in its own right, letting an observer see the reasoning for any modification. Some projects now exist to tell you the state of those living laws over time e.g. http://portsea.austlii.edu.au/pit/xml/sa/act/
ReplyDeleteFrom what the competition sounds like, my friend has a real opportunity to build something valued by the industry. I hope he goes for it.
ReplyDeleteTherefore a two-pronged approach where Digg, Wiki, etc. along with conventional paper based publications will create the biggest buzz for any paper.
ReplyDeleteAssignment | Coursework | Dissertation
Thus the law is contained in a set of living documents that are refined through amendments. Each amendment is a well-formed document in its own right.
ReplyDeleteEssay | Thesis
Many institutions limit access to their online information. Making this information available will be an asset to all.
ReplyDeleteMany institutions limit access to their online information. Making this information available will be an asset to all.
ReplyDeleteWhenever i see the post like your's i feel that there are still helpful people who share information for the help of others, it must be helpful for other's. thanx and good job.
ReplyDeleteMasters Dissertation Writing
I really gathered much from this.Thanks for sharing.
ReplyDeleteLogo Design | Logo Designs | Business Logo | Website Design
I should state my result four times: once in the abstract, once in the introduction, cara meninggikan badanonce in the body, and once in the conclusion.
ReplyDeleteA dream is your creative vision for your life in the future. You must break out of your current comfort zone and become comfortable with the unfamiliar and the unknown.
ReplyDeletepayday loans alberta, payday loans edmonton, payday loans ottawa
The reason for this restriction is that, 30 years ago, papers were published using physical paper. Such a restriction makes no sense today.tinggi badan
ReplyDeletenice post thnks for the info. i will be doing my P.hD after two years from now. useful info thnk you all..
ReplyDeleteAfter all, it is the process of writing and choosing the right books to use to support your claim that will create impact on your essay.
ReplyDeleteValuable information which you have discussed in your article.I enjoyed very well while reading.Thanks for providing this for us. Really Great post, I look forward to reading more.
ReplyDeleteCustom Essays Writing Services UK
Writing Services for Custom Essays UK, USA, and Canada
I found this a bit too late. :( I really needed it few days ago. But now it will be in my list.
ReplyDeleteI like the activity in the commenting beginning. It is like second article.
ReplyDeleteyes definitely this type of blogs are really help for people great job.
ReplyDeleteLord Lemon,
Buy University Essays UK, Custom Assignments Services UK, Dissertation Samples UK, Coursework Writing UK, Thesis Proposals & Term Papers UK