A "good" academic paper is defined as "presenting a new idea". Half of a good paper is about content: how great is your idea? That alone is a difficult task, highly controversial, and a gut feeling.
The other half is how well the idea is presented. Every presenter, book and teacher can present new ideas in something you can understand, given enough time. But nobody has time, so a paper has to be incredibly easy to understand, which makes writing a good paper exponentially harder. A paper also has to explain not only the idea, but it's context. A ground breaking climate model is useless unless I know why I need one.
A "good paper", needs to "present a new idea in an easy to understand way".
Present doesn't say much nor does it imply any kind of usefulness for the reader. Press releases present something new in an easy to understand way, but most are useless. The whole point of a paper is to expose your idea to the rest of the community, so that they may be able to use it to solve their problems. Perhaps we should be trying to "teach" someone. But that isn't the case either as teaching implies a hierarchy - a teacher/student relationship. In reality, they are colleagues that have equal weight, and you are hoping that they will like your idea. Which maybe means you are trying to sell a new idea. But "selling" has a horrific connotation of stuffing something down your colleagues throat. The only word that I could think of that implies respect, while still explaining an idea, is inform. A paper isn't trying to teach, or sell, but trying to inform a colleague of a new idea. We can change our definition of "good" to:
"Inform a colleague of a new idea in an easy to understand way"
This is a fairly abstract definition, but a good starting point. The definition of "new" is murky and depends on the field. How much stuff has to be new? Does the whole system have to be new? Does every paragraph have to present a new piece of information? I'm not sure, and I don't know how to define "new". My guess is, it'll be like what the U.S. Supreme Court said about explicit images: "I'll know it when I see it". Just make sure in your paper, the reader can see the "new" part. Put big circles, stars, lots of noise, and in big bold letters, the "new" part of your research.
The hardest part of writing a good paper is making it easy to understand. The most difficult part of making something easy to understand is culture. Even determining the responsibility of understanding something is cultural. In Outliers, Malcolm Gladwell explains the notion of "transmitter" versus "receiver" oriented communication. Most western or "transmitter oriented" cultures say that the writer has to ensure that the reader understands the message. In "receiver oriented" cultures such as eastern cultures, the person listening has to decode the message. Since most scientific papers are published in English, a "good" paper means it is the responsibility of the author to ensure that the reader gets it. The paper has to be proofread by many different people, rewritten, and edited many, many, many times. Most papers are difficult to understand because they are just written, not edited.
Culture also brings up the issue of writing style. I'm unfamiliar with how all Europeans write English, but American English is very different from English written in Germany/Austria. American English, at least after speaking with friends who majored in English, is good when it is concise. Short and simple. On the other hand, Germanic English, is concerned with being "precise", or adding lots of details. And by precise I mean that every thought related to the sentence is explained, in the sentence, with every nuance covered trying to eliminate all possible avenues where a point can be attacked, which of course creates sentences that are strung together to make a very long sentence without the use of periods, making each sentence difficult as an American to read. (eg. The JIT does xyz versus A JIT compiler can do xyz. We do escape analysis versus we do escape analysis for single threaded programs). American readers will hate your writing if you make it overly precise (me!) and Germans will hate your writing if you make it short and concise as your sentence is no longer "precise". I'm sure there is an "Asian" version of English, but I have no idea what that is yet.
The other non-culture hurdle of making something easy to understand is deciding on what is background information. How much can you assume someone knows? Most people guess too much. Dumping all the background information in the world makes a great textbook for an undergraduate course. Assuming everything will ensure that nobody knows what you're talking about. Try to imagine someone who has taken one or two courses in the subject and nothing more. If in doubt, give an extra sentence for background explanation. Don't feel bad if you scratch your head over what's background information because it's a really hard problem. (A website that contains a list of terms and definitions that everyone in the field should know has wiki potential written all over it.)
The easiest way to overcome these hurdles is to use examples because it solves the problem of abstract ideas, background information, and writing style. As much as scientists say they love the abstract, people can't think that well in the abstract. Self contained examples solve a lot of the background information problem. And many writing style issues go away if I already have experience with the example.
The best examples are the ones that start out with a simple case and slowly add new ideas, creating a holistic picture. It's hard to create one example to cover everything. But if you can't do it, your paper isn't focused enough or you haven't thought hard enough. Keep thinking.
A good academic paper, needs to "inform a colleague of a new idea through concrete examples". I've only written two papers, read too many, and I'm sure my definition will change with more experience, but that's seems to be better than most after two years in grad school.
- How to Get Your SIGGRAPH Paper Rejected
- How to write a good research paper and give a good research talk
- How to write a good [systems] paper