How to skim "scientific" papers? Does that include peer reviewed articles

adonis49

May 3

When it's better to be "novative" (incremental addition), not innovative

Benham. Mar 31, 2022

Listen

How Not to Read Scientific Papers

Learn Not wasting your time reading useless papers.

I'm a Ph.D. student and in my job, I get to read/skim lots of papers.

Always being on the watch for new ideas, I'm very comfortable with checking out various papers with seemingly innovative concepts.

The problem is: Often times these papers are not going to be useful for my research, not because they're wrong (well, some are), but because their context is very different than the kind of papers published in my field.

Reading papers from other contexts/literatures isn't bad per se. In fact, if you can read an interesting idea and apply it to your own field, that'll be a novel contribution. (Especially if you consider yourself a quick reader?)

It feels like the low hanging fruits have already been picked, and if an idea isn't used in your field, it's likely because somebody tried it and it didn't work out. (Didn't work out for him. How about trying your hand at it?)

Most people don't even bother trying new methods in their field. ( They dread learning new methods that their peers would Not appreciate: the paradigmatic handicapping tendencies)

For these people, the safest path moving forward is to learn what's already been done in the field and try to make an incremental contribution to it. In reality, most papers in my field are like that; incremental and boring. (Simply, it is the safest and quickest way to be published! Kind of letting their assistants do the work for them)

After reading a number of seemingly innovative papers and realizing that my time was literally wasted reading content that's never going to help me in my field, I now have a much higher bar for even considering to skim a paper.

Every now and then, I see an interesting title which piques my curiosity. But before getting excited about how I can use it in my field for the first time ever, I now try to see who the authors are, in which department, which journal is the paper published in, and in what context (literature). (Not interested in the terminologies of other fields that actually correspond to specific terms in your field)

Doing this, I can safely ignore most papers even if their title sounds very exciting.

But why?

The reason is because a vastly different context/literature necessitates reading a whole bunch of additional papers in that area to understand where the current specific paper is coming from. (Too much backlog reading and Not paid for it)

Do I have time for that? No.

Do I even find those extra papers as exciting as this other one? Probably not.

Do I have a bad feeling about spending my time the findings in another field of study as opposed to improving my knowledge in my own field? Absolutely yes.

That's why I now try to be more conservative in reading papers with seemingly innovative ideas. My moto is: "Not so fast!"

Special strategy for ML papers

I used to get excited about ML papers (what that acronym means?), a lot. But after reading many and realizing that the results are not robust enough to justify the claim in the title, I now have a skepticism about ML papers.

My strategy is:

If the authors' code is not readily available (e.g., on Github), forget about this paper. As Linus Torvalds said, "Show me the code, talk is cheap."
Look at the number of citations. Since I'm not an ML researchers, I use the citations number as a proxy for the soundness of the paper. In a way, it's like trusting the wisdom of the crowds. If a paper is cited many times, it's probably not full of bs. (As if the sited papers are ever read by the author?)
Don't get too excited about new papers published in the past 2–3 years. Let papers stand the test of time. Usually the best ideas are the ones that people keep using for years to come. If I find a paper from 10 years ago but with many citations, I prioritize it to another paper which was published last year but only has 40 citations. (How much time it takes you to do these kinds of search?)
Established papers and ideas often get featured in blogposts, too. This is super helpful because it means there's probably someone out there who can explain the ideas in the paper better, and possibly with his/her code, too. (Now we are talking. The more shared ideas and explanation the better)

These are my criteria for reading papers now. If you have any comment on that, I'll be happy to read it!

Comment