[Rant] REVIEWER #2: How Peer Review is FAILING in Machine Learning
717#ai #research #peerreview
Machine Learning research is in dire straits as more people flood into the field and competent reviewers are scarce and overloaded. This video takes a look at the incentive structures behind the current system and describes how they create a negative feedback loop. In the end, I'll go through some proposed solutions and add my own thoughts.
OUTLINE:
0:00 - Intro
1:05 - The ML Boom
3:10 - Author Incentives
7:00 - Conference Incentives
8:00 - Reviewer Incentives
13:10 - Proposed Solutions
17:20 - A Better Solution
23:50 - The Road Ahead
PS: If it is not entirely clear to anyone already, stealing ideas as a reviewer is against most conferences' code of ethics and I disapprove of any such behavior. I mention it because it is being done regularly and good luck proving it in any particular case.
Sources:
https://thecognitivevortex.wordpress.com/category/phd/
https://susannapaasonen.org/2019/05/31/observations-on-peer-reviewing/
https://www.radicalhistoryreview.org/abusablepast/forum-1-1-on-peer-review/
https://www.meme-arsenal.com/en/create/meme/2012988
https://imgflip.com/i/1pydon
https://uqkdhanj.wordpress.com/2015/02/18/10-best-reviewer-comments-in-meme-part-2/
https://susannapaasonen.org/2019/05/31/observations-on-peer-reviewing/
https://www.memecreator.org/meme/what-if-i-told-you-reviewer-2-wanted-more-experiments/
https://www.emaze.com/@ATFTTRRF
https://thegradient.pub/neurips-2019-too-big/
https://www.videezy.com/backgrounds/6199-switzerland-flag-4k-motion-loop-stock-video
http://blog.mrtz.org/2014/12/15/the-nips-experiment.html
https://twitter.com/tdietterich/status/1292217162103316481
https://www.pinterest.de/pin/192951165261323337/
Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n