I'd love to have one that randomly fails, so we could debug e.g. databases and other apps that are supposed to be resilient. I'd call it "havocfs". Doesn't corrupt data, just randomly fails reads/writes/etc. Some software is supposed to handle that gracefully. Would be a great test tool...
I feel the pain. We have been delaying release because sooo many bug reports have come in. AI is finding a lot of obscure, and interesting, and annoying bugs.
LOL I love this and also hate it. You are the best and also the worst. I am actually laughing :D Who cares about deepening our understanding? Curiosity? Hoping to understand some deeper connections/meaning? Let the AI write 10 papers to top conferences per year, and be done with it. If this is what people think science is, that's what we should do. Incentives -> outcomes, in the purest form. Loving it. And hating it!
Fair, but I just don't know how to fix it. Eventually, whatever metric you come up with, it will be gamed -- p-hacked, if you will. Not sure what the solution is. I am just dissatisfied with whatever is going on, and I felt like sharing it. Maybe you have some good ideas? It'd be nice to fix this...
You'd have to look at the US constitution: set groups against each other. Off the top of my head you have the 3 groups of Researchers, Referees/Publishers, Grant Funders
Referees/Publishers are set against Researchers, in, I dunno here, that their incentive is to make sure the researchers are wrong in some little way.
Researchers are set against the Grant Funders. Like, maybe you have to elect them somehow?
Grant Funders are set against the Publishers in that they select journals for funding. Look, it's Monday morning, I'm grasping at straws here, alright?
Anyways, take whatever many groups you have, at least 3 for proper roshambo mechanics, and set their incentives against each other, round and round, and such that at the end, you have 'Science' spinning towards greater public service.
FWIW, I thought that initial quote was very relevant but was also glad you didn't waste a paragraph explaining what everyone knows, that over-emphasizing publication and citation counts and grant money raised is killing academic research. I don't have good ideas either. In the age of AI, cheap metrics and metric hacking are only going to become more prominent.
I'm sad to hear that. I know of a PhD student who confided in me that his professor didn't believe his suspicion that the results of a well-known, supposedly top tool, are wrong. According to his view, not being able to replicate or improve that tool has prevented him from being allowed to graduate. While I cannot confirm this, I suspect there to be a grain of truth in the story. The results are indeed wrong, but I suspect the results are too big to fall -- too many people made too many mistakes (reviewers, editors, awards committees, hiring committees, etc.). It's sad.
It's a fine line. Fraud? No, they are just sloppy, and so one questions whether they ought to be amended, retracted, or a note by the editor attached. The solution you propose, about a follow-up paper is that who will accept a paper that simply demonstrates that the original work was so full of bugs/issues/misconceptions, that many/most of the claims of the paper don't hold? Would you accept that paper into a top-rated conference? And if it's in a 2nd or 3rd rate conference, wouldn't most people question it authenticity (remember, 2nd and 3rd rate conferences have less rigor in reviews) or simply not hear about it? I think these are legitimate concerns.
A slightly sad side-effect of the follow-up-paper idea, is that it sometimes means that the authors get one more citation, and furthermore, since some of the authors may themselves write part of it, it would mean even more papers by the authors, potentially further cited. So we may be unfortunately incentivizing sloppy research -- the authors can now make two papers: one sloppy, and one fixed. Not sure that's great.
I am not saying what you are proposing is wrong. But it has side-effects that are not great.
What’s the difference between sloppiness and fraud when you stand behind the paper all the same? I would argue that sloppiness in academia is a form of fraud. It’s one thing to be bad at writing, and another to have poor methodology
An honest researcher who has made a mistake on a matter of substance will seek to correct the scientific record, even at a cost to their reputation. Not doing so breaches professional ethics and is arguably worse than plagiarism, because of the potential to mislead.
Obviously. Nice people by definition act nicely. It's all the rest that need the right incentives. And the nice ones also need good incentives else their niceness ejects them from the competition if the competition is set up in perverse ways.
It's like as if someone talked about tax fraud or "tax optimization" and you retorted that an upstanding business owner doesn't cheat on their taxes and even donates some extra funding towards public causes and if he makes a mistake in the taxes he quickly publicly admits it and pays the fine. But if honesty bankrupts a business, all businesses that stay alive in such an environment will be ones that do some form of tax fraud. No matter how much someone stands there wagging their finger and declaring from the cathedra what upstanding business owners do, in the abstract.
Perhaps I can educate you a little further about the academic world.
There are indeed constraints other than one's own personal or professional virtue. The principal one is that if your work does not convince your professional peers, it will not get funded or published.
You can't just invent a bogus research program out of thin air, hold a press conference, and expect to get hailed as the new Einstein.
For example: claims of cold fusion, room temperature superconductors, and similar marvels appear fairly frequently and are quickly knocked down because experts in the field can tell they are false. (NB these are not necessarily frauds, just bad science.)
There are also institutional constraints provided by the universities and professional societies to which academics belong.
Scandals about plagiarism and fraud only hit the news when these institutions take action against senior academics. This typically only happens in the worst cases where there is a clear pattern of behavior and the guardrails have obviously failed.
IMO long running and successful frauds like Cyril Burt tend to already have the benefit of a well-earned professional reputation, but have gone about boosting their credibility in an unethical way.
Small bore fraud is hard to catch. It's like padding your resume. If you say you won the Nobel prize, people will smell a rat. But a lot of it probably gets under the radar.
I've known it from the inside for quite some time. I'm not saying you'll be hailed as the next Einstein just randomly. What I wrote is much more subtle than that.
You simply declared in your comment what the ideal researcher would do and how other courses of action are bad and other normative and moralizing "ought" stuff. I'm more interested in observing what people actually do in the wild, like David Attenborough. Is-ought, prescriptivism-descriptivism.
> My Chase bank account was hacked early this year despite having 2 factor authentication, and when I contacted them to ask how, they said because the person used my actual driver’s license to verify their identity and remove my security features from the account.
I feel like that should require an in-person visit, as troublesome as that might be. A picture of an ID is not the same thing as presenting the actual ID
I almost mentioned that situation in my reply, but that's kind of the exception that proves the rule. Even in that case, someone should need to physically intervene, especially with high profile people like politicians or celebrities
I agree, particularly when it comes to removing security features. However these days it's often possible to open an account without ever showing up in person. So I think the requirements would need to vary per-account and ideally be selected by the customer. (Personally I would opt out of any and all account changes without an in person visit if I could.)
Sometimes I feel it's a blessing that for even a minor change my banks make me visit the branch and ask for my real Govt issued IDs (often more than one) while also make me do the online auth (Govt ID; needs OTP and biometric) and still make me submit self-attested print-outs and then add a delay to it (and even reject once in a while if they feel the signature didn't match..and then the cycle restarts :D). I know, I know - there is a solution of this on hn and it will even scale handsomely. But I'd rather have my hard earned savings safe.
Wouldn't this also mean Gmail, Facebook, etc are no longer safe? The person can simply provide this documentation as proof they own the accounts and claim they were hacked.
> "The source brought it to my attention because the proprietor of this identity theft service offered my Virginia drivers license as a free sample in their initial sales thread on Exploit."
Mr Krebs is dealing with all this mayhem and idiocy with remarkable sang froid if I may say so. Good heavens.
Lol in Germany, internet is soooo slowwwwwwww. I used to live in France, and Free (i.e. free.fr = kind of a bad choice of name) was absolutely demolishing the local big carrier (France Telecom?), but in Germany, there is very little competition to Telekom, so it's really bad.
