Agreed, this just reads like the universities have no idea how to actually serve data effectively. Just bad architecture and lazy or incompetent administrators.
I would encourage you to re-think your comments. You just labelled some people "lazy or incompetent" while having barely any knowledge of the situation, or what resources they have or don't have available. Libraries aren't start ups focusing on scalability, their focus is on preserving knowledge and helping academics and the public find it.
If you believe that they should focus on the scalability required to handle bot traffic perhaps you would be best donating money to them to help give them the resources to be able to do that, or volunteering to try to help understand their problem and fix it if you believe that it's a skill issue.
Reframing what you said in a more positive way could be:
- I wonder what architecture they used for this and how it could be improved?
- I wonder what resource limits they have that prevent them from handling this issue?
- I wonder whats unique to a library that limits their ability to do what tech companies do?
Well I have experience with universities, and their IT is horrendous. Departments are little kingdoms, none of them agree on procedures, architecture, data formats or even care about the absolute basics of software development or security.
It really isn’t a resource problem. It’s that academia is ruled by little princes who are full of themselves.
No one is lazy or incompetent, but they built and use systems that are scaled for pre-LLM usage. Now they need to upgrade and they should. They challenge is of course who pays for the extra capacity?
Couldn’t possibly be that serving data to scrapers isn’t a core function for a university or that they can’t pay big tech salaries to draw sysadmins and engineers that know how to handle large scale deployments
The core function is being obstructed by irresponsible AI bot traffic taking down their catalogs and archives and preventing humans from having access. Bots have never been the intended audience of library content.
There isn’t mission critical work being done on GH. You start charging more and the people in charge of finance at companies paying for GH will actually start paying attention. Those finance people don’t care rn because GH is affordable and helps get stuff done. There’s a threshold that exist at every company and once it’s crossed the people who can make money decisions start asking questions
> key organizer for the Islamist militant group al-Qaeda.
Yeah, I don't see a problem here. Due process, slippery slope sure... but this was a cut and dry case. I've got better things to do with my time than be concerned with this.
I tried NixOS but i'm gonna switch, as there is some stuff that annoys me:
- I couldn't get the language syntax in my head.
- A lot of things need special care on NixOS (python environments, docker, vs codium remote explorer
- it's not a rolling relase
- during every update I usually have to perform the "error, fix, retry, new error, fix again"-dance multiple times
I really like the idea of NixOS, but it somehow doesn't flow for me
reply