This is what happens when an important position to head the Leading AI lab of the company is given to a startup marketing guy, who pretends to be an Lead AI scientist, whose biggest credential is to sale the same startup at exhorbitant valuation to the the same company. This is the price for embracing better marketers over real innovators.
In your bot detection systems I do not see open source system like anubis, safeline or sibuna (I am the author of sibuna). Sibuna is built as self-hosted alternative to cloudflare with console UI providing OWASP CRS, JA4 TLS signature from ingress, these are still necessary besides bot detection and cloudflare offer it as enterprise offering.
I have a smaller benchmark for them in my book at https://insanai.github.io/sibuna/. If you can provide your benchmarking code as open source will incorporate this report into sibuna book.
It works similar to cloudflare WAF but is an open source software. It can be on your own infrastructures even if its a $5 VPS.
The idea is same let the bot or AI agent do more work than it takes to serve that request. Also it needs to follow rules, similar to cloudflare enterprise web application firewalls. It was OWASP CRS and J4A TLS support.
Yes name was anubis backwards and thats the only resemblance between the 2 projects and also I used anubis, safeline for benchmarks compparison. Each of the design records are in the repo if you want to see the evolution and algorithms used.
Its not a rewrite and is more inspired by https://github.com/chaitin/safeline. Wanted a web interface and a distributed waf similar to cloudflare as an alternative. It supports OWASP CRS, J4A TLS trrmination handling. Comes with console-ui and also the proof of work used a different algorithm which is proven to be resistant to quantum attacks.
You can check two projects on top of sqlite using paxos and implement reciprocal rank fusion (RRF) and distribution-based fusion score (DBSF) on top of sqlite with sqlite's FTS and sqlite-vec extensions.
Just you need to remember sqlite support single writer at a time so both needs to work with that limitation. PostgreSQL uses MVCC, so performance for concurrent writes from multiple clients might be better. Did not do benchmark against it so cannot say.
Congratulations to zig team. Good to know they are trying pragmatic approach to LLM now. I left the zig eco-system due to zig core members hostile behaviour toward humans not just LLM, so good to see the change they are becoming pragmatic. For me I am slowly porting the same work to odin language [1].
Background is I created issue and one pull request to fix them in zig compiler version 0.16.1 issue numbers 36812, 36811 (you cannot access them as my account is banned can see my fork at [2]). Respecting the community’s no AI stand. For these specific issues I wrote the issue and code myself and not let AI write it. Spend a lot of time on it. Subsequently without any notice my account was banned because my projects on github using zig uses LLM. This was done without message or any information. My account was banned on zig repository.
I can now understand the other side of coin how bun team might have been treated with disdain when they used LLM.
I wrote an email and left the zig community, have many work in zig but slowly moving them to odin.
I feel personal disdain should not be spilled on to people who are pragmatic on using LLM. I was a very big evangelist of zig for their no LLM stand and promoted them among my community, but with poor treatment by community I just left. You can still see projects I wrote in zig [3].
I have worked with postgreql community since 1997 and python community since 1998. Never felt such hostile community. So all the best and I wish zig continue its progress
> because my projects on github using zig uses LLM
To be clear, we do not block people for merely having LLM-related projects. Obviously we have opinions about LLMs in a broader context, but in terms of rules enforcement, we only care about LLM usage taking place within official Zig spaces.
It's possible you were blocked in error. LLM detection is not foolproof, so unless it's an open-and-shut case, our usual approach is to just unblock if people reach out to us by email.
I would understand if its by error but based on how it was banned it did feel deliberate. Also I have written email to core member before making a decision to leave the community. I think a community can only thrive when they are nice to humans. Here it felt like zig is worse then a oligopoly where few core members can block anyone without any reason and don't even have a courtesy to send a message. I worked with python community since its beginning, same worked with linux community since 1992, never seen such attitude even though both of the projects were driven by benevolent dictators, but they did care about the community around it. I still have one major work [1] in zig, which I will still release but after this won't work on zig. Zig community felt like a hostile ologipoly of few core members without proper due process, when it comes to treating humans.
Well of course the block itself was deliberate. By "in error" I meant that we might have misjudged the text as being LLM.
Who did you email, when, and did you get a response? We've all been very busy leading up to the release, and most of the team is also currently traveling for SYCL.
Regarding sending a message, consider our side: We're tired of having our time wasted on slop, so we don't also want to have to write formal block notifications to people we block on suspicion of using LLMs. I'm fully aware that this can lead to an unfortunate situation like this, however. For what it's worth, this is why we want to move to an invite tree for Zig development; it'll allow us to create a high-trust environment where this kind of suspicion is unnecessary.
Thanks for the reply alex. I send an email to andrew, but not to reinstate my account just to send the details of the incident before I move on.
I wish all the success to zig because diversity keeps the innovation alive.
The reason I moved on is, after 2 times consecutively facing the same issue, I felt I will be helpless in zig community and felt like my voice will be muzzled without any reason done in an opaque and abrupt manner. I prefer to err on the side of openness than caution.
Initially I posted my project on ziggit forum. Respecting the rules, in my post I did mention use of LLM extensively on those projects. One of the post asked "If I used LLM", I did not get a chance to replay, right away my account was blocked on ziggit for 1 year. Thats ok I accepted it, I thought I should respect the community.
In the second incident I opened a small PR with an issue which I fixed for my own project, later I created a PR. I did understand that zig community do not accept LLM written post or PR on their official repository, so I went ahead and wrote the issue myself, also for PR I wrote the code, run the test locally on my own PC, tested if it compiles and pass the test on my osx and linux. Since it was my first PR, did not expect it to be accepted and thought may be some core developer will pick the code and merge or write another code.
Also I created one more issue which was closed by the core developer with a message its fixed in 0.17, I thanked the person.
I saw one issue from another person facing similar issue as mine. I give a comment and also put a code snippet for him to try on his own machine, since on my machine it compiled correctly and created windows cross compiled library on my linux machine. But then suddenly I cannot see my issues, my comments and PR. I initially thought issue with my own system it was not so. Then thought may be codeberg blocked it, but that was not the issue either. Then I spend significant amount of time going through forgejo documentation to find out that a person running a project can remove the person from a repo and their whole history and work will be gone from that repo. I have other repo on codeberg which worked fine with my account, so then tried to create new issue in zig repo to test if I am being blocked.
This is when I realized then my account has been banned by some zig core developer deliberately without a single message, including no message that my account has been blocked. This is like a tyranny of minority and community do not build like this.
A community requires a due and a transparent process even if its benevolent dictatorship. Since it happened twice in a span of just 2 weeks, it really felt like zig is not an open community and breeds strong disdain towards people working in AI or LLM. I then realized that I made too early decision on bun incident, where I took the side of the zig core team, now could understand the other side of the coin.
I have been following zig since it was started, just didn't take the plunge, but once I took the plunge and met hostile community, personally I thought I should move on. Still I believe in diversity of opinion so will still be happy to see zig thrive as a language because I liked its simplicity and also diversity breeds innovation.
I don't think the zig community is necessarily hostile to LLMs, they just don't want it in their "maintained by 10-ish core people not even full time" language impl. Mitchell Hashimoto, a big zig contributor (both money and effort), for example, uses LLMs a lot and no particular shade is thrown.
This is from my own first hand experience both on ziggit forums as well as raising bugs in zig repository. My account on codeberg is still banned on zig repository without any single comment, my issues or pull request raised on that project just went in ether without being accessible, if you see the issue yourself could have made the decision if it was warranted to ban the account, but that right also taken by the core contributor out of spite and disdain for LLM.
I have been in open source world with linux kernel since 1992, never every had seen a community so hostile, especially towards humans who spend time and efforts just out of curiosity, inquisitiveness and trying to support some simple alternative when odds are already against them.
Facing hostile behavior from inside the community made me switch to odin language where they also do not use LLM but are pragmatic about people using it. Cannot comment on Mitchell because I do not contribute with money.
> my account was banned because my projects on github using zig uses LLM.
Its not Zig. Its codeberg. Codeberg has a strict no-AI policy. Codeberg is ALL about the community and not the lone hacker. I think its a good policy to prevent overload and clearly distinguish project ecosystems from each other.
I mean I pr'd something (and it was not accepted) but the grounds have nothing to do with LLMs. I'm pretty vocal about using LLMs to write zig code. Obviously I did not use LLMs in my PR
I would be fine if they reject the PR, given I was first time raising it to fix POSIX related issue. It was just few lines of code. But my account was banned all issues removed my own PR not visible and looked like tyrannical approach not a benevolent dictator approach. I think being nice to another human being is the basis for building community. Here felt like if you are among the few who dictate the terms there is no process just my way or highway.
It is not codeberg, I can work on other repository properly on it. Its only zig repository where my account is blocked. My initial thought was same as you, only later realized it. Also on ziggit these people did the same even though based on policy when I put the post i clearly mentioned use of LLM. This all without any proper communication. After working on zig for over 6 months, now I am moving slowly to odin [1] language. This decision I did not take lightly, I had spend significant resources in zig earlier and now moving away.
don't let people gaslight you here. i mean, look at how the creator treated a person who was a zig evangelist and was donating tens of thousands of dollars to the project:
"Jarred was already writing slop well before he had access to LLMs."
"Jarred was a stinky manager. Poor communication, unrealistic expectations, low empathy, no experience. Just a total shit show"
"When Jarred announced the Rust rewrite, we were ecstatic. It seemed too good to be true. I have to admit, I didn't think the technology was there, to pull off this stunt. But he did it, and now I'm metaphorically sipping delicious tea from a mug that says "It Tastes Like It's Not My Problem Anymore"." [1]
or what about when he called people "monkeys" because he didn't like their product [2]
or when their vp of "community" called github employees clowns [3]
so don't let people tell you that there isn't a pattern here. and when this kind of behaviour comes from the top it shapes the rest of the community
Irony an article written by AI, wants to say do not use AI for writing. Following sentence looks like written by an AI, human do not use double dash, it seems AI instead of emdash wrote double dash;
“I think you should almost never use AI to write -- that is, to do the thing you’re doing when you type words on a page -- whether for a blog post, a research report, a memo,”
This absolutely doesn't look like AI writing and this is a pretty weird signal to use.
I have a much better theory: as it happens, Substack automatically converts "--" to em-dash as you type. I'm guessing the author knew that and wanted to use this feature, but composed the article in another editor and then copied-and-pasted the whole thing. Because of how Substack implements the feature, the substitution doesn't actually kick in on Ctrl-V, so all these "--" remained in place.
Haha, I do actually use double dash, as I write in org-mode. I then convert that org-mode text to Markdown and copy and paste the Markdown into the Substack editor, which preserves the double dash. In case it needs to be said, the article is 100% human-written.
In some of your other writing can see use of emdash probably thats also intentional, emacs might have very special key combo for it to use unicode emdash instead of ASCII “-“. I commented about double dash (“--“) because because thats what LLM does when asked not to use emdash for writing.
“Another promising approach is to conduct on-site inspections regularly—say, every six months— to make sure chips are not just installed to fool inspectors and then removed and smuggled, as has happened before. However, conducting on-site inspections at this scale would likely require additional budget for export enforcement, or for US authorities to implement a third-party export auditor program.”
Yes, for that newsletter, where I write in my day job as a policy researcher, we use em dashes. For my personal writing (example: https://www.lesswrong.com/users/erich_grunewald), I use double dashes.
Double dash has long been the way to clearly indicate in plain ASCII text an em dash as distinct from a simple hyphen. So I think the author here is here emphatically reclaiming the use of the em dash while pointedly not using the Unicode character so beloved by LLMs.
Comments like these do injustice with Indian constitution which grants:
1. Protection of Language and Script (Article 29)
Article 29(1): "Any section of the citizens... having a distinct language, script or culture of its own shall have the right to conserve the same.
Article 29(2): "No citizen shall be denied admission into any educational institution maintained by the State or receiving aid out of State funds on grounds only of religion, race, caste, language or any of them."
Article 30(1): "All minorities, whether based on religion or language, shall have the right to establish and administer educational institutions of their choice."
Article 350: "Every person shall be entitled to submit a representation for the redress of any grievance to any officer or authority... in any of the languages used in the Union or in the State..."
Article 350. Language to be used in representations for redress of grievances- Every person shall be entitled to submit a representation for the redress of any grievance to any officer or authority of the Union or a State in any of the languages used in the Union or in the State, as the case may be.
Article 350A. Facilities for instruction in mother-tongue at the primary stage - It shall be the endeavor of every State and of every local authority within the State to provide adequate facilities for instruction in the mother-tongue at the primary stage of education to children belonging to linguistic minority groups; and the President may issue such directions to any State as he considers necessary or proper for securing the provision of such facilities.
Article 350B. Special Officer for linguistic minorities-
There shall be a Special Officer for linguistic minorities to be appointed by the President.
It shall be the duty of the Special officer to investigate all matters relating to the safeguards provided for linguistic minorities under this Constitution and report to the President upon those matters at such intervals as the President may direct, and the president shall cause all such reports to be laid before each House of Parliament and sent to the Government of the States concerned.
The only unequal part which is only applicable to Hindi and not other languages:
Article 351. Directive for development of the Hindi language- It shall be the duty of the Union to promote the spread of the Hindi language, to develop it so that it may serve as a medium of expression for all the elements of the composite culture of India and to secure its enrichment by assimilating without interfering with its genius, the forms, style and expressions used in Hindustani and in the other languages of India specified in the Eighth Schedule, and by drawing, wherever necessary or desirable, for its vocabulary, primarily on Sanskrit and secondarily on other languages.
Amendment Act Year Languages Added Resulting Total
Original Constitution 1950 14 Original Languages 14 languages
21st Amendment Act 1967 Sindhi 15 languages
71st Amendment Act 1992 Konkani, Manipuri, Nepali 18 languages
92nd Amendment Act 2003 Bodo, Dogri, Maithili, Santhali 22 languages
96th Amendment Act 2011 "Oriya" to "Odia" 22 languages
Doubt in Germany a user can submit documents in French and can get have special protection by Germany for promoting French language.
These laws mean zilch - the state of Karnataka, which hosts Bengaluru, where demagogues routinely raise Xenophobia based on language, has exactly 1-school that teaches Science/Mathematics at the high-school level (PUC, 11/12th) in Kannada.
Try using these "laws" and "rights" when you're trying to apply for a university, or asking a IIT to teach you in Hindi/Tamil LOL. These provisions are mostly used to skirt around the (generally onerous) laws by so-called "minority institutions" (eg. Manipal).
It's very disingenous to make such arguments by anyone with their eyes open in India. It's almost like the classic quip (almost a criticism of proto-liberalism) about the British empire: everyone is free in it as long as he's British!
Indians will go to Germany and learn German; go to Japan and learn Japanese; but will resist building anything of value with the culture/language of their own states of residence. The Anglophone-Indians will religiously preach a particularly seductive form of nationalist-liberalism internally for India that continues all the colonial structures of the British empire, because it helps them optimize for their own economic outcomes (ofc. they do this internationally too).
Absurd state of affairs tbh - and it has (eg. Blume Ventures' report) and will keep India poor with the wealth limited to ~5-10% of the population.
> Indians will go to Germany and learn German; go to Japan and learn Japanese; but will resist building anything of value with the culture/language of their own states of residence. The A
I wish they do. If you read Facebook group Indians in Frankfurt - you will the entitled Indians - especially those coming for IT jobs - in deputation for Infosys etc (that speak in many ways not the best english) trying to talk in English with all bureacrats or even 85 year old - and expect - every damn plumbber or cleaner to speak English - as if they have scored 99.% in TOEFL or so.
They constantly say: See in India everyone speaks English. why can't you all speak English? When I point out if your go to Coimbatore and speak English in Tahsildar office or RTO - you will get f*ked.
I know the feeling because - I am from South - grew up in North. average person has so much hatred. Until you fix that you cant do anything.
> but will resist building anything of value with the culture/language of their own states of residence. Th
You have some horrible feeling for UK. Forget that. Go and tell Narayanamoothy or Modi to give jobs in software (and every other whitecollar) exclusively to people that have linguistic ability.
Blaming others is the escapism that you Indians are happy for.
> You have some horrible feeling for UK. Forget that.
You have a serious issue with general comprehension, sir.
I've not once brought up UK or Western countries - my issue was/is/will-be with the Anglophone-Indian elites (who think they're honorary part of UK now ?), who will paint themselves as saints to the world with M.K Gandhi's face, but who are vicious corpulent vessels of the most vile forms of racism from the worst periods of the British-empire of 1 1/2 century before. India has more British laws inherited from this period, while UK itself has completely transformed itself (including the Civil-Service).
I'll not reply to the rest of the screed - that well-studied pedagogical gap of having to spend inordinate amount of time "muggin-up" things in English instead of actually understanding things means that I can't get much across anyway...
Last I checked, the Union Govt is not going around imposing English everywhere like they are pushing Hindi.
Also India's wealth is always going to be limited to 5-10℅ of the citizenry. India is pretty much USA on steroids at this point - a handful of capitalists controlling a lot of public infrastructure, ministers openly indulging in corruption, routine exploitation of the working class, piss poor healthcare and education provision where they try to skirt around the evolution question and Gandhism. The bowing down of India to the US is no surprise, given all of these other factors already playing around in the background.
> Doubt in Germany a user can submit documents in French and can get have special protection by Germany for promoting French language.
I did submit a non-German language document to get my German citizenship.
At the end, whatever you say - India is f*ked up by regional politics. With English a poor person from region A can go to B and find some job/livelihood.
Are you against it? I understand you want homogeneity - it is not there.
But people like you creating divisions by telling
>>>> It shall be the duty of the Union to promote the spread of the Hindi language, to
will promote more chaos. Good for rest of the rich world especially west and China.
Like my physics teacher used to say in Chennai... We don't need foreigners to mess us - We Indians do it for us.
All you want is India to fight like Arab states - internally. Divide and conquer. Thank you from the West and China.
reply