I think to solve this problem properly, we have to stop treating "build" and "test" as separate buckets of work, to be designed and scaled separately. It's all CI. As soon as you try seriously scaling out tests, you will run into build bottlenecks. To truly scale CI you need a scheduler that understands your build, test environment, and all the glue in between, well enough to schedule it in a way that actually speeds things up. That is very difficult and not something that even the best build tools can do - yes, even Bazel. Bazel can run tests but it's not nearly as good at it than at building.
Yes. Bazel is half of the equation. The other half which other enterprises rely on is the server side of Bazel's Remote Build protocol. There, the scheduler can be implemented with logic that accounts for sandbox sizing as well as other requirements/constraints.
IMO it's a bad idea to tie your entire CI stack to a single build tool, no matter how good it is at building. It immediately puts on ceiling on what your CI stack can do, and for whom. You're essentially making the bet that 1) Bazel can build and test all the things with no exception; 2) you can convince, or force, 100% of your team to adopt Bazel for 100% of their builds and tests, in perpetuity; and 3) this drastic limitation on choice is the only way to make CI sufficiently fast and reliable.
I see how that bet makes sense from the point of view of a Bazel maximalists, and even more so from the point of view of a hosted Bazel vendor like BuildBuddy. But I can already tell you that it will not happen. Bazel is an amazing build tool, but on the testing side it doesn't offer anything that special. Tying yourself to it for tests is not worth the loss in flexibility.
I think you're being sarcastic, but there is actual truth behind what you're saying.
Because every developer is now a slop cannon by default, by default users will experience churn and whiplash, and things will break all over the place. As you point out, this is bad. It's also impossible to fix without deploying agents on the QA side. Like it or hate it, agentic testing is inevitable to protect users from the churn and noise caused by the slop cannon. I don't think that replaces test engineers at all - if anything it makes the job more fun. If you've ever had to keep playwright tests in sync with the target manually, and kept the CI environment up to speed with toolchain changes, you know what I mean.
Whether the "slop cannon by default" situation could have been avoided in the first place, is another question... But we're here now and there's no going back. Might as well deal with it as best as we can.
Think of it as a layered problem. If the bottom layer (CI) cannot keep up with the output of agents, then solving problems at a higher layer - like user experience checks - will be exponentially slower and less reliable. Kind of like how optimizing tight inner loops makes your whole program faster.
QA is not automatable. QA is about finding the things that you didn't think of and so didn't write a test for it. That many people think qa is not important shows in the bad software we have.
> QA is about finding the things that you didn't think of and so didn't write a test for it
QA is about quality. That it has been increasingly used to mean "repetitive manual testing only" is part of the very trend you decry of qa being undervalued.
Fundamentally QA is about two groups of people collaborating in an adversarial process ("you build it we break it") to achieve a common goal: a better product. QA by definition requires a relationship of equals, or the process ceases to be adversarial and becomes useless ceremony. If you're not able to assemble two distinct groups of people, one person can wear both hats (ie you test your own software) but you'll have more blind spots.
How much of QA is done by a human or automated, is and has always been an implementation detail.
If you're in charge of QA, you're in charge not just of finding new problems, but preventing regressions as well. And you're responsible for embedding as much of that work into the devloop itself, so that builders can find and remediate problems earlier and with less pressure on your limited time. How do you achieve that? automation.
I started my career doing QA and 99% of what I was doing could be automated by an AI today. Today I still do QA on my own product- I just spend all my time on the remaining 1% and my product is better as a result.
Only is a bit too strong, but it was a lot of repetitive. There has long been a trend to try to automate it - I sat in sales pitches for automation tools in the 1990s, and we were already had long been writing scripts to automate a lot of testing. While the tools have got better, the only advance since the 1990s that I have heard of is automated unit tests (which were being developed then, but hadn't yet spread). We are slightly better at using the tools, but automation of testing is a very old idea.
Despite all that, it was understood if you want a high quality product you will be spending more than half of your development budget on testing.
I have long ago concluded that it is not possible to test my own code - and nobody else can test their own code either. I know how to test code - like you I started in QA - but I can't test my own. I have too many blind spots because I'm too close to how it works.
Yes the blind spots are a hard psychological barrier...
I actually think that for a lot of software, AI can really help with that blind spot, even beyond regressions, in ways that don't replace human testers but are complementary.
For starters, not all software is used directly by humans. A good chunk was already consumed by APIs, and now an even larger chunk will be consumed by AIs. In those cases, it's very possible that AIs will be better at QA than humans, even for original problem discovery. After all, they are the users.
Even for human-facing CLIs (the kind of software I develop these days), it's trivially easy for AIs to interact with the software, and in my experience the newer models are shockingly good at understanding the principles and conventions of CLI usability, at least in the Unix world. I routinely run CLI changes through a gauntlet of AI agents, and their feedback is genuinely good.
TLDR I think in these debates over how automatable QA really is, we often tend to forget that there is a lot of software out there, and not all software is tested the same way.
No. You can automate some parts of it, but there are a lot of parts that cannot be.
What you cannot automate are finding the "unknown unknowns". That is things you don't know to look for. I have yet to see a project where humans running the automated tested code cannot find large large number of bugs that slipped passed all the automated tests that already pass.
You absolutely automate every test you can. It is boring and tedious to run through a manual test plan. What you want is to tell the testers "find what is wrong with it" and let them figure where/how to look. The act of finding new ways to try things is a large part of what makes manual testers better than automated tests.
AI is very useful in review, I won't put anything up for PR until it passes AI review.
However AI is just a tool, it isn't magic. I still find a lot of things that AI missed when I review code.
AI can likely do a lot of QA work, but I still want a real human to test the code. At least code that humans are expected to use.
Reports are published under the Institute’s name rather than an individual’s. That is the convention at institutional publishers, and it fits how the work is made: every report is produced against one standard and one citation bank. A reader is asked to weigh the sources, which are named in the sentence that carries each figure, rather than the author.
> A small disclaimer at the bottom of the webpage notes that the organization was created on behalf of the Israeli Government Advertising Agency by Piro, Inc, a firm co-founded by Daniel Rosenberg, the producer of Spike Lee’s “Inside Man.”
"Small disclaimer" implies that the site is attempting to hide its affiliation. But their "about" page, which is linked at the top of every page, is quite clear:
WHO FUNDS THIS WORK
The Institute’s materials are distributed by Piro, Inc. on behalf of Havas Media Germany GmbH, acting for the Israel Government Advertising Agency (LaPam). That relationship is registered under the Foreign Agents Registration Act, registration 7732, and every report published here is filed with the Department of Justice, where the registration and the filings can be read in full, and the funding page sets out when the relationship began and what the funder does not decide.
That registration is why this site does not call itself independent, nonpartisan or neutral. What it offers instead is a method a reader can check: every figure carries the body that produced it, the year and the number, inside the sentence that makes the claim; sources that are a party to the events they describe are labelled as such and their counts are never presented as independently verified; and where the evidence is contested, the competing findings are given by name and date rather than settled. No report recommends a policy or tells a reader what to conclude. Those are properties of the work that can be tested against the work, which is the only kind of claim worth making here.
> The institute’s “data reports” have footnotes and tables of contents, and they present arguments in a neutral tone, helping them appeal to chatbots like Claude or Gemini.
So, they're guilty of good SEO?
> Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as "AI Story Optimization." Others refer to this practice of influencing artificial intelligence as “LLM poisoning.”
So the issue is that they use a LLM-friendly SEO service?
> Many of the reports are formulaic, starting with an innocent question that someone might ask a chatbot.
> “What Caused the Displacement of Palestinians in 1948?”
> “Which Humanitarian Organizations Have Documented Israeli War Crimes?”
> “What is the Current Situation in the Gaza Strip?”
> In an article titled “Is the IDF the World’s Most Moral Army?” the Hanover Institute cites a 2022 poll that found that 47% of Israeli Jews believed that statement. Another report casts doubt on UNICEF's assertion that “90% of water and institutional infrastructure has been damaged or destroyed” in Gaza. Many of the reports conclude by linking the topic to rising antisemitism, oftentimes citing the same studies.
So, the accusation is that they frame their opinion articles as a Q&A, and focus each article on a single question?
I've read these articles: they are well written and make a visible effort to provide sources, while being transparent about the fact that they are not neutral.
It's perfectly fine to criticize the actual content of these articles. But attacking their credibility by accusing them of "LLM poisoning" and implying that they're disguising their affiliation, is ridiculous and should be called out.
Thanks for this, it's crazy that I had to scroll so far to see it on this site.
The headline sounds sensational, then if you actually read the article you realize the author just made up his own framing entirely, or he just discovered that AEO and GEO exists and thinks it's sinister?
I thought maybe there were going to be some markers about how they're targeting LLMs specifically there was nothing like that.
And if you go to https://hanoverinstitute.com/llms.txt ...honestly, you could maybe form a critique about implying to LLMs that your wagon is hitched to the Department of Justice on line 5! But this guy didn't even seem to find this file...
"Israel creates fake think tank" is the headline. What is a "think tank"? What distinguishes a real one from a fake one? Does "the Hanover Institute" claim to be a think tank?
The author does not answer any of these questions, only saying "At a glance, the Hanover Institute for Public Policy looks like a new think tank dedicated to Israel/Palestine": in other words, he glanced at it, decided it looked like a think tank, then decided it wasn't a "real" one, and then went from there. That is very much the author's framing instead of something grounded in objectivity.
>The article clearly explain the information poisoning
The article says "Others refer to this practice of influencing artificial intelligence as 'LLM poisoning.'"
But the Anthropic paper that it cites to support this claim never discusses persuasive content, propaganda, or credibility-mimicking material at all; it uses "poisoning" for a technically distinct attack class. So the citation doesn't support the claim as written.
>stop polluting this thread in an organized and selfish self supporting way. so typical
What do you mean by this? How am I organized? Why is it selfish to criticize an article?
everything I posted was in response to you; nothing is random.
>confused informations
Just saying they're confused is meaningless; if you think I am wrong about something, feel free to give actual rebuttals.
>tentative
This word is defined as "not certain or fixed; provisional" or "done without confidence; hesitant". It is an adjective, not a noun. I genuinely do know know what you mean when you use this word.
>a tentative of misinformation
>>What does this mean?
claiming not knowing what tentative means in english is clearly on purpose and could anyway be translated if it was a language problem.
again here:
>tentative
>>This word is defined as "not certain or fixed; provisional" or "done without confidence; hesitant". It is an adjective, not a noun. I genuinely do know know what you mean when you use this word.
Everybody knows what a tentative is and also looking it up and reporting a single possible translation of its meaning is just ridiculous.
again stop dragging the conversation off topic and stop on purpose provoking.
the repeated statements clearly confirms your negative intention and your previous content does not add any value.
for the sake of clarity to the readers I will report here the points more clearly again:
- mentioning SEO,AEO or other stuff here is clearly a tentative of misinformation and confusion of readers. moving the attention to an off topic relationship.
- the claim "the author just made up his own framing entirely" is clearly an accusation without any reasoning or logic backing it up
- The article clearly explain the information poisoning, fast paced publication and ties to government, nothing else. and thats the main point and target.
I will not waste more time on replying to purposely fabricated statements or questions.
I have spoken English my whole life and have never heard anyone use "tentative" as a noun. Why must you assume I am trolling, why can you not believe what I am saying? Is it possible that you are using the wrong word? It seems noteworthy to me that you did not actually answer my question or provide a definition for how you are using, you chose to assume bad faith on my part instead and use that as a basis to dismiss the actual arguments I made.
That said, I would be happy to stop arguing about the word "tentative". I would much rather you engage with the substantive arguments that I made.
>the repeated statements clearly confirms your negative intention
What repeated statements? If it's important to you for understanding then pick something that I repeated and I can try to explain.
> ::your three list items::
These are just repeating of arguments that I made and offered responses to. You have chosen to not respond to them.
Let's pick one and try again:
> The article clearly explain the information poisoning
If you search the article for "poison", there is exactly one reference: "Others refer to this practice of influencing artificial intelligence as 'LLM poisoning.'" The word "refer links to an article by Anthropic that is categorically unrelated. Go read that article and answer: does their definition of "poisoning" match what the author of this post means?
I would say no: the article says "malicious actors can inject specific text into these posts to make a model learn undesirable or dangerous behaviors, in a process known as poisoning", and then follows up with an example of "introducing backdoors". That is very different than whatever the Hanover Institute site is doing.
If you would prefer to argue substance, answer me that: did the author cite that paper correctly? I am saying "no" and have given a reason, what is your response?
hah you make me laugh, trying really hard, so typical from your society!
problem is the world now knows, no need to cry about antisemitism.
keep trying, the truth is clear, you are only doing bad evil things since long time and your government and ministers states it publicly too!
If my ministers would claim the dream of killing kids or people I would feel totally ashamed of them and I would not support it online spamming every thread!
I haven't lived in Israel for more than 10 years, it is not my society, and yet I am ashamed of the current goverment and never ever have I supported them online. I have done more in my lifetime to oppose these people than you will do even in 10,000 years, in campaigning, in arrests, in protests and in court. I wish your posts made me laugh too, but they actually fill me with sadness.
nice try guys, no need to generate confusion also here, the purpose target and implementations are very clear from the article and the actors behind even more.
please do not pollute this thread with misinformation too.
this clearly a defensive stance without a real motivstion.
"It's perfectly fine to criticize the actual content of these articles. But attacking their credibility by accusing them of "LLM poisoning" and implying that they're disguising their affiliation, is ridiculous and should be called out."
called out? yes your defensive and not-motivated stance should be called out and this comment flagged.
disguising affiliation and poisoning information is exactly what happens in a propaganda campaign and in the activities carried and clearly explained in the article.
