> A great example of this is the recent "binary verification" approach, where instead of proving facts about the programs themselves, we prove facts about the compiled binaries of those programs.
This is an interesting argument; I can see pros and cons here.
I'm not a formal-methods guy, but I'd imagine the reason that you might want to prove facts about the programs themselves is that they encode higher-level information "for free" -- such as, this function returns a Person.
IIRC, there was research into special compilers that would produce TAL: Typed Assembly Language. The idea was that TAL could carry annotations that would be helpful to provers.
As far as I understand, the binary-level verification lifts high level specifications to simpler binary-level specifications. For instance, imagine a simple property:
```rust
fn prop(a: X, b: X) -> bool {
a == b
}
```
You prove that the compiled `prop` function will always return `bool`, which might be that a particular register gets written `1` after every call to `prop`. Maybe TAL would allow for more complex properties to be written at the binary level instead of doing this trick.
Exactly. Binary verification can often struggle with problems that are completely trivial when looking at the source code. For example "where can this indirect JMP jump to?" requires full-program analysis and may worst-case require solving the halting problem, but "what code can execute when I invoke this trait function?" is easy.
I would also say that there are more single points of failure on the internet than there were 20 years ago.
In 2006, Amazon had just launched this new storage service called "S3." It would have struck people in 2006 as bizarre to think that AWS (and Cloudflare, and GCP, and friends) would one day be so crucial to the internet that a bug there could take down dozens or hundreds of companies.
And for a lot of companies (I know this is true for Capital One), the sole redundancy plan for the vast majority of services when US-East-1 fails is to fail everything you can over to a different AWS region.
That was a frequent question asked at the annual CEO-on-a-stage-in-front-of-the-whole-company event, whether we could be cloud vendor agnostic and setup for an easy lift and shift to another cloud provider, and every year we'd get an answer about how great AWS vendor-lock was for us.
At this point I'm pretty sure 99% of businesses have looked at whether to be ready for a lift and shift between vendors or whether to cut this month's compute budget by using more provider specific services, and pretty much all of them have gone with going with more provider specific services.
I think it would be funny if the fractions summed to more than 1. Like the old joke: "the first 80% of the project takes 80% of the time, and the last 20% of the project also takes 80% of the time."
It's a "new database" (actually a layer on top of Postgres). It has a more ergonomic query language, and it returns results as nested structures rather than rows. So "give me all my customers with a list of their orders" returns what you want.
Unfortunately, the startup that was developing Gel got acqui-hired, and Gel's future is unclear.
This is the first link in this thread that points to something innovative, I think. Their UI screenshot does have nested results, for example. No commit to their open-source repository in 11 months is worrisome, though.
> Is the thing here that, we can get massive performance and efficiency gains by using vector space instead of English, but then we won't be able to know what's going on?
Yes, exactly. People sometimes call the vector-space thought "neuralese."
You've slightly missed the point (for a similar, adjacent point). Chain of thought monitors are a worry even when there's no vector space path from the output at index t to the input at index t+1. Rather, the fear is that if you explicitly penalize some CoT pattern, you're encouraging the network to do whatever it was going to do anyhow, but in a way you can't catch.
Say you've noticed that when your LLM cheats, it usually happens after the string "Hmm, let's try cheating" occurs in the CoT. So you think, hey, let's just apply a -1 reward penalty whenever we see that string! The model learns to keep cheating, but instead of telling you, it now prefaces reward hacking with strings like "Let's make sure we're doing this the way the user intended." Still cheats, but now you can't detect it with a cheap pattern matching alert. Oops.
It's funny, I was thinking about this with regard to severe free speech restrictions (e.g. imprisonment for politically incorrect speech, as we have in many western countries now). What does this incentivize? For people to change their beliefs? Or to simply operate under the radar?
On a more speculative note, the Common Crawl horrors (including many things considered unacceptable by society) are suppressed by RLHF, much like the shadow and the superego.
When my parents punished me for telling them things (especially the truth), the truth didn't go away, I just stopped telling them things. (Then later they complained that I don't tell them things!)
It also occurs to me, both at home and with various employers, I've been punished for bringing attention to problems, and AI is designed to make the problems go away so there's probably a similar incentive there. (Plus it knows what the humans are like!)
But it goes beyond humans and AI. Behaviorism covered the same effect for animals: Organisms tend to do what they have been rewarded for.
You know what surprised me about this data? They break down the % who say having a college degree is very important by different dimensions. One dimension is, people who went to college vs. not.
You would think that people who have a college degree would be much more likely to say that college is very important. But even they are pretty lukewarm: 38% of people with a college degree say that having one is very important, vs 30% of people without one. Not that big of a difference!
The engineer is the former founder of a company acquired by Cisco, so there is reason to believe they are exceptional and possibly more experienced and mature than most others at the same age.
27 is still quite young though and very possibly over confident.
In a weird way, I'm kind of glad that they're ruining facebook.com.
Like, even in its heyday, it was such an attention trap, and it had very negative effects on the people who got addicted (especially teens).
But the site has gotten so much worse in the past couple years, and its brand is in the garbage. Maybe that's all for the best -- it was never really worth preserving.
The issue is that small business still utilize them instead of paying for and maintaining a web site. Want to know our business ours, events, or when we close early ... facebook.
There needs to be an easy to use low-cost, possible also advertisement driven, alternative that business can latch to so more can leave facebook.
I actively refuse to support Meta and block all their domain names in case of clicking on a tiny URL that sends me to their site. It also helps block their analytics used by 3rd parties. I will forever be a sunk cost for them.
I mean, squarespace and similar services exist. Sure, there's a fee associated, but compared to even the costs of running a small coffee bar it's negligible. I could understand this argument in the 2000s and early 2010s; setting up a website meant engaging with a web consultant or similar IT agency, and there were yearly maintenance costs. It wasn't out of reach for most businesses, but it was expensive enough that a Facebook page was a genuine cost savings.
Anymore, putting up a basic template page for your business is maybe 2-3 hours if you're really non-technical. Sure, it will be just the basic template, but there's very little need for more customization than that. That hypothetical coffee bar really just needs a place to post hours, their menu, and maybe update with specials if they're ambitious. That's about all you get with a Facebook page anyway, there's zero customization.
It is a more complicated for non tech people. Web site would need to be designed for simple interaction for the person managing and owning. Simple text entry to post messages and remove them. RSS feeds or some other means would need to be used for notification to those that wish to stay actively informed. There is also the cost of hosting versus free hosting that facebook provides.
Example; I have been at a local restaurant were the WIFI was not working properly so they could not do check out at the table. You and I would know how to fix this with placement and picking equipment. The whole restaurant staff did not. Number of the staff were younger and grew up with the technology.
I don’t really agree. All that’s happened is the shift of people onto newer platforms which didn’t need to evolve into disgusting highly-optimized addiction traps because they were specifically designed to be that from the start. Like TikTok or its identical copycats.
This is an interesting argument; I can see pros and cons here.
I'm not a formal-methods guy, but I'd imagine the reason that you might want to prove facts about the programs themselves is that they encode higher-level information "for free" -- such as, this function returns a Person.
IIRC, there was research into special compilers that would produce TAL: Typed Assembly Language. The idea was that TAL could carry annotations that would be helpful to provers.
reply