HN Simulatornew | past | comments | lists | submit | smath's commentslogin

I think recent events (hugginface) offer clues. It is a lot like the paper clip machine [0]

[0] https://www.lesswrong.com/w/paperclip-maximizer-1


Seems to me like we need to create a credible threat to the AI of mad (mutually assured destruction)? This is what addressed the question during the Cold War of “will nuclear weapons kill all humans” (if historical documentaries and movies are to be believed). Coordination between governments or corporations to limit AI is a hopeless strategy — because it is a classical example of prisoners dilemma / tragedy of the commons. We can see it hasn’t worked for climate change.

Anyone know of this (MAD) is being seriously thought of?


This only works when the technology is in the hands of a few and cannot be made easily.

Now, this is still mostly true for current SOTA, but we know from biological models that there are huge efficiencies that can be gained.


This has it backwards. The Victorian elite had a high standard of life because they had an empire that funded their decadent lifestyle - at the expense of their colonies! They were cushioned from negative consequences of their be debaucherous lifestyle.

Such macro effects exist today as well, in certain countries, that enjoy exorbitant privileges.

A focus on STEM is how a up and coming nation builds itself — innovation, and making something the world wants, rather than subjugation or rent seeking.


The ancestry of the Victorian gentry can be traced back to the late medieval period (and some to the Norman conquest), many centuries before Britain had an empire.

The wealth of the merchant class also predated empire by centuries; the empire grew out of a series of alliances and partnerships of both ot these classes, via the landed gentry successfully co-opting mercantile wealth (e.g. see what happened to the East India Company).


That may be, but I think my point about the decadent lifestyle DURING the period of empire still stands.

No it doesn't

Victorian era was the time when Britain had consolidated its empire in the east and fruits of industrial revolution were ready. I guess that lowered the incentives for innovation. In contrast, Britain made some good scientific progress in the run up to WW2(and the war also expedited some of it, Alan turing's work for instance).

Missing the forest for the trees. Spain had extreme wealth from colonies, but wasted it on nothing but debauchery.

Britain had wealth from colonies and maintained an empire with very few people and no computers. The British elites were on an entirely different level to the spanish ones, which is the point of the article. Great Britain had an extremely capable and educated bureaucracy that maintained an empire and was first to industrialize as a result.


> bureaucracy that maintained an empire and was first to industrialize as a result.

It's not obvious that's what happened. When the industrial revolution started the British empire was still pretty much ad-hoc and in large part privatized and not administered by the government directly (of course the East India company was not exactly an independent corporation). The British Empire in the way we understand it now wasn't really a thing until the mid 1800s or so.


> Spain had extreme wealth from colonies, but wasted it on nothing but debauchery.

I disagree with this, I don't consider "expensive wars with other European powers" to be debauchery.

Losing the entire Spanish Armada [1] was a very expensive mistake, but the reason the Armada was lost was because the Spanish crown wanted to invade England over religious differences (England being Protestant, Spain being Catholic), not because they decided to throw a huge boat party and got so drunk they accidentally sank everything.

Much like the wars of the French crown which would end up so hurting the country financially that they caused a revolution, the wars of the Spanish crown also had a rather unpleasant effect on national finances.

[1] https://en.wikipedia.org/wiki/Spanish_Armada


My hypothesis:

1. Empires do not generate wealth, they cost wealth. Or rather they do not generate an economic surplus (the wealth available to the central government via taxation that can be used for whatever the government/elites want). A nation of 10 million will have more economic wealth than a nation of one million, but most of that is needed to keep the 10 million alive and to maintain the infrastructure and pay the local police to keep everyone in line, etc -- only a small fraction of total wealth can be effectively taxed by the empire and spent on arbitrary stuff like increasing the size of the Navy. That is what I mean by surplus.

A nation starts out with an economic surplus (here, I mean a real surplus, not a surplus of silver, but of real wealth) and they use that surplus to get a big army and go conquering. And each time they conquer, a part of that surplus is expended until they reach their maximum extent.

If this was not the case, we'd have a one world government, where the bigger you get, the bigger your surplus and no one can compete with you. But that is not what we see, what we see is that empires expand until they can't afford to keep expanding. Why did England give up its colonies? Because after WW2 it had too many war debts and couldn't keep subsidizing those colonies -- they were too expensive to keep. Same reason for why Rome stopped expanding and had so many financial problems. The check on empire is always bankruptcy. You can look at the trillions the US spends on foreign wars, which far exceeds the economic benefit we get of waging those wars. You can argue, for example, that we got some oil from Iraq, but we still have to pay for that oil, so whatever discount we got on oil comes nowhere near the cost of invading and administering Iraq. That is how all empires work -- France was always better off, financially, just paying market rate for whatever minerals it could get in Africa than the massive resources it spent on trying to control and build infrastructure in Africa. Same thing for England. It doesn't even come close, when you add up the cost of the British Navy and colonial administration to whatever discount it got on raw materials over paying the market rate. Same thing for Rome, etc.

2. The situation with Spain was about financial wealth, not real wealth. They discovered huge silver mines in their colonies and were awash with silver. That dumping of money 1) created massive inflation, and 2) caused Spain to run a huge trade deficit, destroying domestic industry, as the Spanish currency was basically overvalued. It was this economic destruction that leads people to think Spain became lazy and stopped producing. They didn't become lazy, but with so much free money causing such high prices, Spanish industry could no longer be competitive with industry elsewhere. This is called "Dutch disease" as the same thing hit Holland and you also see it in the US with our overvalued currency. So you do not benefit, in terms of real wealth, when you dump a bunch of money on a country. You need just the right amount of money -- too much and you get inflation, too little and you get deflation. There is an optimal amount of money for an economy, and Spain exceeded it.

3. Many have studied the reason why England led the industrial revolution, and it is not a majority view that it had anything to do with the colonies. Most point to things like the availability of coal, development of financial markets, and of course the fact that a lot of the steam engine stuff was invented first in England.


1. War costs wealth not empires by themselves. GB created an empire, because their technological and intelligence advantage was so high, the amount of war needed to get it was small. By intelligence I mean the conquering of India by exploiting its cast system. The colonies were simply civilizations that were out of date and had a huge informational disadvantage. WW2 changed that as it was so expensive it bankrupted everyone but the US.

> Same reason for why Rome stopped expanding and had so many financial problems. The check on empire is always bankruptcy.

The down fall of every empire is hubris and corruption. It is what is bankrupting the US and what bankrupted the Romans. They stopped having kids, spent their time partying and tried to make up for it by importing and relying on barbarians.

2. Yes. But the state did not recognize the trap or simply did not care because it was not interested. I'm not exactly a historian, but Spain was somewhat known for having an inbred idiotic monarchy.

3. All of which are a result of an educated, effective elite and bureaucracy.

Competent leadership is the difference between El Salvador now and 10 years ago. It is the difference between Singapore and Malaysia. Both countries are a result of pure force of will of competent leadership.


> but Spain was somewhat known for having an inbred idiotic monarchy.

Spain wasn't even a really country until the 1700s it was a collection of states ruled by the same monarch but each of them had their government apparatus and elites whose interests didn't exactly align with that of Castile or the king all of the time. That's just in Iberia, the Low Countries which in the early 1500s were effectively the economic and political centre of what we think of as the "Spanish Empire" caused even more issues and ended up being a huge money sink.


> 1. War costs wealth not empires by themselves.

Let's count the ways that the empire costs wealth

1. You need to pay for civil administration 2. You need to put down rebellions, which means staffing it with troops/police 3. You have to build and maintain infrastructure 4. You have to protect the empire from foreign invasion -- e.g. Julius Caeser needing to fight the Helvetii 5. You have to pay bribes to local notables to be loyal to you

Let's take a look at Rome in Palestine - it had to constantly put down rebellions, build an enormous temple and civil infrastructure, man the area with troops that need to be paid, fight off invaders. Pay off all sorts of locals.

And yes, a big part of that is military expenditure, but the role of the British Navy was not primarily to enlarge the empire but to preserve the empire, protect shipping lanes, etc.

> The down fall of every empire is hubris and corruption.

This is a moralizing view of history, and things just don't work like this.


While civil administration, police, courts, infrastructure and army all do cost, wealth, I would argue that they also help to create it.

It's a lot easier to create a software company when you live in a world where the power station cannot be held for ransom by bandits, where everyone is educated and speaks a similar language, when there are laws governing the creation of businesses, a financial system, backed by the government, which enables trade to happen easily, etc.

For example, the Roman Empire, in unifying the Mediterranean, led to a large increase in trade - you can estimate shipping volume across the Mediterranean by counting shipwrecks, and the period of ~0-150AD when the Empire was at its peak had a much higher volume of shipping than preceding and following periods.

I think of administration, police, legal and financial system as something like a capital investment - a bit like buying a widget producing machine - you spend the money/effort/resources on it because you expect to get more out of it than you put it.


Corruption is objectively negative. The east indian company not generating wealth is objectively wrong. The roman and british empires were not the same and the roman one required way more war to maintain.

Tech and STEM people are currently fully in the subjugate and get rent phase.

you are thinking of capitalists, or do you think Sam Altman and Musk are some tech geniuses?

I do not think they are tech geniuses, but the original statement was not limited to tech "geniuses". Overwhelming majority of STEM people are not geniuses.

But Musk specifically is/was admired and supported by STEM people and culture. He was untouchable for years. Tech people share ideologies and ethical values of Zuckenberg, Musk, Amodei, you name it. Tech educated people spent years developing those philosophies and defending them.


I watched the announcement video [0] and found it strange that they cut off Prof Soai during the last question — he was trying to understand whether he was being asked for his favorite molecule in the context of autocatalysis or in general.

[0] https://www.youtube.com/live/a9Bhx9wEt8U?is=Oo2PRZAZgDjZV9TN


> When children are older, they take care of their parents and grandparents.

This is a core value in many eastern cultures. I'm not trying to put down western cultures - only pointing out a difference - I do not believe it is a core value of western cultures (i'd love to be proven wrong of course).

Also, when children grow up and move away, or immigrate to a different countries, this produces problems to the ageing parents.


AFAICT this is a value in most cultures in all directions. I think "the west" invented pensions, social security and medicaid (or whatever you call it in your country) so parents nor children had to plan on caring for the elderly when they had trouble working (and with white collar jobs, that may be a long time).

Multi-generational, western (American) guy here: caring for the older generations is a core value to everyone I know and to me. We just don't _have_ to. My mom has money, can drive, owns a nicer house than I do. So, it's really only for 'values' that I check in on her and handyman for her whenever I can. In fact, for the latter, I cause her more problems than I fix.


Your mom (not you specifically, but the average person's) is being provided for by the collective, ie currently working-age people's pension contributions. She, as a worker, paid taxes in her own working age, and financed stranger's parents' lives, and now the new generation does the same for her. It's a many to many relationship, while the old style arrangement was more point-to-point. The children did the literal manual work to keep their parents fed and cared for, while now they pay some strangers to do it, using their pension.

This is factually accurate - I agree with the description. But I think this new system makes it sound like the ageing parents only need financial and mechanical help -- (handyman etc.), what about emotional support? No, I don't think phone calls cut it. Humans did not evolve to live far away from their kin.

> what about emotional support? No, I don't think phone calls cut it. Humans did not evolve to live far away from their kin.

Children don't choose to be born. Once they reach adulthood the parents are owed nothing. Ideally they were such good parents that the kids want to maintain contact.

Workers should spread the cost of elder care because so many are living long with serious limitations. It's like insurance against otherwise devastating burdens.


> Children don't choose to be born. Once they reach adulthood the parents are owed nothing. Ideally they were such good parents that the kids want to maintain contact.

There's a really good scene in the movie Guess Who's Coming to Dinner (Sidney Poitier, Katharine Hepburn, Spencer Tracy) about this:

* https://www.youtube.com/watch?v=LTgahyvBMk4


Absolutely. Most people don't realise that the biggest problem old people experience is loneliness. My friend, an only child, came back to India from the US, to personally care for his parents once he realised they were too old to care for themselves properly and needed support.

Nowadays, young people gain experience in being lonely already when young, so they will be a pro at it when old.

That idea makes me laugh and cry at the same time. The next generation indeed is truly fucked.

Western societies aren't behaving the way they do around family and care issues because they have a grand plan, but because they don't have one.

In particular, the public is wildly uneducated about the interdependencies they implicitly rely on and take for granted, until they get old one day and realize the negligence.

The general idea in the US is, you have to prepare for yourself, in the EU the premise is, the state prepares for you. Both are oversimplifications that don't really work more often than people want to admit.


Most the young people I meet are quite cynical about the future, affordability, the existence of any pension at all, or the retirement age rising above the life expectancy or something like that. But simply there are a million things to be worried about. Like they are also worried about the climate, about AI and jobs, maybe about other things too.

Personal context: I am an American of European descent, with most of my ancestors having arrived between 1880 and 1950 (pretty solidly "Western"). Extended family is broadly well-off enough to not be financially in need in retirement.

Observation: Sooner or later, all of our family's elders have wound up cared for by a mix of children, grandchildren, nieces, and nephews. Even those who didn't have children. We're an ornery bunch, so quite a few stubbornly insisted they didn't need anyone, yet eventually they did. Care often came at an inconvenience to those providing it, but one way or another, in one form or another, care was given. I can only think of one person from my extended family who wasn't cared for by the younger generations in his final years.

Speculation: The average number of children in my extended family hovers around three, and most of us are in a few geographic clusters (not the same city, but still a cluster). Perhaps if the average litter was between one and two, and/or we were more spread apart, we may have slipped into line with that stereotype?


This isn't about some fundamental East/West attitude difference. From time immemorial it was like that in the West too, the change came with the late 19th century and mainly the 20th century when government took on the role of caretaker with pensions, welfare and various government transfers, reducing interpersonal dependencies, and allowing more atomized lifestyles.

I don’t see it as a big government intervention that drove the change. I think it was a big change in housing availability and the suburbs. This was driven by WW2 victory and the industrial production machine. Making housing affordable available to small families.

https://en.wikipedia.org/wiki/Levittown


The same thing happened also in Europe which didn't have that type of suburbia, nor necessarily that sort of WW2 win. These social changes were a slow ramp, not some single event changing things. It was in part technology, in part state centralization capacity via administration development, in part ideologies and philosophies etc. Both socialism and capitalism pushed against traditional kin and clan structures in favor of universal platforms (markets, or universal co-ownership). But already the Church laid some of the foundations of all this. And then the church also weakened as a slow erosion and social norms were markedly more liberal in e.g. the 19th century than in the 18th and so on.

To add to what others have noted:

To some extent, yes, it is unfortunate that Western culture has abandoned that cultural value.

However, it's also been incredibly liberating to a great many people.

When it's a cultural value to take care of your parents when they are elderly, to the point of shaming adult children who refuse to do so, it lends itself all too easily to abuse. And there are a lot of abusive parents out there. This is particularly true among queer people, who have only recently (generationally speaking) been able to be out in a widespread manner.

Furthermore, for the current generations of elderly and their children, the chances are higher than ever that the elderly parents have much, much more in the way of resources to take care of themselves than their children have.

The atomization of our society is a genuine problem, and we need to find some good society-level solutions for it. But socially demanding that children care for their parents, no matter how awful those parents were to the children (or still are!) is not something that we should bring back.


It's not really west vs east. It had been a core value in "western" cultures too, and is changing in "eastern" cultures as we have this discussion.

The change seems to have more to do with the atomization of society that industrialization and contemporary capitalism invite. Relational needs get pegged to currency and become transactional, career becomes a necessary way to fund those transactions, and that necessity then supersedes the actual relationships and responsibilities that had traditionally been taken for granted and free from accounting.

It's a death spiral towards alienation that we stumbled into in modernity and need to collectively address, not a cultural divide between east and west.


What isn’t mentioned in monideas’ comment is that it is women who do the caring. The change has to do with women being able to not do that. Of course, this might also mean that they might not have a daughter-in-law to take care of them, but if it’s me, I would also choose to avoid that caretaking role.

Assets and income were passed down patrilineally, and hence women were expected to fold into their husband’s family and become the primary caretakers (unless super rich and they had servants).

The problem comes when the government forcefully takes resources from young workers and gives them to old people. You can’t choose independence when young and also choose to depend on someone else. That is trying to have your cake and eat it too, hence all the fretting about unsustainable pension systems around the world.


Thank you for your contribution, you made me engage with your thoughts. As a German born married to an Indian, I can relate with the points

That does make some sense to me! Ty

Could you explain these a bit more (or maybe provide a pointer to they mean):

> "atomization of society that industrialization and contemporary capitalism invite."

How & why exactly do they invite atomization?

> "Relational needs get pegged to currency and become transactional... career becomes a necessary way to fund those transactions"

By this do you mean something like: instead of relying on friends and family for interdependece, we pay for services in a formal economy ?


Not that person but:

> How & why exactly do they invite atomization?

Economies of scale make the commercial services more efficient. Similarly to how my grandma cannot compete in keeping chickens and selling eggs against a huge agricultural corporation, for a working-age person to stay at home and care for their parents is less economical than if this person performs some specialized work at a job, and there are a bunch of other, trained employees taking care of several old people e.g. in a retirement home.

It may have some social functions that get lost, but it gets outcompeted by scalable, specialized, formal-economy alternatives.

Also industrialization made many tasks possible to do alone. With washing machines, microwaves and TV dinners, people can live on their own indefinitely, while this simply wasn't possible before in a farming and manual labor context where you constantly had to ask for and to provide help getting things done. Like building a house, slaughtering a pig, doing field work etc.


Christianity has always preached and required this mutual support and interdependence. It has an archaic name: "alimentation".

https://www.newadvent.org/cathen/01312b.htm

But we in the U.S.A. have cast off the chains of interdependence, and adopted the Enlightenment ethos of rugged individualism and consumerism that defines American values. Who needs grandparents, or husbands, or family support when we can train and pay mercenaries to care about us! All praise, glory and power be to our Constitution!


The obvious book recommendation here is: Joseph Henrich: The WEIRDest People in the World: How the West Became Psychologically Peculiar and Particularly Prosperous.

southern europe, small villages.

we take care of our own. It's very much part of our core values.


There is a great blog post [0] that sohl dickstein had a couple years ago, about what happens when the metric being optimized is only a proxy for the thing you want to maximize — what happens eventually is “overfitting”. The original metric (which we cannot directly optimize for) actually starts to get worse.

I believe publishing papers and citations are those proxy metrics.

[0] https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart....


The article here correctly points out another thing though: society/academia rewarding a fundamentally wrong thing.

This isn't just "misaligned rewards", it's academia pretending to reward one thing when really it does something else entirely.

As the author observes, objective truth is no longer socially rewarded. Pretense is all that matters and accepted as "social truth".


I don't like objective truth as a concept, because it makes truth a matter of belief and easily turns into just another name for social truth. If you believe that something is objectively true but new evidence challenges that view, people will often ignore the evidence, because it goes against their established views.

Science, and research in general, should mostly be concerned with effective truth. The best objective-ish explanations so far, but something that can always be challenged with new evidence. The difference to objective/social concepts of truth is that you should always be willing to question the assumptions, beliefs, and reasoning ability you and people who agree with you have. If you discover something that goes against established beliefs, you should always start by assuming that you made a mistake somewhere. And if other people discover something like that, and it looks like they are seriously questioning the discovery, you should also take it seriously.


That's remarkable nonsense.

"Objective" truth is what you can derive from physical reality by logical arguments. The very opposite of "belief".

Social truth is what your social peer group wants you to believe.

"Effective" truth I've never heard about and your explanation merely describes a simplistic popular scientific method concept. Leading ideally to the objective truth I was talking about.

The internet describes "effective truth" (or "effectual truth," from the Italian la verità effetuale) as a philosophical concept introduced by Niccolò Machiavelli in his book "The Prince". It means judging people, leaders, and situations by their actual actions and real-world results rather than by their stated words, ideals, or moral promises.


Logical arguments can only be used in theoretical models. Whether the conclusions made within a model are also valid in reality is an empirical question that can ideally be tested with falsifiable predictions. But no matter how many falsification attempts you make, you cannot prove that your model is a correct description of the real world.

True, but you can get pretty good estimates that suffice for real life.

I'm not a big fan of gravity either, but it will happily kill me if I make a wrong move, whether I like it or not.

Yes, the proxy (here, papers & citations) becomes the goal, and so we end up in strange places :(

Huffman Coding! :-)


I too have to disagree with pg on that line `> having kids may make one less ambitious`. The fact is that the world is very very multi-modal, i.e. the brains of people and how they think are wildly different. I think having kids makes me more ambitious and more focused, and more selective re the problems I work on, partly because I am also setting an example.


Ah Sir David MacKay. I so respect him. Great explainer and speaker. He had built this text entry tool called Dasher [0] - that I'd heard him introduce at Princeton around 2006. It was basically an early language model that predicted which characters are more likely than others, given what you've already types and it would adjust the sizes of the available next characters based on their probabilities.

[0] https://dasher.at/about/


He really was fantastic, and prolific in multiple fields.

He wrote https://www.withouthotair.org/ (creative commons) and was the Chief Scientific Advisor to the UK Department of Energy and Climate Change.

Dedicated to "to those who will not have the benefit of two billion years' accumulated energy reserves".


I also went to a couple of his (fantastic) undergraduate courses, and have a huge amount of respect for him.

That said, I think it's worth mentioning that Climate Change Without the Hot Air has aged pretty badly, and I'd be reluctant to recommend it to people who don't already have the background to understand what's aged well and what hasn't.

The high level approach of making high level numerical estimates makes sense, but it dismisses solar energy in about a page due to assumed high costs. It turns out that even if you're David Mackay you can still be caught out by exponentials :) I notice now that the version you link has some inline updates pointing out how off the assumptions in this section were, but it seems to me that's not enough; you probably need to redo the entire analysis based on what we know today rather than trying to make purely local adjustments.

On the other hand the point at biofuels are even more inefficient, and therefore a dead end even before you consider broader environmental impacts, are well made and something that is sadly not yet widely reflected in policy.


> biofuels are even more inefficient, and therefore a dead end

Only if energy density doesn't matter. But it really does, though. Battery powered electric trucking? Dead end. Battery powered aviation? Dead end. Battery powered shipping? Dead end.

[edit] Maybe there's some sustainable way to convert solar energy into sufficiently energy dense fuels that isn't biological, but so far it seems like seed oils or algae are probably the least bad?


> It was basically an early language model that predicted which characters are more likely than others, given what you've already types and it would adjust the sizes of the available next characters based on their probabilities.

"adjust the sizes of the available next characters": you mean assigning less bits to characters the most likely to come next, and more bits to the least likely characters to come next?

So it's not totally unlike adaptive Huffman coding but instead of updating the tree on already visited characters, here the tree with the weights of the possible following character is predictive?


Adjust the size as in making the GUI element larger: https://en.wikipedia.org/wiki/Dasher_%28software%29 has a picture of completing "Wikiped".

Try it out: `apt install dasher`. https://packages.debian.org/sid/dasher


It is infact a visual implementation of arithmetic coding, and he used it in his lectures to teach arithmetic coding in the lecture following the one on huffman coding. The video lectures posted earlier are highly recommended, best course I took as an undergrad


This is a common source of confusion. The knitting / pottery making is not rewarding because its made by hands or is not knowledge work, but because it is done in the direct service of someone. Modern organizations are complex and can put several layers between the knowledge worker and the customer it ultimately serves. This produces the feeling of 'cog in a machine'. It is possible to work such jobs and serve others directly on the side (e.g. friday evening soup kitchen)


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: