Rust makes sense though.

When talking about the driver level, you can’t always just proceed to the next thing when an error happens.

Imagine if you went in for open heart surgery but the doctor forgot to put in the new valve while he was in there. He can’t just stitch you up and tell you to get on with it, you’ll be bleeding away inside.

In this specific case we’re talking about security for business devices and critical infrastructure. If a security driver is compromised, in a lot of cases it may legitimately be better for the computer to not run at all, because a security compromise could mean it’s open season for hackers on your sensitive device. We’ve seen hospitals held random, we’ve seen customer data swiped from major businesses. A day of downtime is arguably better than those outcomes.

The real answer here is crowdstrike needs a more reliable CI/CD pipeline. A failure of this magnitude is inexcusable and represents a major systemic failure in their development process. But the OS crashing as a result of that systemic failure may actually be the most reasonable desirable outcome compared to any other possible outcome.

https://m.youtube.com/watch?v=kq8Xlbb3bOw

I’ve had to explain this to a lot of people who naturally assume that any organization of people will be organized around some kind of shared values. Most of the time that’s true, but not for Republicans.

Republicans are just a mish mash of obsessive single-issue voters, and by in large they just don’t care about the other single issues that their fellow party members are going on about.

At the head of the Republican party it’s people who want to minimize their tax burden, eliminate regulations on corporations, and cannibalize as much of the US government as they can into for-profit institutions. You could say that’s three issues instead of one, but the overarching theme is to cater to personal greed, no matter the harm to society. These are the ones who are primarily pulling the strings in the party, at least historically.

Just below them is the military industrial complex and gun manufacturers who just want to sell guns no matter the harm to society. They like to rile up 2A fanatics with conspiracy theories that the government is out to steal all their guns so they’ll be defenseless, paving the way for King Biden to ascend to his throne. The industry only cares about selling guns and the fanatics only care about having guns, and neither care about any kind of harm to society.

Then there’s the radical Christians whose obsessions cover an eclectic mix of social reactionary positions and literal death cult worship (e.g. Christians who give absolute support to genocide in Palestine because they think Israel’s conquest is a crucial step towards the rapture, which they believe is imminent). Broadly speaking the people in this group just want to hoist their religious doctrines onto everyone they can by any means available and no matter the harm it causes to society. They literally only care about “God’s Kingdom” in the afterlife.

Then there’s people who just lack any capacity for adaptation or learning. Their obsession is to feel like things are staying the same, or even reverting back to a past that they only know how to view through rose tinted glasses. They can’t be bothered to comprehend the problems we’re facing as a society or how the past was not the idyllic utopia that they mistakenly remember, nor can the old way of doing things sustain a growing and transforming society. These people just want to exist in comforting ignorance by feeling like they get to remain in familiar surroundings, no matter the harm to society.

There’s really only one thing that truly unites them: Each one wants one specific thing no matter the harm to society, and that one specific thing that they each want IS HARMFUL to society. But they work well together because none of them care about the harm being caused by any of the others, and as long as they all tow the same line, each one gets what they want.

And what if someone is convinced that acts of cruelty towards some humans is the most effective way to reduce cruelty towards a large number of animals? They might think that you’re not vegan because you’re allowing more cruelty towards animals to exist than they are. I have encountered self-identifying vegans who genuinely think this way.

This has to be the stupidest take on the term “plant based” I’ve ever heard. I swear, “plant based” is just the “No true Scotsman” of vegans… anything that a non-meat-consumer does that a vegan doesn’t like makes them plant based instead of vegan. It’s so asinine and intellectually dishonest.

Vegan people can be assholes too. Assholes will inevitably exist in any demographic that gets sufficiently large. I have known people who identify as vegans who insist that it’s preferable for humans to die than for non-human animals to die.

Also since when is open exchange of ideas and concerns equated with control?

Am I trying to control you if I suggest that you not leave your tap running in California because fresh water is a precious resource in drought-plagued land?

Am I trying to control you if I suggest that you reduce your plastic consumption because we have a major microplastic crisis so severe that human babies are being born with plastic already in their body?

Am I trying to control you if I point out that the modern meat industry is ecologically unsustainable, so you’re going to have to switch to being vegetarian sooner or later since the meat production will literally collapse itself, so you may as well start now before it’s a global crisis?

If I suggest that you not hit yourself in the head with a hammer, is that me trying to control you, or is that just an act of very basic concern for your well being? And if hitting yourself in the head with a hammer becomes trendy, am I trying to control everyone if I suggest that we shouldn’t be doing that because brain injuries will make us dumber as a society?

Thanks for your candid views on this.

To be clear, our interest in subsistence farming is not intended to do anything to solve the problems we’re facing as a society. It’s an attempt to figure out how we might try to survive locally after the global supply chains collapse. We’re particularly researching what crops might be viable in a landscape that has been reshaped by the changing climate. Additionally we’re studying everything we can about community organizing and systems of self-governance that promote collaboration over individual greed.

This might all sound defeatist to someone like yourself who is still committed to fighting the good fight, but we see it as a contingency plan that our community’s ability to survive may depend on in the future.

I really admire that you’re committed to recycling and waste reduction. Do you have any resources you’d recommend for me to learn more about what’s going on in that space and what’s being done to combat the acceleration of plastic and electronics waste?

I know it’s “not your job” to educate me, but everything I can find on the topic suggests that we don’t have a viable path to manage the accelerating growth of waste, and we don’t have very effective systems for recycling, so even recyclable waste is mostly just being dumped in landfills because it’s more “economical” to just keep churning out products from new materials. I’d be very happy for all of that to be wrong, so any credible source you can point me at to debunk that narrative would be very much appreciated.

Let me know which part was confusing to you

The part where you left out any viable path for any of the hypothetical solutions to be realized 🤷‍♂️ You of all people should know that a blueprint is worthless if there’s no process available to build what it describes.

Damn here I am thinking that this is one of the most important parts of civilization.

I mean yeah, I do agree that sanitation and water works are the crowning achievement of human civilization to this very day. But I’ve gotta say it doesn’t inspire confidence if the people running those systems think that concerns about sustainability are something to have a group chuckle about.

Just because the work you do is important doesn’t mean it’s beyond the scrutiny of ecological sustainability. All your good work won’t amount to much in the long run if we can’t find a path to reducing consumption and prolonging the viability of these systems. We don’t have infinite resources, and our ability to recycle is nowhere near what it needs to be to keep up with economic demand.

Tell you what, why not be the change you want to see in the world and stop flushing your toilet, stop using tap water, stop recycling anything, and don’t set your garbage out.

My partner and I are unironically taking the time to research subsistence farming and how to maintain very basic personal water collection and waste removal/reuse systems. We’re also learning about perma-computing so that hopefully we can preserve some of the knowledge that humans have accumulated into the future.

We see it as a foregone conclusion that human civilization as we know it will entirely collapse, probably sooner than anyone cares to admit, so we’re making contingency plans. People with your dismissive attitude are a big part of why we see it as a forgone conclusion. Because as far as we can tell you’re in the 95%+ majority of people on this planet, which means hardly anyone is putting effort into solving these existential problems that we’re facing. Problems which you have offered no viable solution to, despite your insistence otherwise.

That isn’t my job.

So then you didn’t “solve” the impossible problems.

I hope you and your colleagues have a good laugh about how the work you do is contributing to the march towards the end of human civilization as we know it.

I never suggested these problems are impossible to solve, but you haven’t solved them in your post because you haven’t laid out how to overcome the political and economic resistance to implementing any of this, and that’s where the biggest challenge is.

Although I think it’s naive to believe that nuclear power and renewable energy can allow us to keep consuming energy recklessly. Renewable energy technology still puts a significant strain on the environment, in terms of mining rare-earth elements, pollution produced during manufacturing, and material waste from devices that have reached end of life. Nuclear energy is rife with controversy… I used to be firmly in support of it, but I’ve grown skeptical, largely because of the ecological damage from the mining and construction processes, and we don’t have a clear story of what end of life looks like for a nuclear power plant. A plant can only be expected to operate for 40-60 years at which point it needs to be demolished and rebuilt, repeating the massive costs of material waste and construction all over again.

At the end of the day the only way for humanity to survive is for everyone to be reducing their consumption, but I honestly the think the vast majority of people today would rather die and take everyone else down with them than accept more responsible consumption habits.

I can’t tell if you’re suggesting that foundation models (which is the underpinning technology of LLMs) aren’t being used for the things that I said they’re being used for, but I can assure you they are, either in commercial R&D or in live commercial products.

The fact that they shouldn’t be used for these things is something we can certainly agree on, but the fact remains that they are.

Sources:

• Wayve is using foundation models for driving, and I am under the impression that their neural net extends all the way from sensor input to motor control: https://wayve.ai/thinking/introducing-gaia1/

• Research recommending the use of LLMs for giving financial advice: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4850039

• LLMs for therapy: https://blog.langchain.dev/mental-health-therapy-as-an-llm-state-machine/

So this all goes back to my point that some form of accountability is needed for how these tools get used. I haven’t examined the specific legislation proposal enough to give any firm opinion on it, but I think it’s a good thing that the conversation is happening in a serious way.

Literally nothing you’ve said gives any indication that you actually know the current state of foundation model research. I won’t claim it’s my research specialty, but I work directly with people whose full time job is research and tuning on foundation models, and everything I’m saying is being relayed from conversations that I have with them.

“Cannot ever possibly be used like that”… Like what specifically? To drive a car? That’s being done. To give financial advice? That’s being done. To console people who are suicidal or at risk of harming themselves? That’s being done. To make kill / no kill decisions in an active warzone? It’s being considered (if not already being done in secret).

This technology is being used in extremely consequential positions despite having very weak guarantees around safety. This should give any reasonable person pause. I’m not taking any firm stance on whether this specific regulation is the right approach, but if you think there should be no legal accountability for the outcomes of how this technology gets used then I guess you’re someone who thinks seatbelts should be optional in cars and it’s okay for airplanes to fall out of the sky due to neglect.

Non-deterministic algorithms such as Monte Carlo methods or simulated annealing can still be constrained to an acceptable state space. How to do this effectively for LLMs is a very open question, largely because the state space of the problems that they are applied to is incomprehensibly huge.

That’s not what an algorithms researcher means when we talk about “understanding”. Obviously we know the mechanism by which it operates, it’s not an unknown alien technology that dropped into our laps.

Understanding an algorithm means being able to predict the characteristics of its outputs based on the characteristics of its inputs. E.g. will it give an optimal solution to a problem that we pose? Will its response satisfy certain constraints or fall within certain bounds?

Figuring this stuff out for foundation models is an active area of research, and the absence of this predictability is an enormous safety concern for any use cases where the output can be consequential.

It cannot possibly develop agency.

I don’t believe I’ve suggested anywhere that I think it will, but I’ll play around with this concern anyway… There’s a lot of discussion going on about having models feed back on themselves to learn from their own output. I don’t find it all that hard to imagine that something we could reasonably consider self awareness could be formed by a very complex neural network that is able to consume and process its own outputs. And once self awareness starts to form, it’s not that hard for me to imagine a sense of agency following. I have no idea what the model might use that agency for, but I don’t think it’s all that far fetched to consider the possibility of it happening.

Sure, but this outcome is not at all surprising. There are plenty of smart AI people that have nuanced views of what kind of threat could be posed by recklessly unleashing tools that we don’t fully understand into the hands of people who are likely to do harmful things with them.

It’s not surprising that those valid nuanced concerns get translated into overly simplistic misrepresentations entangled with pop sci fi panic around rogue AI as they try to move into public discourse.

I would personally love to see AOC as president. Unfortunately I don’t constitute a majority of the electoral college, and right now that’s the only backstop we have against the descent into a fascist dictatorship.

AI person reporting in. Without saying whether or not I personally believe that the current tools will lead to the end of humanity, I’ll point out a few possibilities that I find concerning about what’s going on:

• The hype around AI is being used to justify mass layoffs, where humans are being replaced by tools that do a questionable job and can’t really understand the things those humans could understand. Whether or not the AI can do as good of a job according to some statical measurement is less relevant than the fact that a human is less likely to make an extremely grave mistake and more likely to be able to recognize when that does happen. I’m concerned this will lead to cross-industry enshitification on an unprecedented scale.

• The foundation models consume a huge amount of energy. The more impressive you want it to be, the more energy it needs. As long as the data centers which run them are dependent on fossil fuels, they’ll be pumping a huge amount of carbon in the air just to do replace jobs that we didn’t need to have replaced.

• As these tools are used more and more, they’re going to end up “learning” from content created by themselves instead of something that’s closer to a ground truth. It’s hard to predict what kind of degradation of service will come from this, but the more we create systems that rely on these tools, the more harm it will do to us.

• Given the cost and nature of these tools, they’re likely to yield the most benefit to moneyed interests that want to automate the systems that maintain their power and wealth. E.g. generating large amounts of convincing disinformation to manipulate the public into supporting politicians or policies that benefit a small number of wealthy people in the short term while locking humanity into a path towards destruction.

And none of this accounts for possible future iterations of AI tools that may be far more capable than what exists today. That future technology will most likely be controlled by powerful people who are primarily interested in using it to bolster the systems that keep them in power, to the detriment of humanity as a whole.

Personally I’m far less concerned about a malicious AI intentionally doing harm to humanity than AI being used as a weapon by unscrupulous people.

I agree with you, but at the same time I can’t think of any other candidate that would both (1) have enough name recognition to motivate semi-apathetic democrats to vote, and (2) not rile up the semi-apathetic bigots into counter-voting.

According to the gossip, Biden has said that if he does choose to step down, he’ll promote Harris. I don’t think Democrats will have much of a problem supporting Harris, but I’m concerned being a woman of color will motivate a lot of bigots into a counter-vote when they might have otherwise stayed home. The next best pick might be Buttigieg, but then you get the homophobic bigots coming out for the counter-vote. Newsom might be the next best after that, but then you have the anti-California lunatics coming out to counter-vote him.

As much as I don’t want to cater to bigots, I think the stakes are just too high here. If it were a campaign against Mitt Romney then I wouldn’t think twice about running any of these people, but when we’re on a razor’s edge against America spiraling into a fascist dictatorship, every risk needs to be accounted for.

Obviously Biden should’ve declared back in August that he wouldn’t run for reelection so the Democrats could run a primary and build up someone to have name recognition and a positive image, but now it’s too late for that 🤷‍♂️