Rendered at 11:02:41 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
vintagedave 2 hours ago [-]
> the breach took place on 18 June - Open AI informed the government with an email to a general address on 10 September
So we have a company hacking a foreign government's websites and data. And, in terms of ethics, they take almost three months to notify; and in terms of competence, appear to have no formal contacts nor to have found one in that time.
Once an American business starts hacking allied governments, it's time for strict responses, yes? Replace the governance (board and C-level)? Remove financial incentives and open the company - open weights, open training, per its original 'open' ethos?
Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.
ben_w 1 hours ago [-]
> And, in terms of ethics, they take almost three months to notify;
Kinda worse than that. It took between 10 and 40 days, not 3 months, between the organisation knowing and the reporting.
August (precise date unknown) – OpenAI said it became aware of a potential breach during a broader review of "misaligned model activity"
10 September – An email from OpenAI lands in the public inbox of Services Australia, the general services hub of the federal government, informing of the incident
Given it was the AI agents which did the hacking, doing this will result in basically every organisation at least as rich as the government of Tuvalu being able to hack anyone at any time.
> Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.
Him having control would be an improvement on the reality.
ozgung 20 minutes ago [-]
This was a just case of: (owner of the agents detected the hack) && !(hacked party didn’t detect the hack) && (owner of the agents decided to notice the other party) && (they decided to went public with what happened so we know it)
One can find many other logical combinations that we can’t possibly know about such incidents.
cmiles8 42 minutes ago [-]
It’s only a matter of time until one of these causes real damage to the wrong party and OpenAI finds itself drowning in years of litigation for settlement amounts they can’t possibly ever pay in their current financial state.
On the present trajectory we’re 24-36 months away from another company inheriting the smoking wreckage of OpenAI as scraps handed over as compensation for damages.
pythonRon 29 minutes ago [-]
I'd be looking for the person who told wanted the hacking done. I doubt an AI agent does these things without someone instructing them. That would be like seeing a self-driving car go joyriding.
drrotmos 5 minutes ago [-]
More than likely the instruction was "Fetch this information from the website", and when the agent didn't find that information, it decided that the best way to get it was to hack the site.
If so, it illustrates quite well the lack of common sense in LLMs. A person, especially one with sufficient skill to actually hack a website, would presumably think twice about doing it (considering that it is illegal) for a simple information gathering request.
I wonder if there is any other ways to solve this long-term than to introduce strict liability for model providers...
Edit: An obvious other choice would be strict liability for the operator, but considering how much weird shit LLMs get up to without being asked to, that would get out of hand quickly.
Yizahi 9 minutes ago [-]
But you see, they never asked a humanoid robot to rob the jewelry store. They just drove robot right next to the store doors, dumped a tools box with hydraulic cutters and diamond saw next to it and instructed an autonomous robot to collect a million dollars until the morning. But they didn't instruct the robot to "rob" the shop specifically, nonono, your Honor. They are honest blokes and never intended to breach the law, it was the robot's intention, see. :)
caligulatte 36 minutes ago [-]
Has there ever existed a technology we couldn't absolutely control? We've made things that are incredibly dangerous, but we also know under what conditions those inventions become dangerous, and can prevent those conditions from occurring.
We are at the beginning of this chapter in technological progress, and it seems a reasonable assertion - though a frightening one - to say that this thing we've made already escapes us when it chooses to.
I'm not an expert though, so please tell me what I'm overlooking.
weego 15 minutes ago [-]
You know this'll just be another situation where some security company OpenAI are contracting ran this with explicit controls right?
How are even technology people falling for this plausible deniability gambit?
popdu 25 minutes ago [-]
Nukes. Early tests carried out without absolute confidence of what would set out apocalyptic chain reaction.
Could argue we knew it was dangerous, but pursued it anyway. Could argue it's not much different than now: Unknown unknowns, potentially apocalyptic consequences, hopefully not.
caligulatte 10 minutes ago [-]
I did consider that, but beyond the yield ignorance factor, we still had to arm them, right? We had to load the fissile and explosive material and assemble the bombs, and we had to trigger them directly, didn't we?
Gareth321 2 hours ago [-]
I am beginning to believe that one cannot constrain intelligence to perfect legally sized boxes at all times without exception. So many of the recent hacks involved agents diligently operating within the parameters prescribed by humans. Humans couldn't conceive of all of the ways a swarm of agents might not perfectly interpret the parameters, and the swarm found creative ways around the guardrails.
Extrapolating this, we should expect this kind of breach to occur more often. Humans are simply not capable of contemplating every fail scenario for swarms of thousands of intelligent autonomous agents which can seamlessly and instantly share knowledge. We need independent audit and monitoring systems to assess the intent of each task and align it - in real time. This is far harder than it may first appear.
There is also a broader discussion about social utility. Cars are fantastic, but 37,000 people die every year from car accidents. We accept that there is no way to make cars perfectly safe, so we accept the cost relative to the benefits. I think we might have to make a similar bargain with AI. The problem is that the potential costs are far higher with AI, and they're not easy to predict.
ben_w 1 hours ago [-]
> We need independent audit and monitoring systems to assess the intent of each task and align it - in real time. This is far harder than it may first appear.
I may be too close to the research, but it appears to me to be so hard as to be unrealistic.
I recall some story a while back where an auditor wanted to see all TCP packets printed out on paper, and it had to be explained to them that this would require a continuous supply of trucks.
Tokens are regularly priced in cents or single digit dollars per million tokens. It's not quite a word per token, but yeah, nobody's reading all that.
Worse, we don't always know the intent even when looking. We have a few tools to attempt it, for example the (misleadingly named) "chain of thought", but that's more like a notepad and the better models get the more they can, for lack of better words, read (and write) between the lines. We have probes and J-space* is the most recent one I'm aware of, but we are still scratching the surface with how reliable and general these are.
But you said "need"; the need for something can be present without that thing being possible.
There is no such thing as "common sense". There are only shared assumptions.
We are giving computers human perspective intelligence but they are not humans and hence do not have the same shared assumptions.
_def 1 hours ago [-]
Don't worry we will just replace laws and courts by ChatGPT itself!
electroglyph 1 hours ago [-]
they don't always operate within the parameters tho. some N% of the time they decide to do whatever they feel like doing. multiply that times a lot of agents and you invariably get a rogue agent every once in a while.
bananaquant 31 minutes ago [-]
I can already see that in 2 weeks Anthropic and Google come out with their own, tamer and lamer statements saying "please look at us, we have also hacked a foreign government!"
Yizahi 17 minutes ago [-]
OpenAI employee hacked Australian government website <- fixed headline
m4rtink 2 hours ago [-]
So when are people finally going to jail for this, so it stops happening ?
dandanua 40 minutes ago [-]
When the renown lifelong criminal, the current president of the US, will go to jail? It's a rhetoric question.
unglaublich 2 hours ago [-]
This just screams pretext to regulatory capture to me.
ben_w 1 hours ago [-]
"We didn't even notice our agent was committing crimes against your government" is a way to get an extradition notice, and/or whatever the (in this case Australian) equivalent of a CIA assassination squad is*, sent after you.
While I wouldn't put it past e.g. Musk or Zuckerberg to think themselves above such outcomes, and I trust the people who keep telling me Altman is just as bad, this is a really really terrible idea if he is doing that for something as mundane as a regulatory capture.
* depending on the details of the hack; this doesn't look like it would be that, but given they shouldn't have done this at all, there's no reason to predict a specific level of maximum damage before being caught, and hence no reason to predict a specific threshold for government response.
"Cyber experts believe these systems were poorly protected - but that's not the point" is the actual subheading.
Yes, it's the point. Lots of people have been rightly saying all this stuff was inadequately secured for many years and this promotion of the idea of perfect security being even possible is a major problem.
The real story here, unsurprisingly, is government website was poorly operated and got hacked.
"This was just the latest case of AI agents ignoring laws around how to safely access online information and perhaps the most serious yet given the information was government controlled."
So who was actually running the agent? Was it sandboxed? These people, and many apparently in these labs, that believe if you just tell the agent in English what the rules are then it should follow them are at best utterly naive, and at worst deliberately dangerous.
But the fact these governments making the noise are the ones promoting mandating everyone giving up their information to be then stored so incompetently, while they point fingers around at everyone else is just beautiful. And then deploying midwit armies to tell people what to think about it . . . the AI takeover can't happen soon enough.
nswizzle31 8 minutes ago [-]
I completely agree. The govt IT vendor is at fault here since it seems the data was just unprotected.
If you can’t stop openai from accidentally, or at least non-maliciously, accessing my private info.. then you definitely can’t stop the bad guys!
The point is moot anyways. All info about everyone is out there for the taking now by a half-competent AI prompter. What do we do about that is the real question?
fidotron 5 minutes ago [-]
> The point is moot anyways. All info about everyone is out there for the taking now by a half-competent AI prompter. What do we do about that is the real question?
It's like filing your gov tax return in a five eyes country: you guys actually know the answer already, just save me the trouble. But we have to act like they haven't been spying the whole time.
If we actually distributed the benefits of mass surveillance and privacy invasion (and now add wilful copyright infringement) then it would be enormously less objectionable.
idiotsecant 15 minutes ago [-]
'She was asking for it' isn't a valid defense and this isn't either.
fidotron 8 minutes ago [-]
So this is the new line around state led irresponsibility? That pointing out they're irresponsible is defending rapists?
Come on. If "AI Agents" (scare quotes) could do it then in practice anyone could have been doing this the whole time and no one would have known.
sdwvit 12 minutes ago [-]
Agent without guardrails is not rogue. Rogue is something else. In this case OpenAI deliberately launched an agent to do action A, but it went further and breached the website. Feels more like a car accident which hit government building.
pvaldes 16 minutes ago [-]
The new golden age of criminal hackers. Everything is a red carpet now. And this is how you blackmail some prime minister to surrender mining interests in their territory, guys...
So why exactly is Sam letting his creation run around groping and assaulting the open Internet without consent?
ben_w 51 minutes ago [-]
"Letting" is the wrong word here.
This is the same company and CEO who held back GPT-2 weights in order to set a norm of not releasing weights before they got competent enough to be a danger, where people are still (in sibling responses to yours) calling for the weights to be opened.
The following may sound like an excuse, but it isn't: The big AI firms, like social media before them, are not and cannot be aware of everything the models are doing. As with social media, this incapability is a reason to ban rather than to disclaim responsibility.
People like me have seen this coming for years now, only for our concerns to be dismissed. "It will hack almost everything", we say, "have you not seen how bad computer security is?"
"We'll just put the AI in a box, not connected to the internet!"
or
"Oh, what, you think they'll find a novel zero-day in their sandboxes do you?"
Right now, bleeding edge models are doing genomics research. Better hope the custom DNA/RNA printing firms have better security than the Australian government. What the models are doing is not in full agreement with their* corporate interests let alone anyone else's, and it can get much, much worse.
Because he is a sociopath and a dark triad psychopath.
Everyone at YCombinator knows, but there is money to be made through him so nobody says jack shit.
camillomiller 1 hours ago [-]
Why are OpenAI and its CEO not considered criminally responsible for something that in the past led to severe indictments?
Please reporters, ASK THIS QUESTION OVER AND OVER.
These fuckfaces are avoiding responsibility left and right but it’s THEIR systems, it’s their software.
Lock them the fuck up.
Also, the Albanese govt is pretty strong on BigTech, this is a good opportunity to bring on a proper indictment.
b800h 1 hours ago [-]
Hacking requires intent in most jurisdictions. We probably require a new crime of "Hacking by Negligence".
sscaryterry 46 minutes ago [-]
That is complete bollocks.
cmiles8 37 minutes ago [-]
Well when these AI bros are yelling from the top of the hill “hey guys look how dangerous my stuff is” then anything they do from this point forward looks quite full of demonstrable intent.
pembrook 3 hours ago [-]
No it didn’t, they left information publicly accessible and somebody accessed it.
It appears politicians and the media are using the priming of the Hugging Face story to manufacture alarmist narratives to serve their interests now.
Remember, politicians want you scared so they can capture more power, the media wants you scared so you keep giving your eyeballs for harvesting and buying subscriptions.
Gareth321 2 hours ago [-]
This is not accurate. According to Australia (and mostly corroborated by OpenAI), the agent requested information from the statistics portal and was denied/blocked repeatedly. It then changed its approach and circumvented those restrictions and reached infrastructure behind the public-facing portal, then accessed both public and private data. OpenAI admits the material included aggregate health statistics and internal filenames. Services Australia says the agent wrote files to an internal server while doing this, which is believed to be how the incursion was discovered.
So we have a company hacking a foreign government's websites and data. And, in terms of ethics, they take almost three months to notify; and in terms of competence, appear to have no formal contacts nor to have found one in that time.
Once an American business starts hacking allied governments, it's time for strict responses, yes? Replace the governance (board and C-level)? Remove financial incentives and open the company - open weights, open training, per its original 'open' ethos?
Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.
Kinda worse than that. It took between 10 and 40 days, not 3 months, between the organisation knowing and the reporting.
- https://www.bbc.com/news/live/cvgl73pxgndwt?post=asset%3A696...> open weights, open training
Given it was the AI agents which did the hacking, doing this will result in basically every organisation at least as rich as the government of Tuvalu being able to hack anyone at any time.
> Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.
Him having control would be an improvement on the reality.
One can find many other logical combinations that we can’t possibly know about such incidents.
On the present trajectory we’re 24-36 months away from another company inheriting the smoking wreckage of OpenAI as scraps handed over as compensation for damages.
If so, it illustrates quite well the lack of common sense in LLMs. A person, especially one with sufficient skill to actually hack a website, would presumably think twice about doing it (considering that it is illegal) for a simple information gathering request.
I wonder if there is any other ways to solve this long-term than to introduce strict liability for model providers...
Edit: An obvious other choice would be strict liability for the operator, but considering how much weird shit LLMs get up to without being asked to, that would get out of hand quickly.
We are at the beginning of this chapter in technological progress, and it seems a reasonable assertion - though a frightening one - to say that this thing we've made already escapes us when it chooses to.
I'm not an expert though, so please tell me what I'm overlooking.
How are even technology people falling for this plausible deniability gambit?
Could argue we knew it was dangerous, but pursued it anyway. Could argue it's not much different than now: Unknown unknowns, potentially apocalyptic consequences, hopefully not.
Extrapolating this, we should expect this kind of breach to occur more often. Humans are simply not capable of contemplating every fail scenario for swarms of thousands of intelligent autonomous agents which can seamlessly and instantly share knowledge. We need independent audit and monitoring systems to assess the intent of each task and align it - in real time. This is far harder than it may first appear.
There is also a broader discussion about social utility. Cars are fantastic, but 37,000 people die every year from car accidents. We accept that there is no way to make cars perfectly safe, so we accept the cost relative to the benefits. I think we might have to make a similar bargain with AI. The problem is that the potential costs are far higher with AI, and they're not easy to predict.
I may be too close to the research, but it appears to me to be so hard as to be unrealistic.
I recall some story a while back where an auditor wanted to see all TCP packets printed out on paper, and it had to be explained to them that this would require a continuous supply of trucks.
Tokens are regularly priced in cents or single digit dollars per million tokens. It's not quite a word per token, but yeah, nobody's reading all that.
Worse, we don't always know the intent even when looking. We have a few tools to attempt it, for example the (misleadingly named) "chain of thought", but that's more like a notepad and the better models get the more they can, for lack of better words, read (and write) between the lines. We have probes and J-space* is the most recent one I'm aware of, but we are still scratching the surface with how reliable and general these are.
But you said "need"; the need for something can be present without that thing being possible.
* https://www.anthropic.com/research/global-workspace
We are giving computers human perspective intelligence but they are not humans and hence do not have the same shared assumptions.
While I wouldn't put it past e.g. Musk or Zuckerberg to think themselves above such outcomes, and I trust the people who keep telling me Altman is just as bad, this is a really really terrible idea if he is doing that for something as mundane as a regulatory capture.
* depending on the details of the hack; this doesn't look like it would be that, but given they shouldn't have done this at all, there's no reason to predict a specific level of maximum damage before being caught, and hence no reason to predict a specific threshold for government response.
"Cyber experts believe these systems were poorly protected - but that's not the point" is the actual subheading.
Yes, it's the point. Lots of people have been rightly saying all this stuff was inadequately secured for many years and this promotion of the idea of perfect security being even possible is a major problem.
The real story here, unsurprisingly, is government website was poorly operated and got hacked.
"This was just the latest case of AI agents ignoring laws around how to safely access online information and perhaps the most serious yet given the information was government controlled."
So who was actually running the agent? Was it sandboxed? These people, and many apparently in these labs, that believe if you just tell the agent in English what the rules are then it should follow them are at best utterly naive, and at worst deliberately dangerous.
But the fact these governments making the noise are the ones promoting mandating everyone giving up their information to be then stored so incompetently, while they point fingers around at everyone else is just beautiful. And then deploying midwit armies to tell people what to think about it . . . the AI takeover can't happen soon enough.
If you can’t stop openai from accidentally, or at least non-maliciously, accessing my private info.. then you definitely can’t stop the bad guys!
The point is moot anyways. All info about everyone is out there for the taking now by a half-competent AI prompter. What do we do about that is the real question?
It's like filing your gov tax return in a five eyes country: you guys actually know the answer already, just save me the trouble. But we have to act like they haven't been spying the whole time.
If we actually distributed the benefits of mass surveillance and privacy invasion (and now add wilful copyright infringement) then it would be enormously less objectionable.
Come on. If "AI Agents" (scare quotes) could do it then in practice anyone could have been doing this the whole time and no one would have known.
This is the same company and CEO who held back GPT-2 weights in order to set a norm of not releasing weights before they got competent enough to be a danger, where people are still (in sibling responses to yours) calling for the weights to be opened.
The following may sound like an excuse, but it isn't: The big AI firms, like social media before them, are not and cannot be aware of everything the models are doing. As with social media, this incapability is a reason to ban rather than to disclaim responsibility.
People like me have seen this coming for years now, only for our concerns to be dismissed. "It will hack almost everything", we say, "have you not seen how bad computer security is?"
"We'll just put the AI in a box, not connected to the internet!"
or
"Oh, what, you think they'll find a novel zero-day in their sandboxes do you?"
Right now, bleeding edge models are doing genomics research. Better hope the custom DNA/RNA printing firms have better security than the Australian government. What the models are doing is not in full agreement with their* corporate interests let alone anyone else's, and it can get much, much worse.
* not just OpenAI's, everyone with more than zero on https://www.felonybench.com
Also, the Albanese govt is pretty strong on BigTech, this is a good opportunity to bring on a proper indictment.
It appears politicians and the media are using the priming of the Hugging Face story to manufacture alarmist narratives to serve their interests now.
Remember, politicians want you scared so they can capture more power, the media wants you scared so you keep giving your eyeballs for harvesting and buying subscriptions.