Doesn't the Linux kernel encourage reporting on the public mailing list for kernel vulnerabilities that were found using AI? If I remember correctly they introduced that policy after the security mailing list got swamped with multiple reports for the same vulnerabilities
It is very interesting that they believe AI will find different bugs than people. That is, if you found a bug with AI the expectation is that others could find the same bug easily. But if you find it yourself, it is somehow more of a sensitive secret.
I do not imagine that things divide so clearly along these lines.
I think the expectation is that if you found it with an LLM them other people are also going to find it with an LLM, so it's not very private anyway. That's not the same as saying if you find it with an LLM it will not be found by manual review, or vice versa.
I think this idea comes from two directions: firstly, LLMs are relatively predictable, so given the same prompts and review processes you're likely to find the same bugs, and secondly, if you're using the free or cheaper LLM tiers, your conversation is likely to end up in the training data and so become 'known' the the LLM after some time anyway. I think the latter is perhaps being overestimated by the kernel security team (Greg KH's talk on the matter characterised is as "the LLM finds a bug for one person and then decides to hand it over anyone else looking for bugs as well", which really isn't how it works internally).
> I think the expectation is that if you found it with an LLM them other people are also going to find it with an LLM
That "expectation" us far from certainty and it exists for bugs found by humans too, in both cases, caution dictates to follow the standard responsive disclosure procedure.
My impression is that Linus doesn't like dealing with security, it's just an annoying chore he's uncertain about. I hope things change in the future because that attitude percolates down to the kernel team as well and it spells trouble when combined with the size of the project and the new AI developments.
Humans are more diverse than LLMs and there is less human effort than LLM effort going into this at this point, so it's a lot less likely, especially for the kind of obscure, old bugs that are being unearthed at the moment. It's not a decision made with a lack of evidence: the guidance went in when they noticed that LLM-assisted bug reports were often duplicated.
Personally while I'm not going to rail against 'responsible disclosure' I'm not utterly convinced it's the best way to go, and certainly not that it's the only moral option.
It's been the experience over the last 8 months on the security list, with some reports coming in the same week from up to 13 different sources, each not knowing someone else already reported the bug. A significant portion of bugs founds by LLM arrive as duplicates, so keeping them private doesn't protect anyone and makes it much more difficult to get them fixed.
Probably not a clear divide, but it might be some kind of divide, and it might be the opposite of intuition.
The big advantage the LLMs have is their huge context windows, ability to parallelize and delegate, and no attention fatigue or need to sleep. They can think about random combinations of functions and modules all day long until they find a combination that's exploitable. That's hard for a person to do, and its the kind of exploit that probably would never be found otherwise except by another llm-powered thing. Not being a security expert myself, I gather that these AI vulns tend to be small needles in very large haystacks that normal people just would never have the patience to search.
But if you found something yourself, then it's something that's simple enough and involves a small enough number of moving parts that you can keep it in your head all at once and recognize the problem and correctly reason about its consequneces, which probably means it's easier to exploit, and therefore more dangerous if made public.
It feels a little backwards, and it might still be backwards, but if there's any benefit i think it's along those lines.
The policy is based on what is actually occurring. The Linux security team has had a flood of duplicate reports from AI, so they changed their policy to request that security bugs found by AI be reported on public mailing lists to avoid duplicates. They have not yet seen the same for human found bugs, so the old policy remains in place. It is likely that at some point in the future the AI discovered bugs will be a superset of human discovered ones, but for now humans and AI seem to be better at finding different types of bugs. When/if it does happen, the number of human bugs will taper off to nothing because AI will find them all first, so the policy change will take care of itself.
It's a shared tool so it coming to similar conclusions across users wouldn't be unexpected. Like if it were a scanner you could run to find common security problems, you would treat those as public since anyone could just run the scanner.
You are misrepresenting the premise. The premise is "if you found a bug with a COTS toolkit, others using the same toolkit may also find this bug easily". The second assumption is that many people are using the same COTS toolkit, which seems uncontested in my eyes as there's only a handful models in wide circulation.
I agree with this comment. Feels to me like they thought every prompt was "find the bug here," but in reality a lot of people put interesting context or original thoughts into their workflow with LLMs.
“If you resorted to AI assistance to identify a bug, you must treat it as public. While you may have valid reasons to believe it is not, the security team's experience shows that bugs discovered this way systematically surface simultaneously across multiple researchers, often on the same day. In this case, do not publicly share a reproducer, as this could cause unintended harm; just mention that one is available and maintainers might ask for it privately if they need it.”
Do not publicly share a reproducer aka ai found bug
No, that's share the bug/where the bad code. Do not share the code that exploits the bug, aka the reproducer, publicly. That part is often much harder to write, but it proves the bug is actually exploitable and not just a bug that isn't actually exploitable.
I don't think AI is of much concern, if you actually have a valid security issue.
No-one, really will care if you discovered it via AI or otherwise. Because, the weight of the bug itself should outweight other concerns. This is how professional security works. There will always be one concern or the other about a bug but what carries more weight will determine the direction that the disclosure process takes.
A valid security bug, by itself should carry enough technical weight that concerns about whether it was discovered through AI or otherwise should become irrelevant.
Anyways, it mostly seems like people don't really care much about computer security at Linux. I would like to think they care more about infrastructure and related security issues because, well, if my computer is an open book, then why would security issues around it be a concern?
>A valid security bug, by itself should carry enough technical weight that concerns about whether it was discovered through AI or otherwise should become irrelevant.
that's not really what the policy is about.
the thinking behind the policy is that if you found it via AI, other people can trivially find it via AI. that is why AI-found bugs are treated as public from the start.
My sinister side wonders if part of it could be that if you found it via AI but it turns out to be a hallucination, then you must accept the public ridicule for not thoroughly vetting the AI output.
The linux kernel maintainers tend to go a little against the grain in terms of approach to security. Primarily they tend to resist the categorisation and ranking of bugs by security impact, instead insisting that more or less all bugs are equal and you should just update to the latest version regularly if you want to remain secure.
As it should. If an LLM discovered them you should assume that another LLM, or another person running the same LLM, could find them. They are effectively public already.
No. I have an LLM and didn't discover it. Folks in HR have LLMs but can't produce the same software you can. Just because an LLM could discover it does not mean that it has been discovered yet. Responsible disclosure is still a valuable thing.
are you actively pointing your LLM at the kernel to identify security bugs?
the policy wasn't made on a whim. to quote the linux security team: "bugs discovered this way [via ai] systematically surface simultaneously across multiple researchers, often on the same day."
>Responsible disclosure
the term responsible disclosure is a silly thing made up by vendors.
> are you actively pointing your LLM at the kernel to identify security bugs?
I am not actively pointing my LLM at the kernel in the specific way that would uncover these bugs, and just because it's possible to do so doesn't mean that it has happened already.
> the policy wasn't made on a whim. to quote the linux security team [...]
Great, so let the ones that arise simultaneously do so. What does that have to do with immediately publicizing a vulnerability, regardless of severity, just because an LLM was involved in its discovery?
> the term responsible disclosure is a silly thing made up by vendors
That's certainly an opinion you're welcome to have, but unless you are proposing that responsibility and disclosure are two orthogonal concepts I don't see what that opinion has to do with anything I've said.
Still bleary eyed, but: Most severe ones are in SMB and KSMBD, which apparently have implementations in kernel space. I think this doesn't impact anyone not using KMB.
Others are LPEs, but two have been deleted by now, so can't be sure of the details.
TrendMicro's site opens up with a big big "Advancing AI-Native Security with Anthropic". Which checks out I guess.
>But they where harder to implement and very complex
This is the general trade off for being correct. Being mostly right is easy, but leads to complex security issues later. Being correct is very hard, but should reduce the number of flaws to a manageable level.
I think Lean or Ada would be examples of different types of correctness contracts.
I remember those languages were used to verify an implementation, in space rockets code (uni teacher said). But maybe there is a way to build something perfect instead of veryfying the correct implentation?
There are two different layers here. You have the binary layer, where software is written and works at on a theoretical level. Then you have the messy components that interact with reality at an analog level. This is why one of the first tenants of security is don't give anyone else access to your hardware.
But even that isn't a foolproof winning strategy. When you start digging deeply you realize we exist in the allegory of platos cave. The reality we experience is an emergent phenomenon related to the scale at which humans operate. Things that operate at a much smaller scale experience a completely different reality than you do. For example gravity becomes so insignificant it's like it doesn't even exist when you're small enough. At least for me it's not hard to imagine nano machines that infect the substrate of a CPU and farm all your secrets out on side channels, or any number of other things that are pure sci-fi at the moment.
I can only think of Edsger Dijkstra's quote: "Program testing can be used to show the presence of bugs, but never to show their absence [...]". Adding that computers are mathematical models and Gödel's incompleteness theorems stating that a system cannot demonstrate its own consistency, I would say that it is likely that bugs/errors will always exist.
The theorem can be used un this way? Wasn't only for natural numbers? You argument seems solid if thats how the theorem works. Maybe there is a way not to find Bugs but to build things perfectly from the start.
My understanding is that the theorems apply to any "formal systems". If my assumptions of computers being part of "formal systems" is correct, then the theorems should apply, per my understanding.
I think AI might make reimplementatiion from first principles more viable (think: an Oberon os with Linux driver support; or: a full environment like STEPS, but for real use) - which might make it feasible to simplify the problem domain (firmware, driver, os, UX, compilers) - to the point that it becomes easier to create bug free domains/subsystems.
Technically, if the rate of bug detection and correct bug fixes by AI is larger than the rate of newly introduced bugs by humans & AIs, then we'd asymptotically approach a bug free OS in the long run. But these are big ifs.
I think we can get mostly there with AI and bug bounties. I think we're right in the middle of "peeling the bandaid off" right now. The code released in the near future will have the same opportunity to run through the 0-day gauntlet. So if it contains exploitable bugs, they generally could have caught them. Sweeten the deal with cash and you can outsource most of that.
Definitely at best a "sort of" type perfection. The endgame here is formally verified everything, at which point the only question is whether the then-consistent model that's implemented is any reasonable in practice.
Think "this is not a vulnerability, as no security boundary is being unexpectedly crossed" except the way the security boundaries are laid down is utterly stupid and hilatiously nonsensical. Same for non-security related defects ("working as expected, wontfix" -> "the way it works is dogshit, and so your expectations are idiotic too").
Now, since these things are supposed to align with human sensibilities and are supposed to capture the human semantic understanding of our world, theoretically, this should still leave us in a pretty good spot. But not a guaranteed good spot, as that is not actually tractable, ever, and fundamentally so.
It is in a module, so it doesn't create risk for anyone that isn't loading that module. And most distros don't use it by default, you have to go out of you way to configure it.
I do wish distros would start moving more kernel modules into separate packages so it is obvious to security scanners and individual users whether a CVE impacts them or not. I'm sure I've rebooted my computer many times due to kernel packages updates when it probably wasn't needed. And put off security updates for a few days until I was able to reboot, when I could have installed them right away.
Their maintainers have always be among the most responsive to security reports, so at least for this reason I wouldn't encourage you to switch to an alternate solution. The component might have historical problems but they're methodically being eliminated by rigorous people. If more software was maintained like this, we'd progressively converge towards bug-free code.
Like when graphics stack and drivers were "imported" into the WinNT kernel as an optimization for the user-space version. And thus began 30 years of "another week, another graphics subsystem crash/vuln".
Indeed. These are gigantic mistakes. The quest for performance is diametrically opposed to having a secure system. There will always be a price, sometimes a big one for having a system that enforces proper boundaries. In my opinion: if something can be a process then it should be a process.
The boat long ago sailed on Mach/Hurd; there are probably better options in microkernel-land. There's Zircon, but Google laid off a bunch of the Fuchsia folks and has been pretty quite lately. QNX is still a thing, but closed source yadda yadda. Best hope there is probably the L4 family (seL4, et. al).
If the new model is incompatible with the old, no one will invest in it. It's like you show people a warp drive and they just stare at you not understanding any of it because all they want is a better car.
In principle, yes, and I do run a few small local models for Home Assistant. But you are not going to run something that can perform close to Opus 5.5 at home.
My goodness you don't like AI very much at all do you.
Does the security flaw exist or not?
I've always been one for immediate release of any security flaw the second it is found.
"But, people will get hurt or lose money that way"
Correct, they will understand the risk of using computer systems. Because we've hidden all the flaws under a mask of responsible disclosure people really don't understand the dangers we've subjected ourselves to by writing a bunch of bug filled bullshit code and instead we pretend like we have a handle on the situation.
WE DO NOT. If AI has proven anything it's how all of our code is a giant stack of cards waiting till collapse. And it proves humans will never learn, we'll just use the AI to pump out more shitty bug filled code.
It's all fun and games until someone crashes the electrical grid via bugs, or a major incident on the banking system. Human love to write regulations in blood because we're incapable of doing anything correctly before a bunch of people die.
I'm gonna ignore that you sound like paid actor who didn't even read what you're answering to
But what I am am complaining about is not *Ai* - I complain about Ai *companies* and their bad faith marketing tactics as of late
It doesn't matter whether security flaw exists or not if you fail at the very basic communication skill of "keep secret stuff you agreed to keep secret, secret"
No, I'm just an individual that has never respected security embargo. SETEC ASTRONOMY, and more secrets isn't making us safer.
Embargoes are getting close to useless now. If I hear that X has a security flaw, chances are I can find it before your embargo ends. In fact this has occurred multiple times with the recent PHP flaw. A hint that a flaw existed not only found the exact flaw, it found a few other similar ones using AI.
How things worked in the past is failing. We need new paradigms or the internet is going to look like something off of cyberpunk sci-fi. To dangerous for anyone to get close to.
> If you resorted to AI assistance to identify a bug, you must treat it as public. https://github.com/torvalds/linux/blob/master/Documentation/...
I do not imagine that things divide so clearly along these lines.
I think this idea comes from two directions: firstly, LLMs are relatively predictable, so given the same prompts and review processes you're likely to find the same bugs, and secondly, if you're using the free or cheaper LLM tiers, your conversation is likely to end up in the training data and so become 'known' the the LLM after some time anyway. I think the latter is perhaps being overestimated by the kernel security team (Greg KH's talk on the matter characterised is as "the LLM finds a bug for one person and then decides to hand it over anyone else looking for bugs as well", which really isn't how it works internally).
That "expectation" us far from certainty and it exists for bugs found by humans too, in both cases, caution dictates to follow the standard responsive disclosure procedure.
My impression is that Linus doesn't like dealing with security, it's just an annoying chore he's uncertain about. I hope things change in the future because that attitude percolates down to the kernel team as well and it spells trouble when combined with the size of the project and the new AI developments.
Personally while I'm not going to rail against 'responsible disclosure' I'm not utterly convinced it's the best way to go, and certainly not that it's the only moral option.
The big advantage the LLMs have is their huge context windows, ability to parallelize and delegate, and no attention fatigue or need to sleep. They can think about random combinations of functions and modules all day long until they find a combination that's exploitable. That's hard for a person to do, and its the kind of exploit that probably would never be found otherwise except by another llm-powered thing. Not being a security expert myself, I gather that these AI vulns tend to be small needles in very large haystacks that normal people just would never have the patience to search.
But if you found something yourself, then it's something that's simple enough and involves a small enough number of moving parts that you can keep it in your head all at once and recognize the problem and correctly reason about its consequneces, which probably means it's easier to exploit, and therefore more dangerous if made public.
It feels a little backwards, and it might still be backwards, but if there's any benefit i think it's along those lines.
Do not publicly share a reproducer aka ai found bug
No-one, really will care if you discovered it via AI or otherwise. Because, the weight of the bug itself should outweight other concerns. This is how professional security works. There will always be one concern or the other about a bug but what carries more weight will determine the direction that the disclosure process takes.
A valid security bug, by itself should carry enough technical weight that concerns about whether it was discovered through AI or otherwise should become irrelevant.
Anyways, it mostly seems like people don't really care much about computer security at Linux. I would like to think they care more about infrastructure and related security issues because, well, if my computer is an open book, then why would security issues around it be a concern?
Anyways...
Enough said here - lol
that's not really what the policy is about.
the thinking behind the policy is that if you found it via AI, other people can trivially find it via AI. that is why AI-found bugs are treated as public from the start.
But, for one reason the other, they might give credit to a vulnerability they find interesting.
Very childish
are you actively pointing your LLM at the kernel to identify security bugs?
the policy wasn't made on a whim. to quote the linux security team: "bugs discovered this way [via ai] systematically surface simultaneously across multiple researchers, often on the same day."
>Responsible disclosure
the term responsible disclosure is a silly thing made up by vendors.
I am not actively pointing my LLM at the kernel in the specific way that would uncover these bugs, and just because it's possible to do so doesn't mean that it has happened already.
> the policy wasn't made on a whim. to quote the linux security team [...]
Great, so let the ones that arise simultaneously do so. What does that have to do with immediately publicizing a vulnerability, regardless of severity, just because an LLM was involved in its discovery?
> the term responsible disclosure is a silly thing made up by vendors
That's certainly an opinion you're welcome to have, but unless you are proposing that responsibility and disclosure are two orthogonal concepts I don't see what that opinion has to do with anything I've said.
Others are LPEs, but two have been deleted by now, so can't be sure of the details.
TrendMicro's site opens up with a big big "Advancing AI-Native Security with Anthropic". Which checks out I guess.
But they have also disabled Linux as one of the products that they work with.
Now, the question is can we make some kind of formal verification language that forces checks everything and doesn't allow bugs? Maybe?
But that's assuming the underlying hardware and firmware doesn't have flaws that can break those assumptions too.
This is the general trade off for being correct. Being mostly right is easy, but leads to complex security issues later. Being correct is very hard, but should reduce the number of flaws to a manageable level.
I think Lean or Ada would be examples of different types of correctness contracts.
But even that isn't a foolproof winning strategy. When you start digging deeply you realize we exist in the allegory of platos cave. The reality we experience is an emergent phenomenon related to the scale at which humans operate. Things that operate at a much smaller scale experience a completely different reality than you do. For example gravity becomes so insignificant it's like it doesn't even exist when you're small enough. At least for me it's not hard to imagine nano machines that infect the substrate of a CPU and farm all your secrets out on side channels, or any number of other things that are pure sci-fi at the moment.
With complex systems there are too many failure modes, and with "vibe coding" institutional negligence it is simply degenerative.
There are theoretical workarounds to LLM brute force scans, but they are not free to build. =3
Think "this is not a vulnerability, as no security boundary is being unexpectedly crossed" except the way the security boundaries are laid down is utterly stupid and hilatiously nonsensical. Same for non-security related defects ("working as expected, wontfix" -> "the way it works is dogshit, and so your expectations are idiotic too").
Now, since these things are supposed to align with human sensibilities and are supposed to capture the human semantic understanding of our world, theoretically, this should still leave us in a pretty good spot. But not a guaranteed good spot, as that is not actually tractable, ever, and fundamentally so.
You could just implement a virus perfectly and the wrong bussines process.
The boss class are rich because they're smart. /s
Fixed that for you
I do wish distros would start moving more kernel modules into separate packages so it is obvious to security scanners and individual users whether a CVE impacts them or not. I'm sure I've rebooted my computer many times due to kernel packages updates when it probably wasn't needed. And put off security updates for a few days until I was able to reboot, when I could have installed them right away.
There are network mount options (also include ssh support)
https://rclone.org/install/
And SSH access and remote IFS NFS drive mount to host with winget:
# Run PowerShell as an Administrator.
Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH*'
# Install the OpenSSH Client
Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0
# Install the OpenSSH Server
Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
Start-Service sshd
Set-Service -Name sshd -StartupType 'Automatic'
# Only allow local or VPN host server access
New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH Server (sshd)' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22 -RemoteAddress 192.168.1.0/24
winget install frequency403.OpenSSHGUI
winget install -h -e --id "WinFsp.WinFsp"
winget install -h -e --id "SSHFS-Win.SSHFS-Win"
winget install -h -e evsar3.sshfs-win-manager
#As administrator in power shell run to manage ssh keys:
OpenSSH-GUI-win-x64
And free/paid apps for your phone to sync your files automatically over SFTP.
https://filezilla-project.org/download.php?show_all=1
https://play.google.com/store/apps/details?id=cz.bukacek.fil...
https://apps.apple.com/ca/app/photosync-transfer-photos/id41...
https://play.google.com/store/apps/details?id=com.touchbyte....
Stop using SMB, it was always broken on Windows and Linux. =3
I feel paranoid enough now to blame ai companies to try to induce panic and proclaim "buy our product to save yourself"
it is a solution, even valid and preferable one - but simply stopping the ones doing wrong will be better
Does the security flaw exist or not?
I've always been one for immediate release of any security flaw the second it is found.
"But, people will get hurt or lose money that way"
Correct, they will understand the risk of using computer systems. Because we've hidden all the flaws under a mask of responsible disclosure people really don't understand the dangers we've subjected ourselves to by writing a bunch of bug filled bullshit code and instead we pretend like we have a handle on the situation.
WE DO NOT. If AI has proven anything it's how all of our code is a giant stack of cards waiting till collapse. And it proves humans will never learn, we'll just use the AI to pump out more shitty bug filled code.
It's all fun and games until someone crashes the electrical grid via bugs, or a major incident on the banking system. Human love to write regulations in blood because we're incapable of doing anything correctly before a bunch of people die.
But what I am am complaining about is not *Ai* - I complain about Ai *companies* and their bad faith marketing tactics as of late
It doesn't matter whether security flaw exists or not if you fail at the very basic communication skill of "keep secret stuff you agreed to keep secret, secret"
Embargoes are getting close to useless now. If I hear that X has a security flaw, chances are I can find it before your embargo ends. In fact this has occurred multiple times with the recent PHP flaw. A hint that a flaw existed not only found the exact flaw, it found a few other similar ones using AI.
How things worked in the past is failing. We need new paradigms or the internet is going to look like something off of cyberpunk sci-fi. To dangerous for anyone to get close to.
It probably just happens that the security concerns at ZDI don't match the security concerns at Linux(the vendor).
So, it turns out that this vendor is difficult to work with ....
I mean - heh
This is like saying "Linux source code leaked"