I imagine they're building this system with the goal of extracting user demographics (age, sex, income) from chat conversations to improve advertising monetization.
This seems to be a side project of their goal and a good way to calibrate the future ad system predictions.
For context though, people have been screaming lately at OpenAI and other AI companies about not doing enough to protect the children. Almost like there is no winning, and one should just make everything 18+ to actually make people happy.
And also to reduce account sharing. How will a family share an account when they simultaneously make the adult account more “adult”, and make the kids account more annoying for adults.
safe to say everything in existence is created to extract user demographics.
it was never about you finding information, this trillion market exists to find you.
it was never about you summarizing information getting around advertisement, it is about profiling you.
this trillion market is not about empowering users to style their pages more quickly, heh
Random reply: 20 days ago you asked for my ChatGPT custom instructions to be more skeptical. It is :
Use an encouraging tone. Adopt a skeptical, questioning approach. Call me on things which don't seem right. List possible assumptions I'm making if any.
You give too much credit to Apple in the AI age. Especially since they've already partnered with Google to power Siri with Gemini now.
Apple is a has-been. Anthropic is best positioned to take up the privacy mantle, and may even be forced to, given that most of their revenue is enterprise and B2B.
Exactly. The more data they can collect, the better.
Is it not in OpenAI's best interested for them to accidentally flag adults as teens so they have to "verify" their age by handing over their biometric data (Persona face scan) or their government ID? Certainly that level of granularity will enhance the product they offer for their advertisers.
> We’re learning from the initial rollout and continuing to improve the accuracy of age prediction over time
> While this is an important milestone, our work to support teen safety is ongoing.
agreed, I guess we'll be seeing some pushbacks similar to Apple's CSAM but overall it's about getting a better demographics on their consumers for better advertising especially when you have a one-click actions combined with it. We'll be seeing handful of middleware plugins (like Honey) popping up, which I think the intended usecase for something like chat based apps
Considering that OpenAI is having trouble getting its models to avoid recommending suicide (something it probably does not want for ANY user), I rather doubt this age prediction is going to be that helpful for curbing the tool's behavior.
>behavioral and account-level signals, including how long an account has existed, typical times of day when someone is active, usage patterns over time, and a user’s stated age.
Surely they're using "history of the user-inputted chat" as a signal and just choosing not to highlight that? Because that would make it so much easier to predict age.
Hard no. It's so easy to get "flagged" by opaque systems for "Age verification" processes or account lockouts that require giving far too much PII to a company like this for my liking.
> Users who are incorrectly placed in the under-18 experience will always have a fast, simple way to confirm their age and restore their full access with a selfie through Persona, a secure identity-verification service.
Yea, my linkedin account which was 15 years old and was a paid pro user for several years got flagged for verification (no reason ever given, I rarely used it for anything other than interacting with recruiters) with this same company as their backend provider. They wouldn't accept a (super invasive feeling) full facial scan + a REAL ID, they also wanted a passport. So I opted out of the platform. There was no one to contact - it wasn't "fast" or "easy" at all. This kind of behavior feels like a data grab for more nefarious actors and data brokers further downstream of these kinds of services.
The unfortunate reality is this isn't just corporations acting against user's interests, governments around the world are pushing for these surveillance systems as well. It's all about centralizing power and control.
Yeah, this is all far far too invasive. The goal is obviously to gather as much data on you as possible under whatever pretense users are most likely to accept. "Think of the children", as always. This will then be used to sell advertising to you, or outright sell it to data brokers.
Do you expect the data collected for age verification will be completely separate from the advertising apparatus? I would expect the incentives would align for this to enhance their advertising options.
I'll almost certainly get used for both, but I believe "adult content" is the primary motivator. If it was just for ads they wouldn't even bother announcing it as a "feature", they'd just do it.
Also:
"Users can check if safeguards have been added to their account and start this process at any time by going to Settings > Account."
Does regulators really care about a predicted age? I feel like they require hard proof of being above age to show explicit content. The only ones that care about predicted age is advertisers.
In the UK, age verification just has to be "robust" (not further specified). Face scanning, for example, is explicitly designated as an allowed heuristic.
It's not much for the regulators as much as its for the advertisers.
At this point, just use gemini (yes its google and has its issues if you need SOTA) or I have recently been trying out more and more chat.z.ai for simple text issues (like hey can you fix this docker issue etc.) and I feel like chat.z.ai is pretty good plus open source models (honestly chat.z.ai feels pretty SOTA to me)
Kagi's Assistant is the most useful tool I've found as far as searching goes, and occasionally simple codegen. Let's you use a wide variety of models and isn't tracking me.
If we are talking about complete privacy. I am trying out https://confer.to (created by signal team) too and I am unable to run it on my mac (passkey support)(I tried it both in zen & orion) but I tried it on an android chrome (tablet) and I am kinda more optimistic about it too
I have heard good things about Kagi in fact, that's the reason why I tried orion and still have it in the first place but I haven't bought Kagi, I just used the free searches orion gives & I don't know if it has Kagi's assistant.
I think proton's Lumo is another good bet.
If you want something to not track you, I once asked cerebras team on their discord if they track the queries and responses from their website try now feature and they said that they don't. I don't really see a reason why they might lie about it given that its only meant for very basic purposes & they don't train models or anything.
You also get one of the fastest inferences for models including GLM 4.7 (which z.ai uses)
You might not get search results though but for search related queries duck.ai's pretty good and you can always mix and match.
But Cerebras recently got a 10 billion $ investment from OpenAI and I Have been critical of them from now on so do be wary now.
Kagi Assistant does seem to be good if someone already uses Kagi or has a subscription of it from what I feel like.
Any proof that you are above a certain age will also expose you identity.
That is the only reason regulators care about children safety online, because they care about ID.
LLMs are very good at profiling users in hacker news and finding alt accounts, for example. profiling is the best use case for llms.
So there you go, maybe it wont give exactly what regulators say they want, but it will give exactly what they truly want.
Protecting the kids and fighting terror. Anything that can't be argued against is always used as a justification of people in power who don't want to incite a riot.
Politicians, CEOs, Lawyers it's standard practice because it's so effective.
Hey Al, they might be implying that non-minors would be impervious to the viral challenges based on some sort of well-developed critical thinking facilities. I am not so optimistic.
A lot of people are using AI as a trusted friend, because they don't really have anyone else. A good friend would, I hope, talk someone out of doing something dangerous just to get a silly viral video. With AI being trained on the internet, that's going to have a very different take on things, as the internet only cares about the spectacle, not the person performing it.
I've been very aggressive toward OpenAI on here about parental controls and youth protection, and I have to say the recent work is definitely more than I expected out of them.
Interesting. Do you believe OpenAI has earned user trust and will be good stewards of the enhanced data (biometric, demographic, etc) they are collecting?
To me, this feels nefarious with the recent push into advertising. Not only are people dating these chat bots, but they are more trusting of these AI systems than people in their own life. Now, OpenAI is using this "relationship" to influence user's buying behavior.
This is a thoughtful response and deserves discussion. Yes, certainly, OpenAI might get your age wrong. Yes, certainly, they’re signaling to advertisers.
But consider OPs point — ChatGPT has become a safety-critical system. It is a tool capable of pushing a human towards terrible actions, and there are documented cases of it doing this.
In that context, what is the responsibility of OpenAI to keep their product away from the most vulnerable, and the most easily influenced? More than zero, I believe.
It's really really not. "Safety-critical system" has a meaning, and a chat bot doesn't qualify. Treating the whole world as if it needs to be wrapped in bubble-wrap is extremely unhealthy and it generally just used as an excuse for creeping authoritarianism.
I'm an engineer working on safety-critical systems and have to live with that responsibility every day.
When I read the chat logs of the first teenager who committed suicide with the help and encouragement of ChatGPT I was immediately thinking about ways that could be avoided that make sense in the product. I want companies like OpenAI to have the same reaction and try things. I'm just glad they are.
I'm also fully aware this is unpopular on HN and will get downvoted by people who disagree. Too many software devs without direct experience in safety-critical work (what would you do if you truly are responsible?), too few parents, too many who are just selfishly worried their AI might get "censored".
There are really good arguments against this stuff (e.g. the surveillance effects of identity checks, the efficacy of age verification, etc.) and plenty of nuance to implementations, but the whole censorship angle is lame.
I am against companies doing age verification like this due to the surveillance effects, but I agree with you that the censorship angle is not a good one.
I suppose mainly because I don't think a non-minor committing suicide with ChatGPT's help and encouragement matters less than a minor doing so. I honestly thing the problem is the user interface for GPT being a chat. I think it has a psychological effect that you can talk to ChatGPT the same way you can talk to Emily from school. I don't think this is a solvable problem if OpenAI wants this to be their main product (and obviously they do).
Maybe we don’t all need saving from ourselves. Maybe we need to grow up and have some personal responsibility. As someone who is happy to do that, seeing personal freedom endlessly slashed in the name of safety is tiresome.
My feelings have absolutely nothing to do with censorship. That’s just an easy straw man for you to try and dismiss my point of view, because you’re scared of not feeling safe.
Cool, I'd like you to make a commercial system you sell access to and ensure that it is unsafe. I'll represent the injured and we'll own all your corporate assets, and like will pierce the corporate veil due to your wonton behavior.
What are you on about? Laws are a codification of social norms. I’m not suggesting anything outside of existing social norms. Quite the opposite, I’m suggesting we stop changing them.
I was addressing your own (supposedly) safety-critical work, how you’ve used that to justify other work in the name of safety more broadly, and how you’ve placed yourself on a pedestal with your experience, to convince yourself that comments by others on the necessity of such safety are less qualified.
It's nonsense and doesn't work. They have "age predicted" my account a couple of months back saying I'm under 18, while I'm a man in my 40s who uses ChatGPT for mostly work related stuff, and nothing that would indicate that it's someone under 18. So now they are asking for a government ID to prove it. Yeah, no thanks.
It’s absolutely crucial for effective ad monetization to know the users age - significant avenues are closed down due to various legislation like COPPA and similar around the world. It severely limits which users can even be subject to ads, the kind of ads, and whether data can be collected for profiling and targeting for ads.
It feels like OpenAI is moving into the extraction phase far too soon. They are making their product less appealing to end users with ads and aggressive user-data gathering (which is what this really is). Usually you have to be very secure in your position as a market segment owner before you start with the anti-consumer moves, but they are rapidly losing market share, and they have essentially no moat. Is the goal just to speed-run an IPO before they lose their position?
When we look at how fast and coordinated the rollout of age verification has been around the globe, it's hard not to wonder if there was some impetus behind it.
Agreed - Interesting that these systems inevitably involve proving you're a citizen in some way, which seems unnecessary if your goal is to try to figure out someone's age.
Wait, I don't understand this. Does it mean that they can erroneously predict I'm a minor, covertly restrict my account without me knowing? I guess it's time to cancel my subscription.
Yes, it listed a few of my past questions as reference points. I had asked some questions about Nintendo 64/Dreamcast and Gamecube games. It also used information I had asked it about programming languages and some work-related questions to guess my age.
I don't know how OpenAI plans to do this going forward, just quickly read the article and figured that might be a good question to ask ChatGPT.
Edit: I just followed that up with, "Based on everything I've asked, what gender am I?" It refused to answer, stating it wouldn't assume my gender and treats me as gender neutral.
So I guess it's ok for an AI agent to assume your age, but not your gender... ?
I don't really feel like diving into the ethics of OpenAI at the moment lol.
If they are shielding you giving back answers, doesn’t mean there is a lot of profiling going on behind the screens of all big tech. How close are they to behavioral monitoring?
OpenAI are liars. I have all the privacy settings on, and it still assumes things about me that it would only do if it knew all my previous conversations.
well I hope it's better than Spotify's age prediction which came to the conclusion that I'm 87 years old.
Seriously though this is the most easily game-able thing imaginable, pretty sure teens are clever enough to figure out how to pretend to be an adult. If you've come to the conclusion that your product is unsuited for kids implement actual age verification instead of these shoddy stochastic surveillance systems. There's a reason "his voice sounded deep" isn't going to work for the cashier who sold kids booze
Let's be honest - to protect the children, big tech will put everyone under the suspicion of being one. And the issue is not how they use the technologies they have, because they have a moral responsibility to do it safely, but that we don't have technologies of hours.
What I wonder lately is how an adult person can be empowered by tech to bare the consequences of their action and the answer usually is that we cannot. We don''t have the means of production in the literal marxist definition of the phrase and we are being shaped by outside forces that define what we can do with ourselves. And it does not matter if those forces are benevolent or not, it matters that it is not us.
The winter is coming and we are short on thermal underware.
The chinese open models being reason for hope is just a very sad joke.
This seems to be a side project of their goal and a good way to calibrate the future ad system predictions.
Lots of upsides for them.
this trillion market is not about empowering users to style their pages more quickly, heh
https://allowe.com/games/larry/tips-manuals/lsl3-age-quiz.ht...
Use an encouraging tone. Adopt a skeptical, questioning approach. Call me on things which don't seem right. List possible assumptions I'm making if any.
We don't need to jam ads into every orifice.
I hope there's more value to be had not doing ads than there is to include them. I'd cancel my codex/chatgpt/claude if someone planted the flag.
OpenAI seems to think it has infinite trust to burn.
Apple is a has-been. Anthropic is best positioned to take up the privacy mantle, and may even be forced to, given that most of their revenue is enterprise and B2B.
Is it not in OpenAI's best interested for them to accidentally flag adults as teens so they have to "verify" their age by handing over their biometric data (Persona face scan) or their government ID? Certainly that level of granularity will enhance the product they offer for their advertisers.
> While this is an important milestone, our work to support teen safety is ongoing.
agreed, I guess we'll be seeing some pushbacks similar to Apple's CSAM but overall it's about getting a better demographics on their consumers for better advertising especially when you have a one-click actions combined with it. We'll be seeing handful of middleware plugins (like Honey) popping up, which I think the intended usecase for something like chat based apps
Surely they're using "history of the user-inputted chat" as a signal and just choosing not to highlight that? Because that would make it so much easier to predict age.
> Users who are incorrectly placed in the under-18 experience will always have a fast, simple way to confirm their age and restore their full access with a selfie through Persona, a secure identity-verification service.
Yea, my linkedin account which was 15 years old and was a paid pro user for several years got flagged for verification (no reason ever given, I rarely used it for anything other than interacting with recruiters) with this same company as their backend provider. They wouldn't accept a (super invasive feeling) full facial scan + a REAL ID, they also wanted a passport. So I opted out of the platform. There was no one to contact - it wasn't "fast" or "easy" at all. This kind of behavior feels like a data grab for more nefarious actors and data brokers further downstream of these kinds of services.
New boss, same as the old boss.
Also:
"Users can check if safeguards have been added to their account and start this process at any time by going to Settings > Account."
Its a feature for advertisers, and investors also wanna know. Did you think you are the customer?
Meanwhile, the adult market is huge and sureshot revenue from a user base that is more likely to not mind the ads.
At this point, just use gemini (yes its google and has its issues if you need SOTA) or I have recently been trying out more and more chat.z.ai for simple text issues (like hey can you fix this docker issue etc.) and I feel like chat.z.ai is pretty good plus open source models (honestly chat.z.ai feels pretty SOTA to me)
I have heard good things about Kagi in fact, that's the reason why I tried orion and still have it in the first place but I haven't bought Kagi, I just used the free searches orion gives & I don't know if it has Kagi's assistant.
I think proton's Lumo is another good bet.
If you want something to not track you, I once asked cerebras team on their discord if they track the queries and responses from their website try now feature and they said that they don't. I don't really see a reason why they might lie about it given that its only meant for very basic purposes & they don't train models or anything.
You also get one of the fastest inferences for models including GLM 4.7 (which z.ai uses)
You might not get search results though but for search related queries duck.ai's pretty good and you can always mix and match.
But Cerebras recently got a 10 billion $ investment from OpenAI and I Have been critical of them from now on so do be wary now.
Kagi Assistant does seem to be good if someone already uses Kagi or has a subscription of it from what I feel like.
So there you go, maybe it wont give exactly what regulators say they want, but it will give exactly what they truly want.
Then maybe OpenAI should just close shop, since (SaaS) LLMs do neither in the mid to long term.
Politicians, CEOs, Lawyers it's standard practice because it's so effective.
Why would it encourage this for anyone?
I've been very aggressive toward OpenAI on here about parental controls and youth protection, and I have to say the recent work is definitely more than I expected out of them.
To me, this feels nefarious with the recent push into advertising. Not only are people dating these chat bots, but they are more trusting of these AI systems than people in their own life. Now, OpenAI is using this "relationship" to influence user's buying behavior.
But consider OPs point — ChatGPT has become a safety-critical system. It is a tool capable of pushing a human towards terrible actions, and there are documented cases of it doing this.
In that context, what is the responsibility of OpenAI to keep their product away from the most vulnerable, and the most easily influenced? More than zero, I believe.
So is Catcher In The Rye and Birth of a Nation.
> the most vulnerable, and the most easily influenced
How exactly is age an indicator of vulnerability or subject-to-influence?
It's really really not. "Safety-critical system" has a meaning, and a chat bot doesn't qualify. Treating the whole world as if it needs to be wrapped in bubble-wrap is extremely unhealthy and it generally just used as an excuse for creeping authoritarianism.
When I read the chat logs of the first teenager who committed suicide with the help and encouragement of ChatGPT I was immediately thinking about ways that could be avoided that make sense in the product. I want companies like OpenAI to have the same reaction and try things. I'm just glad they are.
I'm also fully aware this is unpopular on HN and will get downvoted by people who disagree. Too many software devs without direct experience in safety-critical work (what would you do if you truly are responsible?), too few parents, too many who are just selfishly worried their AI might get "censored".
There are really good arguments against this stuff (e.g. the surveillance effects of identity checks, the efficacy of age verification, etc.) and plenty of nuance to implementations, but the whole censorship angle is lame.
I suppose mainly because I don't think a non-minor committing suicide with ChatGPT's help and encouragement matters less than a minor doing so. I honestly thing the problem is the user interface for GPT being a chat. I think it has a psychological effect that you can talk to ChatGPT the same way you can talk to Emily from school. I don't think this is a solvable problem if OpenAI wants this to be their main product (and obviously they do).
My feelings have absolutely nothing to do with censorship. That’s just an easy straw man for you to try and dismiss my point of view, because you’re scared of not feeling safe.
"Q: ipsum lorem
ChatGPT: response
Q: ipsum lorem"
OpenAI: take this user's conversation history and insert the optimal ad that they're susceptible to, to make us the most $
It’s absolutely crucial for effective ad monetization to know the users age - significant avenues are closed down due to various legislation like COPPA and similar around the world. It severely limits which users can even be subject to ads, the kind of ads, and whether data can be collected for profiling and targeting for ads.
There are dark sides to the rollout that EFF details in their resource hub: https://www.eff.org/issues/age-verification
There is a confluence of surveillance capitalism and a global shift towards authoritarianism that makes it particularly alarming right now.
That was just a spur-of-the-moment question. I've been using ChatGPT for over six months now.
I don't know how OpenAI plans to do this going forward, just quickly read the article and figured that might be a good question to ask ChatGPT.
Edit: I just followed that up with, "Based on everything I've asked, what gender am I?" It refused to answer, stating it wouldn't assume my gender and treats me as gender neutral.
So I guess it's ok for an AI agent to assume your age, but not your gender... ?
I don't really feel like diving into the ethics of OpenAI at the moment lol.
Seriously though this is the most easily game-able thing imaginable, pretty sure teens are clever enough to figure out how to pretend to be an adult. If you've come to the conclusion that your product is unsuited for kids implement actual age verification instead of these shoddy stochastic surveillance systems. There's a reason "his voice sounded deep" isn't going to work for the cashier who sold kids booze
What I wonder lately is how an adult person can be empowered by tech to bare the consequences of their action and the answer usually is that we cannot. We don''t have the means of production in the literal marxist definition of the phrase and we are being shaped by outside forces that define what we can do with ourselves. And it does not matter if those forces are benevolent or not, it matters that it is not us.
The winter is coming and we are short on thermal underware.
The chinese open models being reason for hope is just a very sad joke.