For those not trying, this allows Deepseek to understand a picture (instead of just extracting text from it), and it can describe what's in the picture, but this is not an image generation system, so you can't ask it to modify an image.
Personally, I'm a bit surprised the DS chat app still doesn't offer its own text to speech and speech to text features (I know DS doesn't have any ASR model for example, but there are quite a few in the open).
Can you explain what the benefits are of actually "talking" with the bot instead of typing and reading?
As someone who would rather send a slack message to a coworker rather than actually walking over and talk to them, the idea of having to talk with my laptop is not appealing at all, haha.
I am someone that prefers a slack message to a coworker than talking to them and I use AI.
My current flow is: Google Eloquent to capture 127WPM (my typing is best case is 65wpm). This lets me get the thoughts out without thinking too much about structure or flow, the same way I would brain-dump type it.
Next I use AI to compress, summarize, and restructure to create a clear coherent message for my peer to read (which is way faster for them).
When communicating with AI, its the same thing, except I skip the second step since AI does a good job at understanding my ramblings.
----
It drives me crazy that some cultures only send voice messages to each other. It drives me crazy they can't be respectful of my time and use STT+AI to convert their 90 second monologue to a few written sentences.
If you spend your life sitting in a chair, that's fine. I tend to get all kinds of ideas, questions, and research needs while I'm walking around. Typing a paragraph or two or context takes too much time and is very risky. Especially when driving. But also just walking, cooking, cleaning, etc. Sometimes it's just not practical - winter, carrying stuff... I mostly feel privileged if I can just sit at a computer and type my question and have the time to read the answer.
When I was still using OpenAI, I used it among other things to translate from English to Spanish while talking to Spanish-speaking people in person.
I understand a bit Spanish but I don’t speak Spanish yet, and they don’t speak English.
I speak English to the AI and end with “translate to Spanish, translation only”, and then the AI says the thing I was saying in Spanish (not perfect but good enough, and also it has a slightly weird accent that might be it using English or English influenced text to speech even when speaking Spanish sentences?).
It's easier, faster, and more natural to talk than to type for the vast, vast majority of people.
This trivial fact of life is observed every day by e.g.:
- students taking notes and finding it necessary to only jot down key facts so that they can keep up,
- stenographers who require special training and equipment to keep up verbatim with live speech in the courtroom,
- annoying colleagues who insist on "hopping on a quick call" or arranging big, wasteful, and disruptive meetings instead of just writing down their problem / sending a message or email,
- friends who insist on sending short voice messages in DMs instead of typing, because it's more "personal" that way (which to be fair it is, but not to the extent proclaimed).
it's very confusing. maaaybe if the stt is good and fast enough, speaking may be faster? english speakers can probably hit 150-180 wpm but seems like a hassle
If so, would other models like ChatGPT benefit from translating the user's prompt to Chinese/Japanese and thinking in Hanzi/Kanji and then converting the response back to the user's language before displaying it?
I believe that most reasoning models actually think in their own "language" which is not really understandable by humans. The thinking traces that are shown in the UI are actually summaries generated by a smaller model in plain english (or user language). Sometimes this leaks through and you see some chinese/japanese characters in e.g. Claude's reasoning.
As far as I'm aware, it's not true for models like DeepSeek or other Chinese open-weight models (at least those that I have seen); their reasoning traces are fully composed from some human language, be it English, Chinese or another one; by the way, most of them can adapt their reasoning based on user language, for example, if user speaks English the reasoning more likely will be in English.
I think that for DeepSeek problem (thinking and replying in Chinese) everything is kinda simpler: in their official chat, they're probably using some kind of system prompt which is (probably) written in Chinese, so that's why model may prefer Chinese in it's output.
Summaries by different smaller models are usually made by closed proprietary models like Claude as a way to combat the distillation of real reasoning traces by competitors. Open weight models show the real reasoning traces. Reasoning traces operate in the same space as the non-reasoning output. It's all just one large text for an LLM. Internally, reasoning is just ordinary chat completion between <think></think> tags.
Are you running out of context? I’ve found that tooling and giberish most of the time happens when I’m butting up against the high watermark of my context window. One other thing it could be, I’ve read that lower quanta like Q1 and Q2 for smaller models can leak Chinese
Turns out, to use Claude Agents SDK, you need to have a vision enabled API. If Deepseek API could see, it can fully drive Claude Code and Claude Agents SDK. A project I'm working on relies on a Claude-in-CloudflareWorker setup and I've been relying on Qwen and gemini flash lite, both more expensive than Deepseek.
same here. I am using Gemini 2.5 Flash as VSCode "vision proivder" for Deepseek V4 Pro, but it is expensive and not accurate. can't wait for native Deepseek vision.
If they'd do one of those little extraneous additions like Qwen does, so that I can have DS4 Flash with Vision that would be great. I've got to run a separate model entirely so that I can get vision and I'd prefer to just put it all in one space.
And it's really good and fast. Have tested with bunch of odd photos on what is happening. Overall the training set seems large enough to know what's what and where
Is that before or after the OpenAI and Anthropic pay off all the people and companies who's copyrights were violated when they used their works for free to train their models?
If everything goes to plan everyone involved with big US models will be trillionaire and everyone else will poor and unemployed. If there are open and cheap to run Chinese models (and please god silicon) the financial house of cards that we have build will fall, people involved with big US models will be poor and unemployed, and everyone else will be slightly less poor and unemployed than in the first scenario.
Personally, I'm a bit surprised the DS chat app still doesn't offer its own text to speech and speech to text features (I know DS doesn't have any ASR model for example, but there are quite a few in the open).
As someone who would rather send a slack message to a coworker rather than actually walking over and talk to them, the idea of having to talk with my laptop is not appealing at all, haha.
My current flow is: Google Eloquent to capture 127WPM (my typing is best case is 65wpm). This lets me get the thoughts out without thinking too much about structure or flow, the same way I would brain-dump type it.
Next I use AI to compress, summarize, and restructure to create a clear coherent message for my peer to read (which is way faster for them).
When communicating with AI, its the same thing, except I skip the second step since AI does a good job at understanding my ramblings.
----
It drives me crazy that some cultures only send voice messages to each other. It drives me crazy they can't be respectful of my time and use STT+AI to convert their 90 second monologue to a few written sentences.
I understand a bit Spanish but I don’t speak Spanish yet, and they don’t speak English.
I speak English to the AI and end with “translate to Spanish, translation only”, and then the AI says the thing I was saying in Spanish (not perfect but good enough, and also it has a slightly weird accent that might be it using English or English influenced text to speech even when speaking Spanish sentences?).
This trivial fact of life is observed every day by e.g.:
- students taking notes and finding it necessary to only jot down key facts so that they can keep up,
- stenographers who require special training and equipment to keep up verbatim with live speech in the courtroom,
- annoying colleagues who insist on "hopping on a quick call" or arranging big, wasteful, and disruptive meetings instead of just writing down their problem / sending a message or email,
- friends who insist on sending short voice messages in DMs instead of typing, because it's more "personal" that way (which to be fair it is, but not to the extent proclaimed).
Is it a new silent update?
I think that for DeepSeek problem (thinking and replying in Chinese) everything is kinda simpler: in their official chat, they're probably using some kind of system prompt which is (probably) written in Chinese, so that's why model may prefer Chinese in it's output.
Or hallucinated
https://github.com/JuliusBrussee/caveman
I use the API however, not the chat interface.
It also happened a handful of times with Anthropic models.
Turns out, to use Claude Agents SDK, you need to have a vision enabled API. If Deepseek API could see, it can fully drive Claude Code and Claude Agents SDK. A project I'm working on relies on a Claude-in-CloudflareWorker setup and I've been relying on Qwen and gemini flash lite, both more expensive than Deepseek.
Can't wait to have it available on deepseek.
does it implies that Liang believes vision/voice is less important on its way to AGI?
At least DeepSeek freely gives back the benefits.
What is good for Dario is good for America.
Any ideas, theories where they get their payoff?