This removes the `low_speed_timeout` setting from all providers as a
response to issue #19509.
Reason being that the original `low_speed_timeout` was only as part of
#9913 because users wanted to _get rid of timeouts_. They wanted to bump
the default timeout from 5sec to a lot more.
Then, in the meantime, the meaning of `low_speed_timeout` changed in
#19055 and was changed to a normal `timeout`, which is a different thing
and breaks slower LLMs that don't reply with a complete response in the
configured timeout.
So we figured: let's remove the whole thing and replace it with a
default _connect_ timeout to make sure that we can connect to a server
in 10s, but then give the server as long as it wants to complete its
response.
Closes#19509
Release Notes:
- Removed the `low_speed_timeout` setting from LLM provider settings,
since it was only used to _increase_ the timeout to give LLMs more time,
but since we don't have any other use for it, we simply remove the
setting to give LLMs as long as they need.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Peter Tripp <peter@zed.dev>
- Closes: https://github.com/zed-industries/zed/issues/19609
Switches us to using `-latest` tags with Anthropic models instead of
pinning to a specific date version.
See: [Anthropic Model
Docs](https://docs.anthropic.com/en/docs/about-claude/models)
This is a no-op for:
- Claude 3 Opus (`claude-3-opus-20240229`)
- Claude 3 Sonnet (`claude-3-sonnet-20240229`)
- Claude 3 Haiku (`claude-3-haiku-20240307`)
For Claude 3.5 Sonnet this will update us from
`claude-3-5-sonnet-20240620` to `claude-3-5-sonnet-20241022`. We will
also pickup any subsequent model updates automatically when Anthropic
updates the `latest` tag.
This matches the behavior for OpenAI where use `gpt-4o` as the
model_name and not `gpt-4o-2024-08-06`.
This PR extends the LLM usage tracking to support tracking usage for
cache writes and reads for Anthropic models.
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Antonio <antonio@zed.dev>
Users of our http_client crate knew they were interacting with isahc as
they set its extensions on the request. This change adds our own
equivalents for their APIs in preparation for changing the default http
client.
Release Notes:
- N/A
Release Notes:
- Allow Anthropic custom models to override "temperature"
This also centralized the defaulting of "temperature" to be inside of
each model's `into_x` call instead of being sprinkled around the code.
This PR makes it so we propagate the `stop_reason` from Anthropic up to
the Assistant so that we can take action based on it.
The `extract_content_from_events` function was moved from `anthropic` to
the `anthropic` module in `language_model` since it is more useful if it
is able to name the `LanguageModelCompletionEvent` type, as otherwise
we'd need an additional layer of plumbing.
Release Notes:
- N/A
This PR adjusts the approach we use to encoding tool uses in the
completion response to use a structured format rather than simply
injecting it into the response stream as text.
In #17170 we would encode the tool uses as XML and insert them as text.
This would require then re-parsing the tool uses out of the buffer in
order to use them.
The approach taken in this PR is to make `stream_completion` return a
stream of `LanguageModelCompletionEvent`s. Each of these events can be
either text, or a tool use.
A new `stream_completion_text` method has been added to `LanguageModel`
for scenarios where we only care about textual content (currently,
everywhere that isn't the Assistant context editor).
Release Notes:
- N/A
This PR updates the Assistant with support for receiving tool uses from
Anthropic models and capturing them as text in the context editor.
This is just laying the foundation for tool use. We don't yet fulfill
the tool uses yet, or define any tools for the model to use.
Here's an example of what it looks like using the example `get_weather`
tool from the Anthropic docs:
<img width="644" alt="Screenshot 2024-08-30 at 1 51 13 PM"
src="https://github.com/user-attachments/assets/3614f953-0689-423c-8955-b146729ea638">
Release Notes:
- N/A
This PR removes the `cache_control` field from the variants in
`ResponseContent`.
This field is used on requests to control the caching behavior, but is
not needed on content in the response.
Release Notes:
- N/A
This PR splits the `Content` type for Anthropic into two new types:
`RequestContent` and `ResponseContent`.
As I was going through the Anthropic API docs it seems that there are
different types of content that can be sent in requests vs what can be
returned in responses.
Using a separate type for each case tells the story a bit better and
makes it easier to understand, IMO.
Release Notes:
- N/A
- Cloudflare provides ISO-3166-1 country code for protectorates. Expand our allowlist to include the territories of countries on the allowlist (US, UK, France, Australia, New Zealand).
- Also include the country_code in the error message when we block.
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Release Notes:
- Adds support for Prompt Caching in Anthropic. For models that support
it this can dramatically lower cost while improving performance.
This PR is a refactor to pave the way for allowing the user to view and
edit workflow step resolutions. I've made tool calls work more like
normal streaming completions for all providers. The `use_any_tool`
method returns a stream of strings (which contain chunks of JSON). I've
also done some minor cleanup of language model providers in general,
removing the duplication around handling streaming responses.
Release Notes:
- N/A
This PR makes it so hitting upstream rate limits from Anthropic result
in an HTTP 429 response instead of an HTTP 500.
To do this we need to surface structured errors out of the `anthropic`
crate.
Release Notes:
- N/A
When Anthropic releases a new version of their models, Zed AI users
should always get access to the new version even when using an old
version of zed.
Co-Authored-By: Thorsten <thorsten@zed.dev>
Release Notes:
- N/A
Co-authored-by: Thorsten <thorsten@zed.dev>
This PR updates the LLM service to authorize access to language model
providers based on the requester's country.
We detect the country using Cloudflare's
[`CF-IPCountry`](https://developers.cloudflare.com/fundamentals/reference/http-request-headers/#cf-ipcountry)
header.
The country code is then checked against the list of supported countries
for the given LLM provider. Countries that are not supported will
receive an `HTTP 451: Unavailable For Legal Reasons` response.
Release Notes:
- N/A
We also eliminate the `completion` crate and moved its logic into
`LanguageModelRegistry`.
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
In this pull request, we change the zed.dev protocol so that we pass the
raw JSON for the specified provider directly to our server. This avoids
the need to define a protobuf message that's a superset of all these
formats.
@bennetbo: We also changed the settings for available_models under
zed.dev to be a flat format, because the nesting seemed too confusing.
Can you help us upgrade the local provider configuration to be
consistent with this? We do whatever we need to do when parsing the
settings to make this simple for users, even if it's a bit more complex
on our end. We want to use versioning to avoid breaking existing users,
but need to keep making progress.
```json
"zed.dev": {
"available_models": [
{
"provider": "anthropic",
"name": "some-newly-released-model-we-havent-added",
"max_tokens": 200000
}
]
}
```
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
<img width="624" alt="image"
src="https://github.com/user-attachments/assets/f492b0bd-14c3-49e2-b2ff-dc78e52b0815">
- [x] Correctly set custom model token count
- [x] How to count tokens for Gemini models?
- [x] Feature flag zed.dev provider
- [x] Figure out how to configure custom models
- [ ] Update docs
Release Notes:
- Added support for quickly switching between multiple language model
providers in the assistant panel
---------
Co-authored-by: Antonio <antonio@zed.dev>
Release Notes:
- Added support for interacting with Claude in the assistant panel. You
can enable it by adding the following to your `settings.json`:
```json
"assistant": {
"version": "1",
"provider": {
"name": "anthropic"
}
}
```
Adds a supermaven provider for completions. There are various other
refactors amidst this branch, primarily to make copilot no longer a
dependency of project as well as show LSP Logs for global LSPs like
copilot properly.
This feature is not enabled by default. We're going to seek to refine it
in the coming weeks.
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
