collab: Temporarily bypass LLM rate limiting for staff (#16089)

This PR makes it so staff members will be exempt from rate limiting by
the LLM service.

This is just a temporary measure until we can tweak the rate-limiting
heuristics.

Staff members are still subject to upstream LLM provider rate limits.

Release Notes:

- N/A
This commit is contained in:
Marshall Bowers 2024-08-11 14:41:49 -04:00 committed by GitHub
parent 492f6b9cdf
commit 3140d6ce8c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -400,6 +400,11 @@ async fn check_usage_limit(
];
for (usage, limit, resource) in checks {
// Temporarily bypass rate-limiting for staff members.
if claims.is_staff {
continue;
}
if usage > limit {
return Err(Error::http(
StatusCode::TOO_MANY_REQUESTS,