Gmail Quota exceeded error / 429 response

Google Limits: GCP Project and Connected Account

There are two types of limits with Google: your GCP project and the connected account. All can be managed with exponential backoffs, and by reducing the limit size and frequency of requests. See below for more details:

V3 vs. V2: Handling Object Retrieval

In V3, unlike V2, you should not need to GET the object. The core concept of V3 is that you can avoid this step due to the feature rich webhooks which contain the full payload. V3 works with provider limits, which are much lower than V2 Nylas limits.

Checking Google Console Project Limits

Firstly, as the developer, check your Google Console project  if it is hitting any limits. The error message should correspond with the Metric.

Quota Limit Example:

<HttpError 429 "Quota exceeded for quota metric 'Queries' and 
limit 'Queries per minute per user' of service 'gmail.googleapis.com' for consumer 
'project_number:XXXXXXXXXX'.">

<HttpError 403 "Quota exceeded for quota metric 'Queries' and 
limit 'Queries per minute per user' of service 'gmail.googleapis.com' for consumer 
'project_number:XXXXXXXXXX'.">

Gmail Imposed Quota Limit

Secondly, every connected account has a Gmail-imposed 250 quota limit per second. For example:

• Sending two emails per second consumes 200 quota points.
• GET /messages?limit=50 consumes 50x5 = 250 quota points, resulting in a 429 response.

This value is not definitive and fluctuates. We suggest aiming for around the 200 mark to reduce 429 errors.

Why Queries via Nylas API Consume More Quota

When making a query via Nylas API V3:

• Sending two emails via Nylas consumes 230 units:
  • Two send = 200 units
  • Two message.created webhooks = 10 units
  • Two thread.created webhooks = 20 units

Additionally, there are any other background API calls, as our webhooks are feature-rich. If you were to GET any of the objects you would be over the quota.

Prevention: Follow Nylas Recommended Limits

The following limits will work with most providers:

• POST /send - rate: once every 2 seconds
• POST /send - volume: max 2000 per day
• GET /messages - max limit: 40
• GET /threads - max limit: 20

Resolution: GMAIL's Recommendations

You can periodically retry a failed request over an increasing amount of time to handle errors related to rate limits, network volume, or response time. This method is called exponential backoff and is used to improve bandwidth usage and maximize the throughput of requests in concurrent environments. As per their docs.

Exponential Backoff Example:

• Retry a failed request after one second, then after two seconds, then four seconds, etc.
• Do NOT continuously retry the same query. Ensure you back off every request exponentially: try 1 second, then 2, then 3, then 5, and finally 10 seconds later.

Resolving Each Limit

Each limit can be resolved either by:

1. Reducing the Limits on Queries:

   • If a GET query, evaluate why and if you need to make the query.

2. Reducing the Frequency of Queries:

   • Implement exponential backoff.
   • Example is folders, we have webhooks for folders.* allowing you to store values locally.

3. Splitting Processing Across Multiple Gmail Accounts.

Note: Per-user limits cannot be increased by Google for any reason.

Resources:

Sending limits 

Google Workspace quota
Google Workspace document on handling 429 responses
Rackspace conversation - with details on exponential backoff

Updated September 05, 2024 19:43