Pre-migration test checklist for a deprecated AI model

Run the same tasks on the old id and the replacement before the shutdown date. Providers tell you to test. They do not promise the new model accepts the same fields, the same token caps, or the same price.

Last reviewed October 20, 2026. Publisher: SunsetDiff.

Steps

Freeze the old id. Call the pinned snapshot you run today, not a short alias, so the baseline cannot move during the test.
Run the same eval set on the replacement. OpenAI points developers at evals. Anthropic says to test well before the retirement date. Vertex says to test mission-critical features. Azure points at Evaluations for custom workload testing.
Replay one real request unchanged. A 400 from a field the new model rejects is a different failure from the shutdown itself. The request error pages list the fields already documented.
Compare the published caps and prices. Read context, max output, and price on the two official model or pricing pages, then read the SunsetDiff diff.
Measure latency on your prompts. Do not copy a vendor's relative latency label into an SLO. Time your own calls.

Evals

OpenAI's backwards-compatibility note says model prompting behavior between snapshots can change, and that you should pin a model version and run evals.

Anthropic's deprecations page says to test applications with the replacement well before the retirement date. The Vertex model versions page says that if you only have time to stop the errors, you still test every mission-critical feature after you change the model ID. Azure's lifecycle FAQ says to use Evaluations for custom workload testing, plus the Foundry leaderboard and the comparison shown at deploy time.

Sources: OpenAI API backwards compatibility , OpenAI evals , Anthropic model deprecations , Vertex model versions , Foundry Models lifecycle , checked October 20, 2026.

Request fields the new model can reject

Replay a saved production body with only the model id changed. These SunsetDiff pages already quote the official rule for fields that fail today:

temperature, top_p, and top_k on claude-sonnet-5-5 return HTTP 400 when the value is not the default. Source: the Sonnet 5.5 migration guide.
max_tokens on o-series models is rejected. The Chat Completions reference says to send max_completion_tokens.
Function tools on gpt-6-luna need the Responses API, or reasoning_effort set to none on Chat Completions.
thinking disabled, thinking budgets, tool_choice tool or any, and assistant prefill fail on claude-sonnet-5-5. Each page links the migration guide.

Sources: Claude Sonnet 5.5 migration guide , OpenAI function calling , checked October 20, 2026.

Token limits

Compare the published window, not the old code constant. The GPT-5.6 Sol page lists a 1,050,000 token context window, a maximum of 922,000 input tokens, and 128,000 max output tokens.

The Gemini 2.5 Pro model page lists an input token limit of 1,048,576 and a max output of 65,536. The Claude models overview lists a 1M token context window and 128K max output for current Claude models on the synchronous Messages API.

The replace pages show the values SunsetDiff could read. A blank cell was not on the official page.

Sources: GPT-5.6 Sol , Gemini 2.5 Pro , Claude models overview , checked October 20, 2026.

Tools and structured output

Call one tool round trip and one structured response you already depend on. OpenAI's function calling guide is the shape for Responses tools. The Sonnet 5.5 migration guide is the source for the tool_choice values that guide rejects.

Do not assume a JSON schema that validated on the old id still validates. If the official page does not list the field you send, treat a failure as something to read in the error body, then check the request errors.

Sources: OpenAI function calling , Claude Sonnet 5.5 migration guide , checked October 20, 2026.

Price and latency

Read the price on the provider pricing page for both ids before you switch. OpenAI publishes prices at the pricing page. Gemini publishes them on the Gemini API pricing page. The SunsetDiff model page repeats a price only when that page listed it.

The Claude models overview publishes comparative latency as a label relative to the current lineup. It says actual latency depends on prompt length, output length, and thinking effort. Time a sample of your own prompts. Do not turn that label into a millisecond budget.

Sources: OpenAI pricing , Gemini API pricing , Claude models overview , checked October 20, 2026.

Related SunsetDiff pages

All guides

Questions

Is a green eval enough to skip the request replay?

No. An eval can pass while a field such as temperature or max_tokens is rejected. Replay one unchanged production body and read the HTTP status.

Where is the price diff?

On the replace page for that model, when both prices were published. Blank price fields were not on the official page.