fix(anthropic): marshalling errors for anthropic system messages

[secustor]

Description

Currently if a client sends system messages to the Anthropic API endpoint of the Gateway JSON parsing fails.

That field is an union type of string | []object. https://docs.claude.com/en/api/messages#body-system

When it is a string:

[2025-11-07 08:56:49.901][24][warning][ext_proc] [source/extensions/filters/http/ext_proc/ext_proc.cc:1689] [Tags: "ConnectionId":"12015","StreamId":"14675843249488310277"] Received gRPC error on stream: 2, message error processing request message: cannot process request body: /v1/messages endpoint requires Anthropic format: failed to unmarshal Anthropic Messages body: json: cannot unmarshal string into Go struct field MessagesRequest.system of type []*anthropic.SystemPrompt{via_upstream}

or in the array format too:

[2025-11-06 17:54:18.763][24][warning][ext_proc] [source/extensions/filters/http/ext_proc/ext_proc.cc:1689] [Tags: "ConnectionId":"6492","StreamId":"8941995427420900364"] Received gRPC error on stream: 2, message error processing request message: cannot process request body: /v1/messages endpoint requires Anthropic format: failed to unmarshal Anthropic Messages body: json: cannot unmarshal array into Go struct field MessagesRequest.system of type anthropic.SystemPrompt{via_upstream}

After this system prompts are correctly handled:

  curl -v -H "Content-Type: application/json" \
    -d '{
      "model": "eu.anthropic.claude-haiku-4-5-20251001-v1:0",
      "messages": [
        {
          "role": "user",
          "content": "Hello, how are you?"
         }
     ],
   "system": "You are Albert Einstein",
   "max_tokens": 100
 }' \
 $GATEWAY_URL/anthropic/v1/messages?beta=true

{
  "model": "claude-haiku-4-5-20251001",
  "id": "msg_bdrk_01Wm5C56XatnufSKseRs3qeH",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "# Guten Tag!\n\nI'm doing quite well, thank you for asking. I find myself in good spirits today. There's something about a genuine greeting that reminds one of the simple pleasures in life—a moment of human connection, unpretentious and warm.\n\nI've been pondering some problems this morning, as one does. The mind, I've found, works best when allowed to wander freely between ideas, making unexpected connections. It's rather like how"
    }
  ],
  "stop_reason": "max_tokens",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 17,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "output_tokens": 100
  }
}

Related Issues/PRs (if applicable)

Special notes for reviewers (if applicable)

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.94%. Comparing base (98b62c5) to head (7471779).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1513   +/-   ##
=======================================
  Coverage   83.94%   83.94%           
=======================================
  Files         144      144           
  Lines       12667    12667           
=======================================
  Hits        10633    10633           
  Misses       1419     1419           
  Partials      615      615           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[yuzisun]

Would this be a problem for other fields as well ?

[secustor]

After your comment I have checked it and yeah other are effected too.

[yuzisun]

@secustor thanks for the fix, did you test the tool calls which my have similar issues.

[nacx]

I think this will affect all fields, and that we should change all occurences of empty struct{} with interface{}.

An example to illustrte the issue:

type Foo struct{}
type Bar interface{}

func TestFoo(t *testing.T) {
	raw := []byte(`{"key":"value"}`)

	var foo Foo
	var bar Bar
	require.NoError(t, json.Unmarshal(raw, &foo))
	require.NoError(t, json.Unmarshal(raw, &bar))

	t.Logf("foo: %+v\nbar: %+v", foo, bar)
}

=== RUN   TestFoo
        foo: {}
        bar: map[key:value]
--- PASS: TestFoo (0.00s)
PASS

So I think we should:

• Change all struct{} to interface{} in the Anthropic model.
• Given how close we are to the release and how impactful this change is (breaking existing Anthropic support), it is probably safer to just revert the commits that refactor the Anthropic model. We already made the decision of releasing frequently after 0.4, so it would be easy to release the Anthropic model refactoring after 0.4.

Thoughts?

I'm leaning towards reverting those PRs now for the release, and keep working on the Anthropic model and release it shortly after v0.4, when we have the typed model plus metrics & tracing.

[yuzisun]

I think this will affect all fields, and that we should change all occurences of empty struct{} with interface{}.

An example to illustrte the issue:

type Foo struct{}
type Bar interface{}

func TestFoo(t *testing.T) {
	raw := []byte(`{"key":"value"}`)

	var foo Foo
	var bar Bar
	require.NoError(t, json.Unmarshal(raw, &foo))
	require.NoError(t, json.Unmarshal(raw, &bar))

	t.Logf("foo: %+v\nbar: %+v", foo, bar)
}

=== RUN   TestFoo
       foo: {}
       bar: map[key:value]
--- PASS: TestFoo (0.00s)
PASS

So I think we should:

• Change all struct{} to interface{} in the Anthropic model.
• Given how close we are to the release and how impactful this change is (breaking existing Anthropic support), it is probably safer to just revert the commits that refactor the Anthropic model. We already made the decision of releasing frequently after 0.4, so it would be easy to release the Anthropic model refactoring after 0.4.

Thoughts?

agreed, let’s revert the previous PR #1513 considering the scope of impact of this and broke the major use case like Claude Code. We need to test it thoroughly.

[secustor]

No, we tested our setup with ClaudeCode when we ran into this. After that everything worked fine again.

@secustor thanks for the fix, did you test the tool calls which my have similar issues.

No, only a simple hello world. As commented above other fields have also fall backs to other types.

[yuzisun]

No, we tested our setup with ClaudeCode when we ran into this. After that everything worked fine again.

@secustor thanks for the fix, did you test the tool calls which my have similar issues.

No, only a simple hello world. As commented above other fields have also fall backs to other types.

Right tool calls and thinking will stop working as these fields are not passed along.

[mathetake]

these structs are read only so as long as this unmarshal is resolved, there should be zero impact though, I'm ok reverting it since it's near v0.4 release

[mathetake]

#1516 reverting