refactor(error-logging): Improve kafka error logging

[abedatahub]

• Refactored MCLEmitResult
• Log Kafka publish errors

[abedatahub]

@chakru-r The original exception seems to have been thrown from line 1384

          if (!failedMCLs.isEmpty()) {
            log.error(
                "Failed to produce MCLs: {}",
                failedMCLs.stream()
                    .map(result -> result.getMetadataChangeLog().getEntityUrn())
                    .collect(Collectors.toList()));
            // TODO restoreIndices?
            throw new RuntimeException("Failed to produce MCLs");
          }

The error message:

MCP Processor Error
java.lang.RuntimeException: Failed to produce MCLs
	at com.linkedin.metadata.entity.EntityServiceImpl.lambda$emitMCL$57(EntityServiceImpl.java:1329)

https://vpc-euc1-logs-prod-shared-es-01-be2dv7o6zdiraca5kzvm2znmb4.eu-central-1.es.amazonaws.com/_dashboards/app/data-explorer/discover#?_a=(discover:(columns:!(message,level),isDirty:!t,sort:!()),metadata:(indexPattern:'0f598ad0-b0f4-11ef-b003-2575690fe45c',view:discover))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:'2025-10-31T20:33:20.081Z',to:'2025-11-04T21:33:36.626Z'))&_q=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'0f598ad0-b0f4-11ef-b003-2575690fe45c',key:kubernetes.namespace_name,negate:!f,params:(query:bf94a93599-superbet),type:phrase),query:(match_phrase:(kubernetes.namespace_name:bf94a93599-superbet))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'0f598ad0-b0f4-11ef-b003-2575690fe45c',key:level,negate:!f,params:(query:ERROR),type:phrase),query:(match_phrase:(level:ERROR)))),query:(language:kuery,query:'%22Failed%20to%20produce%20MCLs%22'))

Not sure why we didn't see the preceding log statement however. What am I missing?

(And yes, I still haven't added a statement to log the kafka produce failure yet)

[chakru-r]

Its about 0.5 seconds earlier -- there are quite a bit of logs in between, so filtered for ERROR

[abedatahub]

I was trying to achieve a few different results with this refactoring:

• better docs for states
• better validation; prevent invalid combinations
• move the responsibility for logging to the caller of MCLEmitResult via getProductionResult

[codecov]

Bundle Report

Bundle size has no change ✅

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!