Handling of unicode characters in the CLI integration

[kieferrm]

Type: Bug

The screenshot shows it. The model produced a cat command invocation to write a file. The content contains unicode characters. The outcome is scrambled. This is probably a model issue, but we should first verify.

Extension version: 0.33.2025110501
VS Code version: Code - Insiders 1.106.0-insider (1a756d6, 2025-11-05T05:02:46.325Z)
OS version: Darwin arm64 24.6.0
Modes:

Prompt Details

Debug: [streamChoices] choice { index: 0, finish_reason: null }
Debug: [streamChoices] chunk data: {"choices":[{"text":"\n","index":0,"finish_reason":"stop"}]}
Debug: [streamChoices] choice { text: '\n', index: 0, finish_reason: 'stop' }

Debug: [streamChoices] completion.finishReason n {

properties: {

headerRequestId: '4a8f1e23-3eaa-4870-93e6-6f88227e7b99',

opportunityId: 'icr-49db8784-a583-45d1-b01e-0d679d4e8585',

copilot_build: '1',

copilot_buildType: 'nightly',

languageId: 'jsonc',

afterAccept: 'false',

isSpeculative: 'false',

gitRepoInformation: 'unavailable',

engineName: '',

engineChoiceSource: 'default',

isMultiline: 'true',

isCycling: 'false',

endpoint: 'completions',

uiKind: 'ghostText',

temperature: '0',

n: '1',

stop: 'unset',

logit_bias: 'null',

completionChoiceFinishReason: 'stop'

},

measurements: { documentLength: 26903, documentLineCount: 841 },

issuedTime: 66493998.814416,

filtersAndExp: {

filters: hj { filters: [Object] },

exp: n { variables: {}, features: '' }

},

displayedTime: undefined

}

Info: [streamChoices] solution 0 returned. finish reason: [stop]

Debug: [streamChoices] solution 0 details: finishOffset: [undefined]

Debug: [streamChoices] chunk data: [DONE]
Debug: [streamChoices] request done: headerRequestId: [4a8f1e23-3eaa-4870-93e6-6f88227e7b99] model deployment ID: []

Debug: [streamChoices] request stats: ket {

choices: Map(1) { 0 => Tet { yieldedTokens: 2, seenTokens: 2 } }

}

Debug: [ghostText] All choices redacted

Debug: [AsyncCompletionManager] [4a8f1e23-3eaa-4870-93e6-6f88227e7b99] Request failed with all choices redacted

Debug: [AsyncCompletionManager] [4a8f1e23-3eaa-4870-93e6-6f88227e7b99] No matching completions found

Debug: [getCompletions] No ghostText produced -- empty: received no results from async completions

Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: false.

Debug: [context keys] Window state change. Needs offline check: false, active: false, focused: false.

Debug: [context keys] Window state change. Needs offline check: false, active: false, focused: true.

Trace: No test results

Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: true.

Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: false.

Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: true.

Trace: [GitServiceImpl][doOpenRepository] Active repository: {"_repo":{"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"inputBox":{},"state":{},"ui":{}},"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"headBranchName":"satisfied-buzzard","headCommitHash":"1d3fb0e2f92a58b597a2ee72e16a448f14a35781","upstreamBranchName":"satisfied-buzzard","upstreamRemote":"origin","isRebasing":true,"remotes":["origin"],"remoteFetchUrls":["git@github.com:kieferrm/plan-tooling.git"],"changes":{"mergeChanges":[],"indexChanges":[],"workingTree":[],"untrackedChanges":[]},"_onDidChangeSignal":{"_observers":{},"debugName":"Observable Signal From Event"},"headBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"headCommitHashObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamRemoteObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"isRebasingObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"_checkIsIgnored":{"_waitingTimeMs":1000,"_queue":[],"_timeout":null},"_isIgnored":{"_map":{},"_map2":{}}}

Trace: No test results

Trace: [GitServiceImpl][doOpenRepository] Active repository: {"_repo":{"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"inputBox":{},"state":{},"ui":{}},"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"headBranchName":"satisfied-buzzard","headCommitHash":"1d3fb0e2f92a58b597a2ee72e16a448f14a35781","upstreamBranchName":"satisfied-buzzard","upstreamRemote":"origin","isRebasing":true,"remotes":["origin"],"remoteFetchUrls":["git@github.com:kieferrm/plan-tooling.git"],"changes":{"mergeChanges":[],"indexChanges":[],"workingTree":[],"untrackedChanges":[]},"_onDidChangeSignal":{"_observers":{},"debugName":"Observable Signal From Event"},"headBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"headCommitHashObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamRemoteObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"isRebasingObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"_checkIsIgnored":{"_waitingTimeMs":1000,"_queue":[],"_timeout":null},"_isIgnored":{"_map":{},"_map2":{}}}

Debug: Loaded 6 events for session e7a686aa-b17b-4ead-97ad-01da6ce6d3fa

Debug: Loaded 40 events for session 5d4e72b7-89bf-4d14-bfce-27e07f953491

Debug: Loaded 6 events for session 4487feb9-a4fb-415f-ae4f-0cb371be9e93

Debug: Loaded 6 events for session f0f8969c-293a-401c-95c3-aef24a210d6e

Debug: Loaded 21 events for session 7077e290-0907-4b08-a1a7-8fb3939a34f4

Debug: Loaded 58 events for session 751be174-3aaa-4f05-8846-7079256cb9e0

Debug: Loaded 34 events for session df3f600d-8a28-43ac-8e92-2fbd7af43288

Debug: Loaded 55 events for session b7d5b6af-5318-4751-8c9e-f04df599e34b

Debug: Loaded 5 events for session 67b3c130-9399-4463-b777-87269dd8080d

Debug: Loaded 6 events for session d06c5f97-fc17-47b2-91d3-7432e1f56aad

Debug: Loaded 42 events for session 5c334910-5c84-46fb-86fa-b4b3aeeff088

Debug: Found 11 JSONL sessions and 2 legacy sessions

Info: Loaded legacy session 5f2e84f7-f1bf-448c-8feb-291623105e93 from session_5f2e84f7-f1bf-448c-8feb-291623105e93_1760028210888.json

Info: Loaded legacy session 4ccbd271-2bb5-4c7e-8de2-d35781b76be4 from session_4ccbd271-2bb5-4c7e-8de2-d35781b76be4_1759785249774.json

Trace: [GitServiceImpl][doOpenRepository] Active repository: {"_repo":{"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"inputBox":{},"state":{},"ui":{}},"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"headBranchName":"satisfied-buzzard","headCommitHash":"1d3fb0e2f92a58b597a2ee72e16a448f14a35781","upstreamBranchName":"satisfied-buzzard","upstreamRemote":"origin","isRebasing":true,"remotes":["origin"],"remoteFetchUrls":["git@github.com:kieferrm/plan-tooling.git"],"changes":{"mergeChanges":[],"indexChanges":[],"workingTree":[],"untrackedChanges":[]},"_onDidChangeSignal":{"_observers":{},"debugName":"Observable Signal From Event"},"headBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"headCommitHashObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamRemoteObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"isRebasingObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"_checkIsIgnored":{"_waitingTimeMs":1000,"_queue":[],"_timeout":null},"_isIgnored":{"_map":{},"_map2":{}}}

Trace: [GitServiceImpl][getRepositoryFetchUrls] URI: file:///Users/kmaetzel/Development/projects/plan-tooling/cli/integration-tests/validate-plan-test.ts

Trace: [GitServiceImpl][getRepositoryFetchUrls] URI: file:///Users/kmaetzel/Development/projects/plan-tooling/cli/integration-tests/validate-plan-test.ts

Trace: [GitServiceImpl][getRepositoryFetchUrls] Remotes (open repository): {"rootUri":{"$mid":1,"path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"remoteFetchUrls":["git@github.com:kieferrm/plan-tooling.git"]}

Trace: [GitServiceImpl][getRepositoryFetchUrls] Remotes (open repository): {"rootUri":{"$mid":1,"path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"remoteFetchUrls":["git@github.com:kieferrm/plan-tooling.git"]}

Trace: No test results

Trace: RemoteEmbeddingsComputer::computeEmbeddings started

Trace: RemoteEmbeddingsComputer::computeEmbeddings success. Elapsed 297.76320800185204

Debug: Related Information: Remote similarly request took 298ms

Debug: Related Information: Successfully Calculated, returning 302ms

Trace: No test results

Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: false.

Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: true.

Request IDs

System Info

Item	Value
CPUs	Apple M4 (10 x 2400)
GPU Status	2d_canvas: enabled direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: enabled_on trees_in_viz: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	2, 2, 2
Memory (System)	32.00GB (1.12GB free)
Process Argv	--enable-proposed-api lszomoru.lszomoru-proposed-api-sample --enable-proposed-api donjayamanne.kusto --crash-reporter-id ef10dfea-3602-4c74-ba1a-109ce7da7825
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
pythonvspyt551cf:31249598
nativeloc1:31118317
dwcopilot:31158714
dwoutputs:31242946
copilot_t_ci:31333650
g012b348:31231168
6gi0g917:31259950
pythonrdcb7:31268811
pythonpcpt1:31399616
6518g693:31302842
6abeh943:31336334
envsactivate1:31349248
editstats-enabled:31346256
aa_c:31379597
c12345:31413568
cloudbuttont:31366566
aihoversummaries_t:31413033
todos-1:31405333
qwen_all_req:31396049
3efgi100_wstrepl:31403338
trigger-command-fix:31379601
chat-sessions-view-t:31407655
auto_model_enabled:31396818
use-responses-api:31390341
applesignin:31399434
9i271672:31411680
ddidt:31398484
je187915:31401257
control_gfa0i828:31401959
f1721337:31405248
afag3459:31405336
disable_auto_fix_gb0j8799:31411651
agentic_ask_h7d75428:31406124
ec5jj548:31408805
copilot-nes-oct-t:31413017
metis_rerank_treatment1:31411021
ff8f5884:31411053
terminalsuggestenabled:31411718
cmp-ext-treat:31412886
none:31413573

[DonJayamanne]

@kieferrm
please can you try to replicate this again.
If it does repro please can you share the session file from ~/.copilot/session-state/
The file should be one of the latest *.jsonl files in the above directory.
I'd like to see what commands were generated by the tool

Note: We (chat extension/vscode) doesn't make any of the changes to the file, so this most certainly looks like an issue with the model or sdk in applying the changes to the file.

Note: Im unable to repro at my end, tried a few times already.