How to implement filesystem overlay when lsp is running in web worker

[dhrubomoy]

I have implemented a language server for a DSL using Langium. Followed the example from statemachine DSL to set up the language server to run in a webworker. It works great for a single file.

But I also need to add support for multiple file. I tried to implement and register a filesystem overlay (using registerFileSystemOverlay from codingame/monaco-vscode-files-service-override). I was following the example from python config.ts, which uses web socket to run the language server.

I have added all the overrides I saw in the python example (except for the debug one). The workspace loads fine but I am facing two issues. Firstly, the styling is completely broken. Also the language server is not responding at all. On the main thread its not receiving any message except for $/setTrace (when I turn on/off Manubar, Layout controls etc from the top bar). Any other case like changing the doc, hover tooltip, go to definition, etc the worker is not receiving anything at all.

There must be something wrong in my setup. Any help would be appreciated. Thank you in advance.

Here is the code for config if it helps:

import getKeybindingsServiceOverride from "@codingame/monaco-vscode-keybindings-service-override";
import getLifecycleServiceOverride from "@codingame/monaco-vscode-lifecycle-service-override";
import getLocalizationServiceOverride from "@codingame/monaco-vscode-localization-service-override";
import { createDefaultLocaleConfiguration } from "monaco-languageclient/vscode/services";
import getSecretStorageServiceOverride from "@codingame/monaco-vscode-secret-storage-service-override";
import getStorageServiceOverride from "@codingame/monaco-vscode-storage-service-override";
import getSearchServiceOverride from "@codingame/monaco-vscode-search-service-override";
import getRemoteAgentServiceOverride from "@codingame/monaco-vscode-remote-agent-service-override";
import getEnvironmentServiceOverride from "@codingame/monaco-vscode-environment-service-override";
import getTitleBarServiceOverride from "@codingame/monaco-vscode-view-title-bar-service-override";
import getStatusBarServiceOverride from "@codingame/monaco-vscode-view-status-bar-service-override";
import getBannerServiceOverride from "@codingame/monaco-vscode-view-banner-service-override";
import getPreferencesServiceOverride from "@codingame/monaco-vscode-preferences-service-override";
import {
    defaultHtmlAugmentationInstructions,
    defaultViewsInit,
} from "monaco-editor-wrapper/vscode/services";
import { LogLevel } from "@codingame/monaco-vscode-api";
import { MessageTransports } from "vscode-languageclient";
import type {
    LanguageClientConfigs,
    WrapperConfig,
} from "monaco-editor-wrapper";
import { configureDefaultWorkerFactory } from "monaco-editor-wrapper/workers/workerLoaders";
import myScriptLanguageConfig from "./language-configuration.json?raw";
import responseMyScriptTm from "../../../extension/syntaxes/my-script.tmLanguage.json?raw";
import * as vscode from "vscode";
import {
    RegisteredFileSystemProvider,
    RegisteredMemoryFile,
    registerFileSystemOverlay,
} from "@codingame/monaco-vscode-files-service-override";
import getExplorerServiceOverride from "@codingame/monaco-vscode-explorer-service-override";
import {
    BrowserMessageReader,
    BrowserMessageWriter,
} from "vscode-jsonrpc/browser.js";
import workerUrl from "../worker/my-script-server?worker&url";

const createDefaultWorkspaceContent = (workspacePath: string) => {
    return JSON.stringify(
        {
            folders: [
                {
                    path: workspacePath,
                },
            ],
        },
        null,
        2
    );
};

const createDefaultConfigParams = (
    homeDir: string,
    htmlContainer?: HTMLElement
): ConfigParams => {
    const files = new Map<string, FileDefinition>();
    const workspaceRoot = `${homeDir}/workspace`;
    const configParams: ConfigParams = {
        extensionName: "my-script",
        languageId: "my-script",
        documentSelector: ["qvs"],
        homeDir,
        workspaceRoot: `${homeDir}/workspace`,
        workspaceFile: vscode.Uri.file(
            `${homeDir}/.vscode/workspace.code-workspace`
        ),
        htmlContainer,
       // I probably don't need these since I am not using websocket. But what to use for web worker?
        protocol: "ws",
        hostname: "localhost",
        port: 55555,

        files,
        defaultFile: `${workspaceRoot}/hello2.qvs`,
        helpContainerCmd:
            "docker compose -f ./packages/examples/resources/debugger/docker-compose.yml up -d",
        debuggerExecCall: "graalpy --dap --dap.WaitAttached --dap.Suspend=true",
    };
    const helloPyPath = `${workspaceRoot}/hello.qvs`;
    const hello2PyPath = configParams.defaultFile;

    const helloPyCode = `LET orderIdField = 'OrderID';`;
    const helloPy2Code = `LOAD $(orderIDField) from 'lib://Data/orders.csv' (txt, codepage is 1252, embedded labels, delimiter is ',', msq);`;
    files.set("hello.qvs", {
        code: helloPyCode,
        path: helloPyPath,
        uri: vscode.Uri.file(helloPyPath),
    });
    files.set("hello2.qvs", {
        code: helloPy2Code,
        path: hello2PyPath,
        uri: vscode.Uri.file(hello2PyPath),
    });

    const fileSystemProvider = new RegisteredFileSystemProvider(false);
    fileSystemProvider.registerFile(
        new RegisteredMemoryFile(files.get("hello.qvs")!.uri, helloPyCode)
    );
    fileSystemProvider.registerFile(
        new RegisteredMemoryFile(files.get("hello2.qvs")!.uri, helloPy2Code)
    );
    fileSystemProvider.registerFile(
        new RegisteredMemoryFile(
            configParams.workspaceFile,
            createDefaultWorkspaceContent(configParams.workspaceRoot)
        )
    );
    // fileSystemProvider.registerFile(
    //     createDebugLaunchConfigFile(workspaceRoot, configParams.languageId)
    // );
    registerFileSystemOverlay(1, fileSystemProvider);

    return configParams;
};

export const createLangiumGlobalConfig = (params: {
    languageServerId: string;
    configParams: ConfigParams;
    htmlContainer: HTMLElement;
    worker: Worker;
    messageTransports?: MessageTransports;
    messagePort?: MessagePort;
}): WrapperConfig => {
    const extensionFilesOrContents = new Map<string, string | URL>();
    extensionFilesOrContents.set(
        `/${params.languageServerId}-my-script-configuration.json`,
        myScriptLanguageConfig
    );
    extensionFilesOrContents.set(
        `/${params.languageServerId}-my-script-grammar.json`,
        responseMyScriptTm
    );

    return {
        $type: "extended",
        htmlContainer: params.htmlContainer,
        logLevel: LogLevel.Debug,
        vscodeApiConfig: {
            serviceOverrides: {
                ...getKeybindingsServiceOverride(),
                ...getLifecycleServiceOverride(),
                ...getLocalizationServiceOverride(
                    createDefaultLocaleConfiguration()
                ),
                ...getBannerServiceOverride(),
                ...getStatusBarServiceOverride(),
                ...getTitleBarServiceOverride(),
                ...getExplorerServiceOverride(),
                ...getRemoteAgentServiceOverride(),
                ...getEnvironmentServiceOverride(),
                ...getSecretStorageServiceOverride(),
                ...getStorageServiceOverride(),
                ...getSearchServiceOverride(),
                // ...getDebugServiceOverride(),
                // ...getTestingServiceOverride(),
                ...getPreferencesServiceOverride(),
            },
            userConfiguration: {
                json: JSON.stringify({
                    "workbench.colorTheme": "Default Dark Modern",
                    "editor.guides.bracketPairsHorizontal": "active",
                    "editor.wordBasedSuggestions": "off",
                    "editor.experimental.asyncTokenization": true,
                    "debug.toolBarLocation": "docked",
                }),
            },
            viewsConfig: {
                viewServiceType: "ViewsService",
                htmlAugmentationInstructions:
                    defaultHtmlAugmentationInstructions,
                viewsInitFunc: defaultViewsInit,
            },
            workspaceConfig: {
                enableWorkspaceTrust: true,
                windowIndicator: {
                    label: "my-script",
                    tooltip: "",
                    command: "",
                },
                workspaceProvider: {
                    trusted: true,
                    async open() {
                        window.open(window.location.href);
                        return true;
                    },
                    workspace: {
                        workspaceUri: params.configParams.workspaceFile,
                    },
                },
                configurationDefaults: {
                    "window.title":
                        "my-script${separator}${dirty}${activeEditorShort}",
                },
                productConfiguration: {
                    nameShort: "my-script",
                    nameLong: "my-script",
                },
            },
        },
        extensions: [
            {
                config: {
                    name: "my-script",
                    publisher: "my-trial",
                    version: "1.0.0",
                    engines: {
                        vscode: "*",
                    },
                    contributes: {
                        languages: [
                            {
                                id: "my-script",
                                aliases: ["My Script", "my-script"],
                                extensions: [".qvs"],
                                configuration: `./${params.languageServerId}-my-script-configuration.json`,
                            },
                        ],
                        grammars: [
                            {
                                language: "my-script",
                                scopeName: "source.my-script",
                                path: `./${params.languageServerId}-my-script-grammar.json`,
                            },
                        ],
                    },
                },
                filesOrContents: extensionFilesOrContents,
            },
        ],
        editorAppConfig: {
            monacoWorkerFactory: configureDefaultWorkerFactory,
        },
        languageClientConfigs: {
            configs: {
                myScript: {
                    clientOptions: {
                        documentSelector: params.configParams.documentSelector,
                        workspaceFolder: {
                            index: 0,
                            name: params.configParams.workspaceRoot,
                            uri: vscode.Uri.parse(
                                params.configParams.workspaceRoot
                            ),
                        },
                    },
                    connection: {
                        options: {
                            $type: "WorkerDirect",
                            worker: params.worker,
                            messagePort: params.messagePort,
                        },
                        messageTransports: params.messageTransports,
                    },
                },
            },
        },
    };
};

export const createWrapperConfig = (text: string) => {
    const loadMyScriptWorkerRegular = () => {
        // Language Server preparation
        console.log(`Langium worker URL: ${workerUrl}`);
        return new Worker(workerUrl, {
            type: "module",
            name: "My Script Server Regular",
        });
    };

    const worker = loadMyScriptWorkerRegular();
    const reader = new BrowserMessageReader(worker);
    const writer = new BrowserMessageWriter(worker);
    reader.listen((message) => {
        console.log("Received message from worker:", message);
    });

    const configParams = createDefaultConfigParams("/home/mlc");
    const wrapperConfig = createLangiumGlobalConfig({
        languageServerId: "my-script",
        configParams,
        htmlContainer: document.getElementById("monaco-editor-root")!,
        worker,
        messageTransports: { reader, writer },
    });
    return { wrapperConfig, configParams };
};

In my client code:

import React from "react";
import * as vscode from "vscode";
import { MonacoEditorReactWrapper } from "./monaco-editor-react-wrapper.js";
import { createWrapperConfig } from "./config/wrapperMyScriptConfig.js";
import { MonacoEditorLanguageClientWrapper } from "monaco-editor-wrapper";
import { RegisterLocalProcessExtensionResult } from "@codingame/monaco-vscode-api/extensions";

const text = `SET variable = 'Hello, World!';`;

const appConfig = createWrapperConfig(text);

const onLoad = async (wrapper: MonacoEditorLanguageClientWrapper) => {
    const result = wrapper.getExtensionRegisterResult(
        "my-script"
    ) as RegisterLocalProcessExtensionResult;
    result.setAsDefaultApi();

    // const initResult = wrapper.getExtensionRegisterResult(
    //     "debugger-py-client"
    // ) as RegisterLocalProcessExtensionResult | undefined;
    // if (initResult !== undefined) {
    //     configureDebugging(await initResult.getApi(), appConfig.configParams);
    // }

    await vscode.commands.executeCommand("workbench.view.explorer");
    await vscode.window.showTextDocument(
        appConfig.configParams.files.get("hello2.qvs")!.uri
    );
};

export const MyMonacoEditor = () => {
    return (
        <>
            <div >
                <MonacoEditorReactWrapper
                    // style={{ height: "100%" }}
                    wrapperConfig={appConfig.wrapperConfig}
					onLoad={onLoad}
                    // onTextChanged={onTextChanged}
                />
            </div>
        </>
    );
};

[kaisalmen]

Hi @dhrubomoy can please share a reproducible example? It seems you start to solve too many things at once. You have worker and ws connection configuration in your code above. Do you see any messages arriving in you language server?

[dhrubomoy]

Thanks @kaisalmen for the reply. I set up the worker by following statemachine DSL example. But I am not sure how to integrate multiple file support. I followed the python config.ts, but changed it to work with web worker. But it does not work. The css is broken and

I see only some message arriving in the language server worker. But I don't see any textDocument/ related messages coming in, e.g when I type something, hover over some text etc. Here are the logs

INFO getWorker: moduleId: workerMain.js label: TextEditorWorker
log.js:155 DEBUG Initialization of vscode services completed successfully.
my-script-server-start.ts:17 Starting my-script-server...
log.js:165  INFO Init of EditorApp was completed.
log.js:165  INFO No EditorService configured. monaco-editor will not be started.
log.js:165  INFO getWorker: moduleId: workerMain.js label: TextMateWorker
my-script-server-start.ts:22 Received message from main thread: {jsonrpc: '2.0', id: 0, method: 'initialize', params: {…}}
wrapperMyScriptConfig.ts:274 Received message from worker: {jsonrpc: '2.0', id: 0, result: {…}}
main.js:31 [Violation] 'message' handler took 1842ms
log.js:165  INFO languageClientWrapper (unnamed): Started successfully.
my-script-server-start.ts:22 Received message from main thread: {jsonrpc: '2.0', method: 'initialized', params: {…}}
wrapperMyScriptConfig.ts:274 Received message from worker: {jsonrpc: '2.0', id: 0, method: 'client/registerCapability', params: {…}}
main.js:31 [Violation] 'message' handler took 1915ms
wrapperMyScriptConfig.ts:274 Received message from worker: {jsonrpc: '2.0', id: 1, method: 'workspace/configuration', params: {…}}
main.js:31 [Violation] 'message' handler took 603ms
wrapperMyScriptConfig.ts:274 Received message from worker: {jsonrpc: '2.0', id: 2, method: 'client/registerCapability', params: {…}}
main.js:31 [Violation] 'message' handler took 493ms
my-script-server-start.ts:22 Received message from main thread: {jsonrpc: '2.0', id: 0, result: null}
my-script-server-start.ts:22 Received message from main thread: {jsonrpc: '2.0', method: 'workspace/didChangeConfiguration', params: {…}}
my-script-server-start.ts:22 Received message from main thread: {jsonrpc: '2.0', id: 1, result: Array(1)}
my-script-server-start.ts:22 Received message from main thread: {jsonrpc: '2.0', id: 2, result: null}
Received message from main thread: {jsonrpc: '2.0', method: '$/setTrace', params: {…}}

[dhrubomoy]

Hi @kaisalmen , I have updated the example from statemachine DSL to reproduce the behavior I am facing. Here is the code in a my fork of monaco-language-client
https://github.com/dhrubomoy/monaco-languageclient/blob/dae08156f189261f72f3a329f55190db4094c688/packages/examples/src/langium/statemachine/config/wrapperStatemachineConfig.ts

If you run the branch locally and go to the "Langium Statemachine Client & Language Server (Worker)" exampIe you will see the exact behavior I am facing.

My question is what is the config needed to run the file system with web worker? The python example shows how we can run file service with websocket. Is there any such mechanism for web socket?

[dhrubomoy]

Ahh was able to figure it out, it was a silly mistake. In the config the documentSelector prop had to be the same as the languageId. Then the worker issue got resolved.
For the css issue, I had to import ./resources/styles/views.css in the html file.
Here is the working code
dhrubomoy@6c34437

[dhrubomoy]

Found another issue meanwhile. When the theme is changed to "Light (Visual studio)" and "Light+", the background color of activity bar remans dark. For "Light modern" it works fine. Is monaco-languageclient the right repo to open an issue?

[kaisalmen]

Hi @dhrubomoy this happens in the VS Code Desktop app as well. Seems to be a bug in VS Code themes itself.

[CGNonofr]

Whould not say it's a bug, but a choice of the Light+ theme, which is not the default light theme anymore

[dhrubomoy]

Ah my bad. Yes just noticed its the same issue in vs code desktop as well.