Repository navigation
Description
We've talked a lot about how users can extend the set of analyzers that gopls runs so it can include org-specific checkers. Historically the challenge was that users would have to re-build gopls with the extended set of analyzers (which is not difficult but is inconvenient), or gopls would have to communicate with a separate process that contains the analyzers, which means it would have to do type checking yet again. (gopls already does--and must do--type checking of each package twice: once for its main index, and again for analysis.) But today @findleyr pointed out that, as of v0.12.0, there's no real efficiency reason not to move all of gopls' analysis into the sidecar process, so that the net number of invocations of the type checker is unchanged.
In essence, gopls would fork+exec a long-lived child process and communicate with it over a pipe. (The child process wouldn't need any other capabilities.) A request would send the metadata, source files, facts, and export data required to analyze a package, and the child would do the work and return serialized facts and diagnostics over the pipe. By default the child program would be a mode of gopls itself (e.g. 'gopls analyze'), but users could specify an alternative program that implements the same interface, analogous to the way 'go vet -vettool=...' runs an alternative unitchecker-based tool.
This approach still requires the user to build an executable containing both gopls code (the driver and core analyzers) and user code (their analyzers), but this could be done automatically by gopls: it would generate a main file from the user's configuration and then execute it, a bit like 'go test'. But perhaps this approach (and the potential for version skew of both the go toolchain and the gopls source) is more complex than the simple "re-build" approach. Something to think about.
Activity
findleyr commentedon Apr 28, 2023
Per in-person discussion, we need to decide whether this has significant advantages over simply making it easier to recompile gopls itself.
One primary advantage is isolation: analyzers can't mutate other global state that may affect gopls. But on the other hand, does this limit gopls extensibility to analyers? If so, is that sufficient?
adonovan commentedon Apr 28, 2023
Yes, @hyangah raised a similar question: the user still has to build something containing their analyzers; why should that not just be gopls itself? There are fewer moving parts that way.
The only real advantage I can think of is isolation: a sidecar could make gopls robust to panics and crashes in the analyzers, and with some bisection logic it could automatically identify and retire the culprits. Also, we could run completely untrusted analyzer code safely if it ran in a sandbox (e.g. something like seccomp) with no capabilites. But that does seem like a lot of work, and presumably users trust their own org's analyzers.
The security angle is more interesting in the context of an idea that @griesemer brought up yesterday, of allowing each module to define its own analyzers that are automatically run on packages that import it. This would allow a module with a complex API to provide a set of checkers to catch misuses of it. Checkers could run as part of 'go test' (which is already running untrusted code), or they could be run by a hypothetical 'go build -analyze' command if the checkers can be sandboxed so that they run in an unprivileged process that can't access the file system or network.
Also, @ianthehat mentioned that in an earlier experiment with extensible sets of analyzers, there were significant user interface challenges with too many error messages, and cross-talk between analyzers. I don't have details but perhaps he can elaborate.
findleyr commentedon Apr 28, 2023
Another big advantage of an analyzer sidecar is that it could be dynamic.
An idea that has been thrown around is having codebase-specific analyzers (e.g. I have a local analyzer that I use to catch when I forget to add a Go copyright header). If there were a configurable
"workspaceAnalyzers"setting that pointed to a workspace-relative directory with a pre-determined layout similar tox/tools/go/analysis/passes, we could recompile an analyzer binary for each workspace, and allow users to run their own analyzers for their codebase, for each workspace.ianthehat commentedon Apr 28, 2023
Another potential advantage is the ability to run multiple sidecars and cleanly merge the results
findleyr commentedon Apr 28, 2023
I look forward to a future where gopls hotreloads analyzers in x/tools while I edit them.
timothy-king commentedon Apr 28, 2023
A
goplschecker.Mainanalog to *checker.Main seems like it would be easy for others to adapt a list of analysis.Analyzers to talk thegopls analyzeprotocol. Then a minimum but flexible configuration for external checkers is just a list of external binaries and flags. These could be compiled ahead of time. This only really requires anexternal_analyzersetting.A part of the protocol could be a handshake to check for version skew (+config skew). gopls can warn when a a tool is disabled due to version skew.
More flexible users could just give a list of packages and Analyzer variable and the goplschecker + go.mod could be generated relatively easily. gopls could compile and recompile this on demand. It seems like this would be done fairly infrequently. This could just be another
external_analyzer. My gut instinct is that a provided external_analyzer binary would be more attractive to security sensitive organizations.leighmcculloch commentedon Jul 7, 2023
The difficulty, or inconvenience, for me is not the action of recompiling gopls, but that it is an unintuitive workflow. Setting an analyzer occurs like a config, and it's pretty uncommon when configuring an IDE tool to need to recompile the tool to collect the config.
Is it possible for gopls to call out to a binary that is a singlechecker or multichecker instead of a custom gopls API? Then users can build analysis tools as they do today, and build their multichecker, and use it with gopls.
The checkers have a
-jsonoption. If it isn't sufficient, can we improve the API of the checkers so they're usable with gopls?9 remaining items
timothy-king commentedon Jul 25, 2024
#48429 (or a minor extension to support
go vet -vetool=...in addition togo run) could be a user friendly way to get a sidecars tools for specific modules. It would also be nice to play together with that proposal. There is a lot of overlap with the goals of the two proposals.But AFAICT that kinda requires a separate process for a gopls analyzers (whether in a sidecar or the main gopls process) and that implies type checking twice to enable this. Thats a definite cost. Maybe we have a protocal for the tool to say which analyzers it can run, and if there is 100% overlap with the Analyzer list that gopls would run, we do not run the gopls analyzers that use types.
firelizzard18 commentedon Jul 25, 2024
It seems like Alan is leaning towards a separate process, because otherwise you'd need to recompile gopls for every different combination of analyzers required by a workspace, which probably would mean recompiling it for every workspace that specifies any custom analyzers. With a sidecar process, only the sidecar would need to be recompiled for each workspace.
adonovan commentedon Jul 25, 2024
As a matter of fact, gopls already (necessarily) has two completely separate paths for type checking--one for most features and one for analysis--so it already type-checks some packages twice. So, the net effect on type-checking cost of moving all its analysis logic into a (single) sidecar process would be zero.
findleyr commentedon Jul 25, 2024
Perhaps relevant that in a separate discussion, @aclements reports poor gopls performance when having both the
runtimepackage and a high level package open, because the analysis driver has to reprocess every package in between, making even the VS Code process itself painfully slow to respond.Having analysis in a separate process might make it easier to decrease the priority of that process.
adonovan commentedon Jul 25, 2024
I strongly suspect the performance problem is that we didn't implement type-check batching in the analysis driver; we should do that, to reduce the amount of export-data parsing.
jpalawaga commentedon May 2, 2025
Being able to write static analyzers in a way that doesn't require recompiling gopls would be fantastic, especially when dealing with different platforms (development machines/IDEs, CI boxes, virtual machines, etc).
Custom static analyzers are a really powerful tool for code quality.
ufukty commentedon Feb 1, 2026
Worried to see this issue has been evolved so far from the other1 much needed vscode-go feature request closed in favor of this one; now requires months (if not years) of additional work, like moving type checking from gopls to sidecar, ad-hoc creation and compilation of ephemeral binaries for sidecar, a sidecar-gopls communication protocol, custom analyzer discovery from modules, go.mod support on importing foreign analyzers etc.
I hope you consider the inconvenience of the current options for using multiple custom analyzers that work in both IDE and CI environments. Users would instantly utilize a new way that is other than sending linters to 3rd party analyzer loaders (eg. golangci-lint/revive) or injecting custom analyzers to a fork of gopls and maintaining it to keep in sync with upstream.
I liked the roadmap additions mentioned here. They are just lovely, thoughtful and sound very convenient. Looking forward to seeing how and when they will be implemented.
Best,
Footnotes
https://github.com/golang/vscode-go/issues/2853#issuecomment-1624317519 ↩
adonovan commentedon Feb 1, 2026
I hear your concerns, and I do worry that the approach in this issue may be too complicated. We have not yet decided what approach to take, and we will certainly need to do some exploratory research before we commit to anything. But we recognize that the goal of user-extensible analyzer subsets is an important one.
thepudds commentedon Oct 29, 2026
Anyone following this may be interested in #82464 ("proposal: go test should run analyzers specified by the workspace") by Alan Donovan.
firelizzard18 commentedon Oct 29, 2026
@adonovan @h9jiang I think this is a viable alternative implementation path for golang/vscode-go#2445. If the analyzer sidecar is implemented in such a way that it can be rebuilt with "compile-time plugins", or I can create a custom implementation that extends the default
gopls analyze, then the test analyzer can be maintained as a completely separate project, whether as golang/x or as it's own community project. The right extensibility framework would also make it easy to implement support for stretchr and other test suite frameworks as "plugins" instead of needing to include them in the base implementation.When I say "plugin" I am NOT suggesting actual Go plugins (because of their platform limitations and general clumsiness), I am suggesting something more like golangci-lint's mechanism where it's easy to recompile with additional tools built in.
adonovan commentedon Oct 30, 2026
I think there's no fundamental reason why we can't get gopls to do what
go testwould do in #82464. The only missing piece is the issue of consent: users already know thatgo testexecutes the code in their module and its dependencies, so there's no risk in executing analyzers too. But gopls' diagnose operation is, like go vet, morally a static analysis tool for reading code that you might not trust, so gopls would need to obtain explicit consent, perhaps in the form of a--trust=example.comflag or a{'trust': ["example.com"]}configuration setting.hyangah commentedon Nov 2, 2026
While I think supporting analyzer runs in
go test#82464 is innovative (e.g, allowing project owners to enforce extra correctness checks easily and library authors to provideimportersguardrails against API misuse), I don't think it completely replaces what this issue (extensible gopls sidecar) originally aimed to solve, even ifgoplsrunsgo test-based analyzers in the background and surfaces their findings over LSP:go testrequires a much narrower, carefully curated set of analyzers than an IDE. To prevent unnecessary test failures and slow test runs,go testmust avoid stylistic/refactoring checks, analyzers with non-trivial false positives, and expensive fact-heavy analyzers. In contrast, editor analysis routinely benefits from stylistic hints, simplification checks (e.g. staticcheck's ST*, S*, QF*), and informational diagnostics. Also, gopls already has to tune or disable heavy analyzers to maintain reasonable latency.Adding third-party or org-specific analyzers to go.vet pollutes go.mod and participates in MVS in the current proposal. For example, to run
staticcheckusing the prototype of proposal: go test should run analyzers specified by the workspace #82464, project owners end up adding them togo.mod, pulling their transitive dependencies into MVS. Core and minimal-dependency packages (such as x/sys, x/crypto, or foundational libraries) cannot easily add third-party analyzer dependencies to their build graph just to require project contributors to run checks during development.Diagnostics naturally have varying severity levels, whereas
go testis fundamentally binary pass/fail. Tests are well-suited for high-confidence bugs, but poorly suited for surfacing soft guidance, deprecation notices, or quick-fix hints during development.Regarding the security issue (running untrusted analyzer binaries vs executing code in
go test), couldgoplsaddress this via explicit user consent paired with configuration/binary hash caching? When a user opts in to an external, non-default analysis or sidecar configuration for a workspace,goplscould record the hash and only prompt again if the configuration or dependencies change. Ifgoplseventually auto-runs workspace analyzers discovered via module graph, it will likely need a similar consent mechanism anyway.adonovan commentedon Nov 2, 2026
Yes, that's what I meant by my last sentence above. The obstacle to implementing this issue in gopls (or go vet) is not technical, but consent. We just need to come up with a good UX design for users to express trust. I agree that the gopls (or go vet) and go test approaches are complementary: the first is about respecting the analyzer preferences of the IDE user, whereas the second is about those of the project maintainer.