Skip to content

Address feedback to our Kotlin BTAPI impl #6793

Closed

Description

@lefou

Feedback from @ALikhachev submitted via https://youtrack.jetbrains.com/issue/KT-62087/How-to-enable-incremental-compilation-via-CLI-or-generic-build-tool#focus=Comments-27-13289556.0-0

Key points:

  • Classpath contains to many and likely inconsistent jars
  • BT API should always be used with the embeddable compiler, so we need to restrict it's applicability further (when kotlinUseEmbeddableCompiler = true)
  • Should we change KotlinModule.kotlinUseEmbeddableCompiler default to true?
  • Review the classpath snapshot logic, since we likely using it wrong an need to compute it ourselves

Full message:

Thanks, I am interested in taking a look at your implementation and will do this soon. Please be informed the API with the entry point CompilationService was more a prototype which gave us some insights and based on the experience gained, we have deprecated it since Kotlin 2.3.0. The new design is available with the entry point at org.jetbrains.kotlin.buildtools.api.KotlinToolchains. It's still marked as experimental, but we are putting effort into stabilizing it. Mostly it's safe to use already and major changes to the API shape are not expected. Worth mentioning the new API entry point keeps compatibility with the previous versions also. For this, in addition to the org.jetbrains.kotlin:kotlin-build-tools-impl, the artifact org.jetbrains.kotlin:kotlin-build-tools-compat of the version matching the API version should be added at runtime. (https://github.com/JetBrains/kotlin/blob/ce9089dc11b0b8bd9274750c070cab70ffba0096/compiler/build-tools/kotlin-build-tools-compat/README.md)

The exceptions you are getting java.lang.NoSuchMethodError: 'org.jetbrains.kotlin.com.intellij.openapi.vfs.VirtualFileSystem org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreApplicationEnvironment.getJarFileSystem()' might be related to having mixed compiler artifacts in the classpath. It's recommended to have an isolated classloader for the compiler.
By mixed compiler artifacts, I mean we have a confusing set of published artifacts, like org.jetbrains.kotlin:kotlin-compiler and org.jetbrains.kotlin:kotlin-compiler-embeddable (that patterns repeats for many artifacts). We would like to address this confusion in some future (KT-33805 Distribute single version of the compiler (rather than regular and -embeddable variants)). The Build Tools API relies on the -embeddable artifacts.

Here's an example of the recommended approach in the Kotlin Maven plugin with isolated classloader: https://github.com/JetBrains/kotlin/blob/fecfa653742616daa054309116e0e99a4d20f5a0/libraries/tools/kotlin-maven-plugin/src/main/java/org/jetbrains/kotlin/maven/K2JVMCompileMojo.java#L252

Also, I noticed

// Create approach parameters for classpath snapshot-based incremental compilation.
// Pass empty list for newClasspathSnapshotFiles to let the API compute snapshots from classpath JARs.
// See: https://github.com/JetBrains/kotlin/blob/v2.1.20/libraries/tools/kotlin-maven-plugin/src/main/java/org/jetbrains/kotlin/maven/K2JVMCompileMojo.java#L356
val approachParams = new ClasspathSnapshotBasedIncrementalCompilationApproachParameters(
java.util.Collections.emptyList(),
(incrementalCachePath / "shrunk-classpath-snapshot.bin").toIO
)

It's actually not working as you expect. It's a not yet implemented thing in the Kotlin Maven plugin. We do not compute the snapshots automatically, we expect you to compute them explicitly and pass to the API. The reason behind this is that many build systems have concepts of caching, and dependencies are quite often reused between the project modules, so instead of reinventing caches here, we allow explicit control on the snapshotting so that you could use your build system caching mechanisms.

One issue we had was the finishProjectCompilation. We didn't find the proper event to invoke it in a multi-module scenario. We got occasional exceptions like reported here and went with simply never calling it.

It's not dangerous to not call it, but recommended to avoid resource leaks. Though, calling it after each compilation would also be not very efficient, as it will drop compiler caches. We have the same problem in the Kotlin Maven plugin at the moment, so there we drop the caches after each compilation, but I hope we will invent a solution there later.

Activity

  1. added a commit that references this issue on Feb 28, 2026
    4b2fba0
  2. added this to the 1.1.3 milestone on Mar 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions