Repository navigation
Description
Feedback from @ALikhachev submitted via https://youtrack.jetbrains.com/issue/KT-62087/How-to-enable-incremental-compilation-via-CLI-or-generic-build-tool#focus=Comments-27-13289556.0-0
Key points:
- Classpath contains to many and likely inconsistent jars
- BT API should always be used with the embeddable compiler, so we need to restrict it's applicability further (when
kotlinUseEmbeddableCompiler = true) - Should we change
KotlinModule.kotlinUseEmbeddableCompilerdefault totrue? - Review the classpath snapshot logic, since we likely using it wrong an need to compute it ourselves
Full message:
Thanks, I am interested in taking a look at your implementation and will do this soon. Please be informed the API with the entry point CompilationService was more a prototype which gave us some insights and based on the experience gained, we have deprecated it since Kotlin 2.3.0. The new design is available with the entry point at org.jetbrains.kotlin.buildtools.api.KotlinToolchains. It's still marked as experimental, but we are putting effort into stabilizing it. Mostly it's safe to use already and major changes to the API shape are not expected. Worth mentioning the new API entry point keeps compatibility with the previous versions also. For this, in addition to the org.jetbrains.kotlin:kotlin-build-tools-impl, the artifact org.jetbrains.kotlin:kotlin-build-tools-compat of the version matching the API version should be added at runtime. (https://github.com/JetBrains/kotlin/blob/ce9089dc11b0b8bd9274750c070cab70ffba0096/compiler/build-tools/kotlin-build-tools-compat/README.md)
The exceptions you are getting java.lang.NoSuchMethodError: 'org.jetbrains.kotlin.com.intellij.openapi.vfs.VirtualFileSystem org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreApplicationEnvironment.getJarFileSystem()' might be related to having mixed compiler artifacts in the classpath. It's recommended to have an isolated classloader for the compiler.
By mixed compiler artifacts, I mean we have a confusing set of published artifacts, like org.jetbrains.kotlin:kotlin-compiler and org.jetbrains.kotlin:kotlin-compiler-embeddable (that patterns repeats for many artifacts). We would like to address this confusion in some future (KT-33805 Distribute single version of the compiler (rather than regular and -embeddable variants)). The Build Tools API relies on the -embeddable artifacts.Here's an example of the recommended approach in the Kotlin Maven plugin with isolated classloader: https://github.com/JetBrains/kotlin/blob/fecfa653742616daa054309116e0e99a4d20f5a0/libraries/tools/kotlin-maven-plugin/src/main/java/org/jetbrains/kotlin/maven/K2JVMCompileMojo.java#L252
Also, I noticed
// Create approach parameters for classpath snapshot-based incremental compilation.
// Pass empty list for newClasspathSnapshotFiles to let the API compute snapshots from classpath JARs.
// See: https://github.com/JetBrains/kotlin/blob/v2.1.20/libraries/tools/kotlin-maven-plugin/src/main/java/org/jetbrains/kotlin/maven/K2JVMCompileMojo.java#L356
val approachParams = new ClasspathSnapshotBasedIncrementalCompilationApproachParameters(
java.util.Collections.emptyList(),
(incrementalCachePath / "shrunk-classpath-snapshot.bin").toIO
)It's actually not working as you expect. It's a not yet implemented thing in the Kotlin Maven plugin. We do not compute the snapshots automatically, we expect you to compute them explicitly and pass to the API. The reason behind this is that many build systems have concepts of caching, and dependencies are quite often reused between the project modules, so instead of reinventing caches here, we allow explicit control on the snapshotting so that you could use your build system caching mechanisms.
One issue we had was the finishProjectCompilation. We didn't find the proper event to invoke it in a multi-module scenario. We got occasional exceptions like reported here and went with simply never calling it.
It's not dangerous to not call it, but recommended to avoid resource leaks. Though, calling it after each compilation would also be not very efficient, as it will drop compiler caches. We have the same problem in the Kotlin Maven plugin at the moment, so there we drop the caches after each compilation, but I hope we will invent a solution there later.
Activity
Use KotlinToolchains (fixes com-lihaoyi#6793) (com-lihaoyi#6857)