2018-02-13

Koin vs Vanilla Kotlin

Comparing runtime performance of di with and without the Koin framework

Here we go again: I'm searching for a performant, convenient solution for dependency injection in Kotlin. Koin got my attention, because it seems to be simple and straightforward. Since it uses a lot of small inline functions and seems to have only a few hotspots where performance suckers could lurk, it seemed very promising. But I wouldn't want to use it in my game engine project until I can be sure that the performance impact would be negligible. So I did a ... probably totally flawed microbenchmark, that doesn't show anything, but I want to dump it here nonetheless.

So we have a simple service class and another service class that depends on the first service. The main module depends on the seconds service, so the chain has to be fulfilled.

class MainModuleKoin : Module() {
    override fun context(): Context = applicationContext {
        provide { ServiceA() }
        provide { ServiceB(get()) }
    }
}
class MainModuleVanilla(val serviceA: ServiceA, val serviceB: ServiceB)

class MainComponentKoin : KoinComponent {
    val bla by inject<ServiceB>()
}
class MainComponentVanilla(val bla: ServiceB)

class ServiceA
class ServiceB(val serviceA: ServiceA) {
}

Using Koin, one can simply write a few lines and everything is wired together automatically. Note that the Koin context has to be started and stopped, which has to be excluded from the benchmark later.

@JvmStatic fun benchmarkKoin(): ServiceB {
    return MainComponentKoin().bla
}

@JvmStatic fun stopKoin() {
    closeKoin()
}

@JvmStatic fun startKoin() {
    startKoin(listOf(MainModuleKoin()))
}

@JvmStatic fun benchmarkVanilla(): ServiceB {
    return MainComponentVanilla(ServiceB(ServiceA())).bla
}

The benchmark is executed as follows, to ensure all object creation happens and context creation is done outside of the benchmarked code:

@State(Scope.Thread)
public static class MyState {

    @Setup(Level.Trial)
    public void doSetup() {
        KoinBenchmarkRunner.startKoin();
    }

    @TearDown(Level.Trial)
    public void doTearDown() {
        KoinBenchmarkRunner.stopKoin();
    }

}
@Benchmark
public void benchmarkKoin(Blackhole hole, MyState state) {
    hole.consume(KoinBenchmarkRunner.benchmarkKoin());
    hole.consume(state);
}
@Benchmark
public void benchmarkVanilla(Blackhole hole, MyState state) {
    hole.consume(KoinBenchmarkRunner.benchmarkVanilla());
    hole.consume(state);
}

The result is somehow sobering

Benchmark                          Mode  Cnt          Score         Error  Units
BenchmarkRunner.benchmarkKoin     thrpt  200    1425585.082 ±   31179.345  ops/s
BenchmarkRunner.benchmarkVanilla  thrpt  200  106484919.110 ± 1121927.712  ops/s

Even though I'm aware that this is an artificial benchmark that may be flawed, it's pretty much clear that using Koin will have a huge impact on performance, that could make program infrastrucutre slower by a factor of 100. Of course, we're talking about dependency injection at object creation time, which should be a rare case in a game engine. Nonetheless, not too good from my sight.