sbt-assembly "deduplicate: different file contents found" in CI

By Daniel Zoghalchali·Latchkey

sbt-assembly builds a fat JAR by merging dependency JARs. Two of them contain a file at the same path with different bytes, and the default strategy refuses to choose.

What this error means

The assembly task fails with deduplicate: different file contents found in the following, listing the conflicting JARs and the shared path. It is deterministic given the dependency set.

sbt

[error] deduplicate: different file contents found in the following:
[error] .../netty-common-4.1.jar:META-INF/io.netty.versions.properties
[error] .../netty-buffer-4.1.jar:META-INF/io.netty.versions.properties

Common causes

Multiple JARs ship the same metadata file

Files like META-INF service descriptors or versions.properties appear in several JARs with different contents, and the default merge strategy treats that as a conflict.

Duplicate classes from overlapping deps

Two libraries bundle the same class (a shaded or relocated copy), producing different bytes at the same path.

How to fix it

Add a merge strategy for the conflicting paths

Tell assembly how to resolve the specific overlapping files.

build.sbt

assembly / assemblyMergeStrategy := {
  case PathList("META-INF", "io.netty.versions.properties") => MergeStrategy.first
  case PathList("META-INF", xs @ _*) => MergeStrategy.discard
  case x => (assembly / assemblyMergeStrategy).value(x)
}

Remove the duplicate dependency

Inspect sbt evicted to find overlapping libraries.
Exclude the redundant copy or upgrade to a single version.
Prefer one canonical implementation over two shaded ones.

How to prevent it

Define a merge strategy for known META-INF conflicts.
Keep the dependency set free of duplicate classes.
Build the assembly locally before pushing.

Frequently asked questions

What causes ""deduplicate: different file contents found""?