Russell Bateman
October 2018
last update:
I've been trying to do some Java vs. Docker limits testing. The JVM is not written for Docker use, though it works just as any other application. The problem is that the JVM goes deep into the host in search of what it should understand as memory and CPUs. This is a short-sightedness that's expected given its history and the relatively recent advent of Docker. See Java and Docker, the limitations.
import java.util.ArrayList; import java.util.List; public class MemEat { public static void main( String[] args ) { int cpus = Runtime.getRuntime().availableProcessors(); System.out.println( "Found " + cpus + ( ( cpus == 1 ) ? " CPU." : " CPUs." ) ); //noinspection MismatchedQueryAndUpdateOfCollection List< Object > list = new ArrayList<>(); //noinspection InfiniteLoopStatement while( true ) { byte bytes[] = new byte[ 1048576 ]; list.add( bytes ); Runtime runtime = Runtime.getRuntime(); System.out.println( "free memory: " + runtime.freeMemory() ); } } }
Create a simple Dockerfile to do the stuff shown by the article noted above, which was written up on 16 May 2018.
FROM java:openjdk-8u111 WORKDIR / COPY ./MemEat.java /MemEat.java CMD /bin/bash
Legend:
FROM | The Docker base image, in this case, it appears that there is an image for openjdk-8u111; it's exactly what I want. |
WORKDIR | Where you want your current working directory to be. |
COPY | Directive to copy from some file or directory to some place. |
CMD | Command to run first when the container comes up. |
To build an image based on dockerfile:
russ@nargothrond:~/dev/docker-limits$ docker build --tag docker-limits .
Sending build context to Docker daemon 42.5kB
Step 1/4 : FROM java:openjdk-8u111
---> d23bdf5b1b1b
Step 2/4 : WORKDIR /
---> Using cache
---> 9462a3796ff1
Step 3/4 : COPY ./src/MemEat.java /MemEat.java
---> 136b253704f4
Step 4/4 : CMD /bin/bash
---> Running in 226fddc36f42
Removing intermediate container 226fddc36f42
---> 16592d6f505b
Successfully built 16592d6f505b
Successfully tagged docker-limits:latest
To run the image:
russ@nargothrond:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits
root@b319b30dfb88:/# javac MemEat.java
root@b319b30dfb88:/# java -Xmx100m MemEat
free memory: 99090400
free memory: 98041808
free memory: 96993216
.
.
.
free memory: 7603400
free memory: 6554808
free memory: 5506216
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at MemEat.main(MemEat.java:14)
To see all the images:
russ@nargothrond:~/dev/docker-limits$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b319b30dfb88 docker-limits "/bin/sh -c /bin/bash" 6 minutes ago Exited (1) 26 seconds ago elegant_hoover
To discard images you no longer want:
russ@nargothrond:~/dev/docker-limits$ docker rm b319b30dfb88
b319b30dfb88
Docker base images that demonstrate brokenness/fixes using the code above:
russ@nargothrond:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. root@c48e09d4145b:/# javac MemEat.java root@c48e09d4145b:/# java -version openjdk version "1.8.0_111" OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-2~bpo8+1-b14) OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode) root@c48e09d4145b:/# java MemEat Found 8 CPUs. free memory: 249288432 free memory: 248239840 free memory: 247191248 . . . (runs forever, probably because of kernel not supporting swap limit)
russ@nargothrond:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. root@c48e09d4145b:/# javac MemEat.java root@c48e09d4145b:/# java -version openjdk version "1.8.0_111" OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-2~bpo8+1-b14) OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode) root@c48e09d4145b:/# java -Xmx100m MemEat Found 8 CPUs. free memory: 99090400 free memory: 98041808 free memory: 96993216 . . . free memory: 7603280 free memory: 6554688 free memory: 5506096 Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemEat.main(MemEat.java:18)
russ@nargothrond:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. root@93e61266633e:/# java -version Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap openjdk version "1.8.0_181" OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_181-b13) OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.181-b13, mixed mode) root@93e61266633e:/# javac MemEat.java Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap root@93e61266633e:/# java MemEat Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap Found 8 CPUs. free memory: 6430696 free memory: 5382104 free memory: 4333512 . . . free memory: 2204416 free memory: 1155824 free memory: 1155808 Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemEat.main(MemEat.java:18)
russ@nargothrond:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. root@464dab5dab30:/# javac MemEat.java Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport root@464dab5dab30:/# java MemEat Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport Found 8 CPUs. free memory: 6259312 free memory: 5215184 free memory: 4124152 . . . free memory: 3504160 free memory: 2455568 free memory: 1406976 Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemEat.main(MemEat.java:18)
russ@nargothrond:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. root@8c3089e1a3d9:/# javac MemEat.java Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport root@8c3089e1a3d9:/# java MemEat Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport Found 8 CPUs. free memory: 6259160 free memory: 5215208 free memory: 4124160 . . . free memory: 3504176 free memory: 2455584 free memory: 1406992 Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemEat.main(MemEat.java:18) root@8c3089e1a3d9:/# export PATH=$PATH:/opt/java/openjdk/jdk-10+46/bin/ root@8c3089e1a3d9:/# javac MemEat.java Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport root@8c3089e1a3d9:/# java MemEat Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport Found 8 CPUs. free memory: 6259176 free memory: 5215408 free memory: 4109864 . . . free memory: 3504408 free memory: 2455816 free memory: 1407224 Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemEat.main(MemEat.java:18)
russ@nargothrond:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. root@cfb30ae4f458:/# javac MemEat.java Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport root@cfb30ae4f458:/# java MemEat Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport Found 8 CPUs. free memory: 6308328 free memory: 5264032 free memory: 4172872 . . . free memory: 3553264 free memory: 2504672 free memory: 1456080 Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemEat.main(MemEat.java:18)