Solving the Tomcat/Logback nightmare

Please note, before we really get started, that what we're doing here is removing support for juli and replacing it with Logback support. juli refers to Tomcat's own implementation of key elements of the java.util.logging API. Since version 6, Tomcat has used a private, package-renamed implementation of Apache Commons Logging in order to allow applications to use their own independent copies of the Apache Commons Logging library. It's hard-coded to be simplified and based on juli.

To configure Tomcat to use alternative logging frameworks, like Logback, one must replace the logging library with the one built using the full implementation. Apache claims that a web application can use any logging framework of its choice.

Whatever is supposed to work, it's pretty clear that if you want to use Logback, any logback.xml internal to your application, typically on the path WEB-INF/classes, is ignored and, while logging works, for example, Logback with an slf4j façade, happens with a default configuration as if logback.xml (or its alternates) weren't even there. In other words, if you promote a package to TRACE, it will likely come out in the system console of your IDE's unit testing, but it won't reach catalina.out in your production Tomcat (or even your IDE's notion of catalina.out when running on its "internal" Tomcat).

In my experience with Tomcat, log4j and logback used in my application are not configurable. I only get INFO-level statements. Others must have realized this as well; hence the implementation here (the first one).

Why Docker?

Because ultimately, we don't want to have to muck up Tomcat every time we down-load a fresh one.

This is our second try (the first is down at the end of this document. Follow these linked instructions...

  1. https://shibboleth.atlassian.net/wiki/spaces/KB/pages/1404895579/Apache+Tomcat+logging+with+Logback,
    A more encouraging README than the main one, a good place to start.
  2. https://github.com/tomcat-slf4j-logback/tomcat-slf4j-logback/blob/master/README.md,
    The main README. You still pretty much have to read this document's Quick Start section.
  3. https://github.com/tomcat-slf4j-logback/tomcat-slf4j-logback/releases,
    Download the amended Tomcat files in a zip; pick one (not two or more). I put the download I chose on the path ~/Downloads/tomcat-logback. The download I chose, tomcat-juli-9.0.14-slf4j-1.7.25-logback-1.2.3.zip, appeared to be the latest possible version respecting these caveats:
    • Tomcat 9
    • Assumption that version 9.0.14 is later than 9.0.8
    • No alpha or beta in release names

Where to start?

I downloaded another copy of Tomcat 9 for the purpose of bastardizing it with the changes suggested by the links above. I record the steps as I make this happen:

  1. Download a new Tomcat and explode it. Subdirectory dev is where I will do this work:
    russ@tirion ~/dev $ tar -zxf apache-tomcat-9.0.60.tar.gz
    
  2. Undertake what's suggested below. Pay close attention to whether you can merely replace the files in the Tomcat distro or only sections of the files.

Merge tomcat-slf4j-logback changes into formal distro...

Downloads from the linked instructions in item C of the list way above. In bold, what we need in Dockerfile. (We won't build our new Dockerfile until after trying out these changes to make sure they work.)

russ@tirion ~/Downloads/tomcat-logback/slf4j $ ll
total 2008
drwxrwxr-x 6 russ russ    4096 Mar 21 17:41 .
drwxrwxr-x 3 russ russ    4096 Mar 21 17:35 ..
drwxr-xr-x 2 russ russ    4096 May 19  2018 bin
drwxr-xr-x 2 russ russ    4096 May 19  2018 conf
drwxr-xr-x 2 russ russ    4096 May 19  2018 lib
-rw-r--r-- 1 russ russ   11368 Mar  4  2017 LICENSE
-rw-r--r-- 1 russ russ     471 Mar  4  2017 LICENSE-logback.txt
-rw-r--r-- 1 russ russ    1135 Mar  4  2017 LICENSE-slf4j.txt
-rw-r--r-- 1 russ russ   56814 Mar  4  2017 LICENSE-tomcat.txt
drwxr-xr-x 2 russ russ    4096 Nov 20  2016 src
-rw-rw-r-- 1 russ russ 1953493 Mar 21 17:37 tomcat-juli-9.0.14-slf4j-1.7.25-logback-1.2.3.zip
russ@tirion ~/Downloads/tomcat-logback/slf4j $ ll bin
total 896
drwxr-xr-x 2 russ russ   4096 May 19  2018 .
drwxrwxr-x 6 russ russ   4096 Mar 21 17:41 ..
-rw-r--r-- 1 russ russ    558 Feb 11  2018 setenv.bat
-rw-r--r-- 1 russ russ    519 Mar  4  2017 setenv.sh
-rw-r--r-- 1 russ russ 899180 May 19  2018 tomcat-juli.jar
russ@tirion ~/Downloads/tomcat-logback/slf4j $ ll conf
total 28
drwxr-xr-x 2 russ russ 4096 May 19  2018 .
drwxrwxr-x 6 russ russ 4096 Mar 21 17:41 ..
-rw-r--r-- 1 russ russ 1533 Mar 18  2018 logback-access.xml
-rw-r--r-- 1 russ russ 4554 Apr  8  2018 logback.xml
-rw-r--r-- 1 russ russ 7391 Mar  4  2017 server.xml
russ@tirion ~/Downloads/tomcat-logback/slf4j $ ll lib
total 580
drwxr-xr-x 2 russ russ   4096 May 19  2018 .
drwxrwxr-x 6 russ russ   4096 Mar 21 17:41 ..
-rw-r--r-- 1 russ russ 107064 Apr 14  2017 logback-access-1.2.3.jar
-rw-r--r-- 1 russ russ 471901 Apr  5  2017 logback-core-1.2.3.jar

Perform steps below in ~/Downloads/tomcat-logback.

  1. I start in ~Dowloads/tomcat-logback/slf4j/bin exploded from the zip I downloaded from step C in the list above.
  2. Merge bin; this is a straight-over copy since neither the script nor the JAR file exist. Observe here:
    $ diff ./setenv.sh ~/dev/apache-tomcat-9.0.60/bin/setenv.sh
    diff: /home/russ/dev/apache-tomcat-9.0.60/bin/setenv.sh: No such file or directory
    $ cp setenv.sh ~/dev/apache-tomcat-9.0.60/bin
    $ cp tomcat-juli.jar ~/dev/apache-tomcat-9.0.60/bin :
     46895 Mar  9 07:52 tomcat-juli.jar (original)
    899180 Mar 23 11:11 tomcat-juli.jar (replacement)
    
  3. Merge conf; the two logback configuration files are new; server.xml needs careful merging:
    $ cp logback-access.xml ~/dev/apache-tomcat-9.0.60/conf
    $ cp logback.xml ~/dev/apache-tomcat-9.0.60/conf
    $ diff server.xml ~/dev/apache-tomcat-9.0.60/conf/server.xml
    160c163,165
    <   <Valve className="ch.qos.logback.access.tomcat.LogbackValve" quiet="true" />
    ---
    >   <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
    >          prefix="localhost_access_log" suffix=".txt"
    >          pattern="%h %l %u %t "%r" %s %b" />
    

    You see that we're replacing the standard log valve that ships with the Tomcat distro with one that supports Logback.

  4. Merge lib; these are straight copies since there is no conflict:
    $ cp logback-access-1.2.3.jar ~/dev/apache-tomcat-9.0.60/lib
    $ cp logback-core-1.2.3.jar ~/dev/apache-tomcat-9.0.60/lib
    
  5. Additional steps...
  6. Remove conf/logging.properties; these were for juli. This will turn off juli logging altogether; other web applications using java.util.logging will stop logging.
  7. The final step is simply to run this Tomcat, deploy your application to it and observe the logging behavior.
    ~/dev/apache-tomcat-9.0.60/bin $ ./bin/startup.sh
    ~/dev/apache-tomcat-9.0.60/logs $ tail -f catalina.out
            ...after deploying a web application making TRACE statements
            ...and while using Postman to fire a request off to that web application.
    

Troubleshooting

Emerging Dockerfile:

Aborted since it's too difficult if not impossible to get Tomcat to accommodate the use of Logback from web applications.

# Must use Tomcat 9 because Tomcat 10 doesn't work.
FROM   tomcat:9.0.58-jdk8-openjdk-slim
LABEL  maintainer="[email protected]"
# Special instructions:
ADD    tomcat-logback-sources/tomcat-juli.jar      bin/
ADD    tomcat-logback-sources/setenv.sh            bin/
ADD    tomcat-logback-sources/logback.xml          conf/
ADD    tomcat-logback-sources/logback-access.xml   conf/
ADD    tomcat-logback-sources/server.xml           conf/
ADD    tomcat-logback-sources/logback-core-*.jar   lib/
ADD    tomcat-logback-sources/logback-access-*.jar lib/
RUN    rm -r conf/logging.properties
...
RUN    echo 'alias ll="ls -alg"' >> ~/.bashrc
ADD    "target/mdht-restlet##*.war" /usr/local/tomcat/webapps
EXPOSE 8080
CMD    [ "catalina.sh", "run" ]





First try: Using CampToCamp's Tomcat container with Logback...

Our derivative container worked, but it didn't solve the problem: web applications deployed to Tomcat simply cannot make use of Logback.

Dockerfile:
# Must use Tomcat 9 because Tomcat 10 doesn't work.
FROM   camptocamp/tomcat-logback:9.0-jre11
LABEL  maintainer="[email protected]"
ADD    "target/mdht-restlet##*.war" /usr/local/tomcat/webapps
RUN    echo 'alias ll="ls -alg"' >> ~/.bashrc
EXPOSE 8080
CMD    [ "catalina.sh", "run" ]

In the main console where we run Docker...

$ docker build -t mmmm .
$ docker images
$ docker run --name mdht-restlet2 -p 4040:8080 mmmm

In another console...

$ docker ps
$ docker exec -it mdht-restlet2 /bin/bash
        # Once inside the container:
        # cd webapps
        # cd webapps/mdht-restlet##3.3.6-10/WEB-INF/classes/
        # cat logback.xml
        # No catalina.out because container puts them on stdout in the console

This did not work: no TRACE statements came out. It's unclear, but as I don't have control over the Docker image, I am impeded. CampToCamp may simply have botched it. Let's build our own, besides, CampToCamp made choices I don't like (like to-stdout instead of catalina.out).

Additional links