Black box testing is testing without having access to the source code (or by not looking at it). This fits well in with test-driven development.

White box testing is examining the source code (of the subject under test) as tests are written. This ensures that tests collectively cover all of the paths any given action might take with an eye to watching boundaries of loops, respecting all logical conditions, etc.

Testing state refers to performing an action in a test case. then verifying, probably by assertion, that an expected result was achieved (or not achieved. Doing this sort of thing can certainly be part (an objective of) black- or white-box testing.

Writing JUnit tests amounts to getting everything set up for a test case (arranging), executing the action under test) (acting) and verifying that the action completed correctly (asserting).

Advice

• Keep assertions to a minimum; create more test cases rather than "testing the fool" out of something. This makes the test cases more useful, Lego&tm;-like and less brittle (easier to maintain).
• Assertionless test cases should probably be avoided or, at least, commented upon such that it's clear they only exist to demonstrate code functionality, approach taken, etc. as if documentary in nature.
• Don't repeat like-assertions over and over again.
• Don't collect assertion statements together in utility methods to be called from test cases themselves. This obfuscate the meaning of the test case.

Here's how to wire up a JUnit library to your Eclipse project, it's pretty simple to do:

1. Right-click your project.
2. Choose Build Path -> Configure Build Path...
3. Click the Libraries tab.
4. Click Add Library...
5. Select JUnit.
6. Click Next.
7. Select version and click Finish.

At your project root, create a new folder named test.

1. Right-click it, choose Build Path, then Use as Source Folder.
2. Create a new package matching the package name in your project/src folder whose class(es) you want to test.

To Create a new JUnit test class:

1. Right-click the new test/package and choose JUnit Test Case (probably only available if you added the JUnit library in the steps above).
2. Follow the wizard as should be obvious.

Is twofold: a) writing test cases and b) writing a test suite.

Automatic test generation

If using Eclipse, a test case for any class may be generated automatically with only the need to fill out stubs. The stubs are initially set up to generate failures, which concords with test-driven development (TDD) theory (everything should fail initially and implementation consists of eliminating the failures).

To set up a test case, locate the folder/package dominating the location the test source code will be place and right-click. Choose New->JUnit Test Case. Enter the name of the package (if it's wrong), the name of the test case (as the name of the class to test followed by “Test”), then click setUp() and tearDown() unless you don't want these for some reason, and last, the name of the class you're testing. Then, click next.

At this point, you're offered automatic code generation of those methods you select. Click Finish. You also have the option to create final method stubs and to create Eclipse tasks to remind you to finish implementing the test methods that are about to be generated.

Writing the test methods...

The test cases are written thus:

  package test-location;

  import junit.framework.TestCase;

  public class class-nameTest extends TestCase
  {
    protected void setUp()
    {
      // Set up data and objects required for use in any test case. Usually,
      // these would be instance variables.
    }

    protected void tearDown()
    {
      // Release objects set up above.
    }

    public void testmethod-name()        <------ this is a test case
    {
      assertEquals( expected, actual );
    }
  }

Let's make a useful if bogus instantiation of the naming concepts above in order for later, TestSuite sample code to be of any use.

  package com.myapp.utils;

  import junit.framework.TestCase;

  public class FunAndGamesTest extends TestCase
  {
    protected void setUp()
    {
      // Set up data and objects required for use in any test case. Usually,
      // these would be instance variables.
    }

    protected void tearDown()
    {
      // Release objects set up above.
    }

    public void testRobBank()
    {
      assertEquals( expected, actual );
    }

    public void testLivingTheVidaLoca()
    {
      assertEquals( expected, actual );
    }
  }

Let's make a second sample test class for the purposes of later demonstration. We'll even add just enough real code to demonstrate this undertaking further than FunAndGamesTest did.

  package com.myapp.utils;

  import junit.framework.TestCase;

  public class AllWorkAndNoPlayTest extends TestCase
  {
    protected void setUp()
    {
      AllWorkAndNoPlay  today = new AllWorkAndNoPlay();

      System.out.println( "AllWorkAndNoPlay set-up" );
    }

    protected void tearDown()
    {
      System.out.println( "AllWorkAndNoPlay tear-down" );
    }

    public void testMakeADullBoy()
    {
      today.MakeADullBoy( true );
      assertEquals( true, today.boyIsDull );
    }
  }

Invoke the tests for all classes in a package, or for all classes contributing to an important, identifiable functionality, using a TestSuite. In our sample template here, we're assuming that the package consists of classes of a common functionality all of which we'll test in this suite. Note that, unlike the previous code implementing tests of methods in a single class, the variable part of the TestSuite is ONLY the tests we add.

Assume that in com.myapp.utils, we create UtilsTestSuite to run all the individual class tests. We will use addTestSuite to do this.

  package com.myapp.utils;

  import junit.framework.Test;
  import junit.framework.TestSuite;


  public class UtilsTestSuite
  {
    public static Test suite()
    {
      TestSuite suite = new TestSuite( "Utilities tests" );

      suite.addTestSuite( FunAndGamesTest.class );
      suite.addTestSuite( AllWorkAndNoPlayTest.class );

      return suite;
    }

    public static void main( String[] args )
    {
      junit.textui.TestRunner.run( suite() );
    }
  }

Note the distinction between addTestSuite, which adds a test to the suite being defined in this class, and later, addTest, which adds test suites. This is a distinction that the first tutorial linked above did poorly in its test code. Indeed, none of the tutorials accurately addressed this distinction if they mentioned addTestSuite at all.

addTestSuite is a misnomer. You don't use it to add a TestSuite, but to add a single class as a TestSuite (see class UtilsTestSuite above). It's a shortcut for: addTest( new TestSuite( class-nameTest.class ) ). In our case here, the code is the following and we won't even use addTestSuite. I think it's less confusing to do it this way (and never use addTestSuite).

So, above the package leaf-node level collective test, UtilsTestSuite, we'll constitute successively higher collective tests. This one might, if the MyApp is simple enough, have been named AllTests.java. (In the discussion of hierarchy below, we show a more complex application and suggest the names of successively higher levels.)

  import junit.framework.Test;
  import junit.framework.TestSuite;

  public class MyAppTestSuite
  {
    public static Test suite()
    {
      TestSuite suite = new TestSuite( "Tests for all of MyApp" );

      suite.addTest( UtilsTestSuite.suite() );
      //suite.addTest( AnotherTestSuite.suite() ); *

      return suite;
    }

    public static void main(String[] args)
    {
      junit.textui.TestRunner.run( suite() );
    }
  }

* Now, I know this is the same cop-out in the existing tutorials, but here I have add the paragraph clarifying addTest and addTestSuite which they did not do. This oversight led to initial confusion about how best to add test suites to successively higher test suites (which is not done, at least by us, using addTestSuite since its name only misleads you to thinking this is what to use.

It appears that under the Eclipse debugger, at least, you can add a new test method to the whole, but it will execute after the previous ones. To add one in front and get it to execute first, just suck out all the others, run the new method, then paste back in the old stuff you copied. The new one will then execute first.

(I haven't experimented overly with this.)

Beginning in JUnit v4.11, the following annotations influence the order. Ordinarily, however, it's somewhat perverse to insist upon the order and, should test cases develop a behavioral dependency on run order, this would absolutely indict the wisdom of the developer coding the tests. Order should make no difference other, perhaps, than for aesthetics, for example, if you wanted to see an ordered list appear at the console for a set of test cases because it imparts a certain documentation order on a reduced number of test cases or is pleasing to the eye.

import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.FixMethodOrder;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;
import org.junit.runners.MethodSorters;

@FixMethodOrder( MethodSorters.JVM )
//@FixMethodOrder( MethodSorters.NAME_ASCENDING )
//@FixMethodOrder( MethodSorters.DEFAULT )
public class TestFixMethodOrder
{
  // @formatter:off
  @Rule        public TestName name = new TestName();
  @Before      public void setUp() { System.out.print( "  " + name.getMethodName() + "..." ); }
  @BeforeClass public static void beforeClass() { System.out.print( "MethodSorters.JVM--coded order:" ); }
//@BeforeClass public static void beforeClass() { System.out.print( "MethodSorters.NAME_ASCENDING--alphabetic order:" ); }
//@BeforeClass public static void beforeClass() { System.out.print( "MethodSorters.DEFAULT--random order:" );
}

  @Test public void testZoolander()   { }
  @Test public void testMightyMouse() { }
  @Test public void testSnagglepuss() { }
  @Test public void testAntMan()      { }
}

Output (consider that you run this test suite three times exchanging commented out lines above as you may imagine):

MethodSorters.JVM--coded order:
  testZoolander...
  testMightyMouse...
  testSnagglepuss...
  testAntMan...

MethodSorters.NAME_ASCENDING--alphabetic order:
  testAntMan...
  testMightyMouse...
  testSnagglepuss...
  testZoolander...

MethodSorters.DEFAULT--random order:
  testZoolander...
  testMightyMouse...
  testSnagglepuss...
  testAntMan...

Of course, this doesn't really work. It only appears some times to work.

If you don't want to set up and tear down between each test method, you can create methods to perform the set-up and tear-down, call the set-up from at the beginning of the first test and the tear-down at the end of the last test methods. In this case, make setUp and tearDown more or less stubs.

For example, here is the proposed JUnit test hierarchy for a product called MyApp that consists of a server application, worker application and common library projects. Assume the developing company is named myapp.com. using packages such as com.myapp.archive, etc.

There are additional notions illustrated here, like the smoke test, that will be defined afterward. As there are identical package paths in common between MyAppServer, MyAppWorker and MyAppLib (at least), conventions such as ServerClassesTestSuite become necessary.

  (Application projects...)
    MyAppServer                      MyAppWorker
      AllTests (top-level for MyApp Server)          AllTests (top-level for MyApp Worker)
        SmokeTestSuite (structural integrity)          SmokeTestSuite (structural integrity)
          ServerClassesTestSuite                         WorkerClassesTestSuite
            AddressTestSuite                               FiltersTestSuite
              ABSearchTest                                   MainFilterTest
              ABTreeTest                                     QrtzInitializeServletTest
            ArchiveTestSuite                                 QrtzInitializeServletATest
              HcapArchiveTest                                .
            AuthenticateTestSuite                            .
              LDAPTestSuite                                  .
                BlindSSLSocketFactoryTest
                LDAPParamsTest
                LDAPUtilTest
              AbstractGroupWiseAuthenticatorTest
              .
              .
              .
              WorkerAuthenticationMethodTest
            CachingTestSuite
              ABCacheTest
              IndexHandlerCacheTest
              .
              .
              .
              WorkerPasswordCacheTest
            CfgTestSuite
            .
            .
            .
          ContractorClassesTestSuite
            ArchiveTestSuite
              ArchiveFileTest
              ArchiveFileBaseTest
              .
              .
              .
              XMLFileTest
            CfgTestSuite
            .
            .
            .

  (Oblique projects...)
    MyAppLib        EasySoap        SharedJava
      AllTests               AllTests                AllTests
        .                      .                       .
        .                      .                       .
        .                      .                       .

This stratagem adds 1) one test class for every implementation class written, 2) one test suite class for every package's leaf and intermediate nodes, and 3) one AllTests.java and one SmokeTestSuite.java for every project. However, there is nearly no work at all to any of these classes except for the individual class test.

Eclipse Maven support is reputedly less than full or ideal. Maven could be used to generate all these classes except for the individual class tests.

Smoke is testing that ensures the basic, continued stability of the application code by revealing serious flaws in very short order. In theory, failing smoke testing is tantamount to rejecting a release candidate with no need to spend more time or other resources on validating it. “Smoke and build” refers to applying and passing smoke tests, then performing the full build of a product's ISO or other distributive means for the real validation testing performed by a Quality Assurance team.

I remain uncertain as to what extent AllTests.java and SmokeTestSuite.java differ.

...does not exist!” (an alert you get when you attempt to Run/Debug a JUnit test in a project whose test subdirectory has not been qualified as part of the source code. The easiest way to solve this is to right-click on the folder, then choose Build Path → Use as Source Folder.

1. Right-click on the project name. Choose New->Source Folder.
2. Name the new folder test and confirm its creation.
3. Click that folder and create a new package path choosing something that aligns with the packages in the src folder you are testing.
4. Click on the new package just created and choose New->JUnit Test Case. If this is the first one you've created, there will be an opportunity to add the JUnit library JAR to the project. You must do this.

Starting with Java 5, JUnit 4 is possible on account of the use of annotations. JUnit 4 test-writing is, I think, superior in flexibility and easier than JUnit 3. Without deep-ending on the differences of JUnit 4, here's a succinct demonstration:

package com.etretatlogiciels.junit;

import static org.junit.Assert.assertEquals;

import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;

public class MyClassTest
{
  @Test
  public void testMultiply()
  {
    MyClass  tester = new MyClass();
    assertEquals( "Result", 50, tester.multiply( 10, 5 ) );
  }

  @Before public void beforeEach()
  {
    System.out.println( "beforeEach() test..." );
  }

  @After public void afterEach()
  {
    System.out.println( "afterEach() test..." );
  }

  @BeforeClass public static void beforeAll()
  {
    System.out.println( "beforeAll() tests..." );
  }

  @AfterClass public static void afterAll()
  {
    System.out.println( "afterAll() tests..." );
  }
}

These tests run yielding the following output at the console, adequately demonstrating the function of the annotations:

	beforeAll() tests...
	beforeEach() test...
	afterEach() test...
	afterAll() tests...