NiFi templating notes

Russell Bateman
April 2023
last update:


I don't generally write flows, but only custom processors. However, from time to time I'm asked to provide templates that demonstrate how to use my custom processors. Step by step, here's how I go about it. I used Apache NiFi 1.13.2 as I wrote the following methodology.

Table of Contents

Instructions on creating a flow to become a template
Create a template
Download a template (save to the filesystem for distributing)
Upload a template (from the filesystem to your NiFi UI canvas)
Use a template (in your flow on the NiFi UI canvas)

Instructions on creating a flow to become a template...

  1. This step supposes that you already know how to work with the NiFi UI canvas and create flows that do something.
  2. I prefer to design a flow inside a dedicated process group, then make a template of that process group. If you do this, you will need to ensure either an Input Port or an Output Port to begin and/or end the flow, usually both, to get flowfiles into the templated flow and/or out of it.
  3. Create a new process group with a descriptive name, create a a flow, then test it.
  4. Once I'm happy with the result, I erase the non-prescribable configuration of the processors and controllers in the flow. (This is why the thing I'm creating is a "template" that can serve in other flows.)
  5. In the template we create later, it might not be totally obvious what the user of your template needs to do. I use labels to help thing along. In a label of default color (#FFF7D7, see below) near each processor needing configuration, list the properties of that processor (and any controller to which it subscribes) that must be configured to get it working. For example (JdbcToXml): 
    Left to do:
- DBCPConnectionPool configure
- Adjust Base SQL Query if needed

DBCPConnectionPool
- Fix Database Connection URL
- Fix Database User
- Fix Password
  6. I also like to put  a big, green (#4CBB17) label   as a background under the relationship arcs and processor flows that represent successful outcomes and  a big, red (#E34234) label   where failures are taken care of in my flow. It imparts a certain clarity. Use your imagination here—or don't.
  7. Leave the processor group, i.e.: go "upstairs" where our template will be created.

Create a template...

  1. If the name of the template you're about to create is already in use, consider deleting the template you want to replace. You cannot create a new or updated template with a name that conflicts with an existing one.
    1. Click the General (or "hamburger") menu.
    2. Select Templates.
    3. Find and select the template whose name you wish to use.
    4. Click the Remove Template (trashcan icon).
    5. Close the NiFi Templates dialog by click the X button.
  2. Create a template of the process group.
    1. Open the Operate palette.
    2. Select the process group to become a template.
    3. Click the Create Template icon on the Operate palette.
    4. Name the template.
      Note: consider prefixing with a number to aid flow writer with the relative order if this template is part of a greater collection. For example, if the second thing a flow does is to implements importing data from Mirth, name the template that performs this, 2-mirth-flow.
    5. In Description, describe what the template implements.
      In the example immediately above, this might be: JdbcToXml from Mirth, excerpt document, note its type and hash it ready for ETL..
    6. Click OK.
  3. Verify the template you've just created by:
    1. Clicking the General (or "hamburger") menu.
    2. Select Templates.
    3. Finding the template just created.
    4. Examining the Name and Description.
    5. Close the NiFi Templates dialog by click the X button.

Download a template (saving to the filesystem for distributing)...

  1. In the UI canvas containing any created template:
    1. Click the General (or "hamburger") menu.
    2. Select Templates.
    3. Find the template you wish to use based on Name or Description.
    4. Click the Download icon and save the resulting XML file in the place from which you'd like to distribute it.
    5. Close the NiFi Templates dialog by click the X button.
  2. You can distribute the template as a simple XML file for yourself or others to upload and use.

Upload a template (from the filesystem to your NiFi UI canvas)...

  1. In the UI canvas where you need a template that exists in your filesystem, do this:
    1. Open the Operate palette.
    2. Click on the Upload Templates.
    3. In the Upload Template dialog, click the Select Template button (a magnifying glass).
    4. Navigate to find the XML file in your filesystem that is the template you wish to use. Select it and click Open.
    5. Back in the Upload Template dialog, click the Upload button.
    6. Close the NiFi Templates dialog by click the X button.
    7. This enters the chosen template into the list of templates to which the flow you're writing has access.

Use a template (in your flow on the NiFi UI canvas)...

  1. In the UI canvas where you are implementing your flow and wish to use an existing template:
    1. Click and drag an instance of the Template tool, in the Components Toolbar at the top of the UI canvas window to the place on the cavas where you want to deploy the templated flow.
    2. In the Add Template dialog that comes up, select the template you wish to us from the Choose Template drop-down list.
    3. If you have already used this template elsewhere on your "visible" canvas (that is, at the current processor group level), you'll see the template name prefixed with "Copy of" and probably want to change that to a more specific description. (Of course, this hint depends entirely on what is in the template. If it's a process group—how I like to do templates—then what I have just said is relevant. However, if it's a group or pile of loose processors, then there will be no "name" or "prefixed name.")