mikeski.net kotlin java javascript hugo development

mikeski.net development blog

Spring Boot Batch

Spring Boot’s Batch process can be very useful for lots of things. With the flexibility that Spring provides we can make it do anything from ETL to payment processing to planned maintainence.

If you can script it you can automate it with batch. Let’s see a simple example.

Architecture of Spring Batch

Quite simply, Spring Batch consists of 4 main parts:

  • An Item, which is the data we are operating on - in our example we are using a String for the Item. It can be any type of object.
  • An ItemReader which produces items to be processed
  • An ItemProcessor which does something to (and/or with) the item
  • An ItemWriter which takes the processed item for writing

batch diagram

Let’s take a look at an example.

Example service

Our example here will filter profanity from a string. It will simply have a list of words and substitute a non-profane bunch of characters and return the sanitized string.

Item

First, we’ll create our data class with a couple of fields:

data class TextItem(var unfiltered: String, var filtered: String? = null)

ItemProcessor

Next, we’ll create a processor to do the lookup. A processor implements the Spring interface ItemProcessor where K is the input type and L is the output type. We are going to take an unfiltered String as input and output a filtered String, so we will implement ItemProcessor.

Our processor will look like this:

class TextFilteringProcessor: ItemProcessor<String, String> {
    private val badWords = asList("damn", "crap")

    override fun process(textToFilter: String): String {
        var finalValue = textToFilter

        badWords.forEach { badWord ->
            finalValue = finalValue.replace(badWord, "*".repeat(badWord.length), true)
        }
        return finalValue
    }
}

Let’s make sure to write unit tests to make sure it works!

ItemReader

We’ll just create a simple static item generator that returns items from a list. In the real world, we might have a database we are reading, a file we are processing, or some other way of creating items:

class TestFilteringItemReader : ItemReader<String> {
    val items = asList("This has no bad words", "This has one damn bad word")
    var position = 0;

    override fun read(): String? {
        if(position == items.size){
            return null
        }
        return items.get(position++)
    }
}

ItemWriter

For an item writer, we’ll just print to stdout. This makes the implementation very simple, with the caveat that the writers get a list of items so that writing can be done in chunks, if needed:

class TestFilteringItemWriter : ItemWriter<String> {
    override fun write(items: MutableList<out String>) {
        items.forEach { println(it) }
    }
}

Spring Boot Configuration and Application

Now that we have everything setup, we can create the main application and run it. Here is what our main class looks like:

@SpringBootApplication(exclude = [DataSourceAutoConfiguration::class, DataSourceTransactionManagerAutoConfiguration::class, HibernateJpaAutoConfiguration::class])
@EnableBatchProcessing
class TextFilteringSpringBootApplication {

    @Bean
    fun jobLogAllFirewallBluegroups(jbf: JobBuilderFactory, sbf: StepBuilderFactory): Job {
        return jbf.get("demo-text-job").start(
                sbf.get("demo-text-step")
                        .chunk<String, String>(1)
                        .reader(TextFilteringItemReader())
                        .processor(TextFilteringProcessor())
                        .writer(TextFilteringItemWriter())
                        .build()).build()
    }
}

fun main(args: Array<String>) {
    runApplication<TextFilteringSpringBootApplication>(*args)
}

First, @SpringBootApplication(exclude = [ … ]) keeps Spring from trying to initilize database stuff. We do not have a JDBC driver on the classpath and Spring will complain if we try to run it without one unless we disable the database configuration classes

Next, JobBuilderFactory and StepBuilderFactory are helpers to create steps and jobs in the context

Finally, when we run the application we see the following output:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.1.RELEASE)

2019-11-15 19:29:31.138  INFO 7182 --- [           main] d.b.TextFilteringSpringBootApplicationKt : Starting TextFilteringSpringBootApplicationKt on Mikes-MacBook-Pro.local with PID 7182 (/Users/mike/code/public/examples/batch/target/classes started by mike in /Users/mike/code/public/jh-mulchr2)
2019-11-15 19:29:31.146  INFO 7182 --- [           main] d.b.TextFilteringSpringBootApplicationKt : No active profile set, falling back to default profiles: default
2019-11-15 19:29:32.415  INFO 7182 --- [           main] d.b.TextFilteringSpringBootApplicationKt : Started TextFilteringSpringBootApplicationKt in 1.58 seconds (JVM running for 2.327)
2019-11-15 19:29:32.418  INFO 7182 --- [           main] o.s.b.a.b.JobLauncherCommandLineRunner   : Running default command line with: []
2019-11-15 19:29:32.420  WARN 7182 --- [           main] o.s.b.c.c.a.DefaultBatchConfigurer       : No datasource was provided...using a Map based JobRepository
2019-11-15 19:29:32.420  WARN 7182 --- [           main] o.s.b.c.c.a.DefaultBatchConfigurer       : No transaction manager was provided, using a ResourcelessTransactionManager
2019-11-15 19:29:32.440  INFO 7182 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : No TaskExecutor has been set, defaulting to synchronous executor.
2019-11-15 19:29:32.467  INFO 7182 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=demo-text-job]] launched with the following parameters: [{}]
2019-11-15 19:29:32.499  INFO 7182 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [demo-text-step]
This has no bad words
This has one **** bad word
2019-11-15 19:29:32.520  INFO 7182 --- [           main] o.s.batch.core.step.AbstractStep         : Step: [demo-text-step] executed in 20ms
2019-11-15 19:29:32.523  INFO 7182 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=demo-text-job]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 32ms

Notably, the standard out messages show that our batch job has successfully run:

This has no bad words This has one **** bad word

Conclusion

In this article, we saw how to use Spring Boot’s batch capabilities to setup a simple batch processing job.