Spring Boot Batch
Friday Nov 15, 2019
Spring Boot’s Batch process can be very useful for lots of things. With the flexibility that Spring provides we can make it do anything from ETL to payment processing to planned maintainence.
If you can script it you can automate it with batch. Let’s see a simple example.
Architecture of Spring Batch
Quite simply, Spring Batch consists of 4 main parts:
- An Item, which is the data we are operating on - in our example we are using a String for the Item. It can be any type of object.
- An ItemReader which produces items to be processed
- An ItemProcessor which does something to (and/or with) the item
- An ItemWriter which takes the processed item for writing
Let’s take a look at an example.
Example service
Our example here will filter profanity from a string. It will simply have a list of words and substitute a non-profane bunch of characters and return the sanitized string.
Item
First, we’ll create our data class with a couple of fields:
data class TextItem(var unfiltered: String, var filtered: String? = null)ItemProcessor
Next, we’ll create a processor to do the lookup. A processor implements the Spring interface ItemProcessor
Our processor will look like this:
class TextFilteringProcessor: ItemProcessor<String, String> {
private val badWords = asList("damn", "crap")
override fun process(textToFilter: String): String {
var finalValue = textToFilter
badWords.forEach { badWord ->
finalValue = finalValue.replace(badWord, "*".repeat(badWord.length), true)
}
return finalValue
}
}Let’s make sure to write unit tests to make sure it works!
ItemReader
We’ll just create a simple static item generator that returns items from a list. In the real world, we might have a database we are reading, a file we are processing, or some other way of creating items:
class TestFilteringItemReader : ItemReader<String> {
val items = asList("This has no bad words", "This has one damn bad word")
var position = 0;
override fun read(): String? {
if(position == items.size){
return null
}
return items.get(position++)
}
}ItemWriter
For an item writer, we’ll just print to stdout. This makes the implementation very simple, with the caveat that the writers get a list of items so that writing can be done in chunks, if needed:
class TestFilteringItemWriter : ItemWriter<String> {
override fun write(items: MutableList<out String>) {
items.forEach { println(it) }
}
}Spring Boot Configuration and Application
Now that we have everything setup, we can create the main application and run it. Here is what our main class looks like:
@SpringBootApplication(exclude = [DataSourceAutoConfiguration::class, DataSourceTransactionManagerAutoConfiguration::class, HibernateJpaAutoConfiguration::class])
@EnableBatchProcessing
class TextFilteringSpringBootApplication {
@Bean(name=["Batch example 1"])
fun jobLogAllFilteredTextItems(jbf: JobBuilderFactory, sbf: StepBuilderFactory): Job {
return jbf.get("demo-text-job1").start(
sbf.get("demo-text-step1")
.chunk<String, String>(1)
.reader(TextFilteringItemReader())
.processor(TextFilteringProcessor())
.writer(TextFilteringItemWriter())
.build()).build()
}
}
fun main(args: Array<String>) {
runApplication<TextFilteringSpringBootApplication>(*args)
}First, @SpringBootApplication(exclude = [ … ]) keeps Spring from trying to initilize database stuff. We do not have a JDBC driver on the classpath and Spring will complain if we try to run it without one unless we disable the database configuration classes
Next, JobBuilderFactory and StepBuilderFactory are helpers to create steps and jobs in the context
Finally, when we run the application we see the following output:
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.2.1.RELEASE)
2019-11-15 19:29:31.138 INFO 7182 --- [ main] d.b.TextFilteringSpringBootApplicationKt : Starting TextFilteringSpringBootApplicationKt on Mikes-MacBook-Pro.local with PID 7182 (/Users/mike/code/public/examples/batch/target/classes started by mike in /Users/mike/code/public/jh-mulchr2)
2019-11-15 19:29:31.146 INFO 7182 --- [ main] d.b.TextFilteringSpringBootApplicationKt : No active profile set, falling back to default profiles: default
2019-11-15 19:29:32.415 INFO 7182 --- [ main] d.b.TextFilteringSpringBootApplicationKt : Started TextFilteringSpringBootApplicationKt in 1.58 seconds (JVM running for 2.327)
2019-11-15 19:29:32.418 INFO 7182 --- [ main] o.s.b.a.b.JobLauncherCommandLineRunner : Running default command line with: []
2019-11-15 19:29:32.420 WARN 7182 --- [ main] o.s.b.c.c.a.DefaultBatchConfigurer : No datasource was provided...using a Map based JobRepository
2019-11-15 19:29:32.420 WARN 7182 --- [ main] o.s.b.c.c.a.DefaultBatchConfigurer : No transaction manager was provided, using a ResourcelessTransactionManager
2019-11-15 19:29:32.440 INFO 7182 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2019-11-15 19:29:32.467 INFO 7182 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=demo-text-job]] launched with the following parameters: [{}]
2019-11-15 19:29:32.499 INFO 7182 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [demo-text-step]
This has no bad words
This has one **** bad word
2019-11-15 19:29:32.520 INFO 7182 --- [ main] o.s.batch.core.step.AbstractStep : Step: [demo-text-step] executed in 20ms
2019-11-15 19:29:32.523 INFO 7182 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=demo-text-job]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 32ms
Notably, the standard out messages show that our batch job has successfully run:
This has no bad words This has one **** bad word
Conclusion
In this article, we saw how to use Spring Boot’s batch capabilities to setup a simple batch processing job.