Skip to main content

Getting Started

Welcome to Spring Batch RS! This guide will help you get up and running with batch processing in Rust.

What is Spring Batch RS?​

Spring Batch RS is a comprehensive toolkit for building enterprise-grade batch applications in Rust. Inspired by the robust Java Spring Batch framework, it brings battle-tested concepts to the Rust ecosystem with the added benefits of Rust's performance and memory safety.

Prerequisites​

Before you begin, ensure you have:

  • Rust 1.70+ installed (Install Rust)
  • Basic familiarity with Rust programming
  • Understanding of batch processing concepts (helpful but not required)

Installation​

Add Spring Batch RS to your Cargo.toml:

[dependencies]
spring-batch-rs = "0.3"

Feature Flags​

Spring Batch RS uses feature flags to keep your dependencies minimal. Enable only what you need:

[dependencies]
spring-batch-rs = { version = "0.3", features = ["csv", "json", "xml"] }

Available features:

FeatureDescription
csvCSV file reading and writing
jsonJSON file reading and writing
xmlXML file reading and writing
mongodbMongoDB database integration
rdbc-postgresPostgreSQL database integration
rdbc-mysqlMySQL/MariaDB database integration
rdbc-sqliteSQLite database integration
ormSeaORM integration
zipZIP compression tasklets
ftpFTP and FTPS file transfer tasklets
fakeMock data generation
loggerDebug logging writer
fullAll features enabled

Your First Batch Job​

Let's create a simple batch job that reads CSV data and converts it to JSON:

1. Create a New Project​

cargo new my-batch-app
cd my-batch-app

2. Add Dependencies​

[dependencies]
spring-batch-rs = { version = "0.3", features = ["csv", "json"] }
serde = { version = "1.0", features = ["derive"] }

3. Define Your Data Structure​

use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Deserialize, Serialize)]
struct Product {
    id: u32,
    name: String,
    price: f64,
    category: String,
}

4. Create Your First Job​

use spring_batch_rs::{
    core::{job::JobBuilder, step::StepBuilder, item::PassThroughProcessor},
    item::{csv::CsvItemReaderBuilder, json::JsonItemWriterBuilder},
    BatchError,
};

fn main() -> Result<(), BatchError> {
    // Sample CSV data
    let csv_data = r#"id,name,price,category
1,Laptop,999.99,Electronics
2,Coffee Mug,12.99,Kitchen
3,Notebook,5.99,Office
4,Wireless Mouse,29.99,Electronics"#;

    // Create CSV reader
    let reader = CsvItemReaderBuilder::<Product>::new()
        .has_headers(true)
        .from_reader(csv_data.as_bytes());

    // Create JSON writer
    let writer = JsonItemWriterBuilder::new()
        .pretty_formatter(true)
        .from_path("products.json")?;

    // Create processor (pass-through in this case)
    let processor = PassThroughProcessor::<Product>::new();

    // Build the step
    let step = StepBuilder::new("csv-to-json-step")
        .chunk(10)  // Process 10 items at a time
        .reader(&reader)
        .processor(&processor)
        .writer(&writer)
        .build();

    // Build and run the job
    let job = JobBuilder::new()
        .start(&step)
        .build();

    // Execute the job
    let result = job.run()?;

    println!("Job completed successfully!");
    println!("Processed {} items", result.get_step_executions().len());

    Ok(())
}

5. Run Your Job​

cargo run

You should see output similar to:

Job completed successfully!
Processed 1 items

And a products.json file will be created with your converted data.

Core Concepts​

Understanding these core concepts will help you build more complex batch applications:

Job​

A Job represents the entire batch process. It's composed of one or more steps that execute in sequence.

let job = JobBuilder::new()
    .start(&step1)
    .next(&step2)
    .next(&step3)
    .build();

Step​

A Step is an independent phase of a batch job. There are two types:

  1. Chunk-oriented steps: Read-process-write pattern for large datasets
  2. Tasklet steps: Single operations like file transfers or cleanup
// Chunk-oriented step
let chunk_step = StepBuilder::new("process-data")
    .chunk(100)
    .reader(&reader)
    .processor(&processor)
    .writer(&writer)
    .build();

// Tasklet step
let tasklet_step = StepBuilder::new("cleanup")
    .tasklet(&cleanup_tasklet)
    .build();

ItemReader​

An ItemReader retrieves input data one item at a time:

// CSV reader
let csv_reader = CsvItemReaderBuilder::<Product>::new()
    .from_path("input.csv")?;

// Database reader
let db_reader = OrmItemReaderBuilder::<Product>::new()
    .connection(&db)
    .query(Product::find())
    .page_size(100)
    .build();

ItemProcessor​

An ItemProcessor applies business logic to transform items:

use spring_batch_rs::core::item::ItemProcessor;

struct PriceProcessor;

impl ItemProcessor<Product, Product> for PriceProcessor {
    fn process(&self, item: Product) -> Result<Option<Product>, BatchError> {
        let mut product = item;
        // Apply 10% discount
        product.price *= 0.9;
        Ok(Some(product))
    }
}

ItemWriter​

An ItemWriter outputs processed items:

// JSON writer
let json_writer = JsonItemWriterBuilder::new()
    .from_path("output.json")?;

// Database writer
let db_writer = OrmItemWriterBuilder::<Product>::new()
    .connection(&db)
    .build();

Error Handling​

Spring Batch RS provides robust error handling with configurable skip limits:

let step = StepBuilder::new("fault-tolerant-step")
    .chunk(100)
    .reader(&reader)
    .processor(&processor)
    .writer(&writer)
    .skip_limit(10)  // Skip up to 10 errors
    .build();

Next Steps​

Now that you have the basics down, explore more advanced features:

  1. Processing Models - Learn about chunk-oriented vs tasklet processing
  2. Item Readers & Writers - Explore all available data sources
  3. Tasklets - File operations, FTP/FTPS transfers, and custom tasks
  4. Examples - Real-world examples and patterns
  5. Tutorials - Step-by-step guides for common scenarios

Need Help?​

Happy batch processing! 🚀