1. Introduction

In this tutorial, we’ll demonstrate different ways to read and write CSV files in Scala. Of course, our demonstration wouldn’t be complete if we didn’t explore some of the most popular libraries that provide CSV read and write capabilities.

Hence, we included sections for Scala CSV, Open CSV, and Apache Commons CSV libraries.

2. Building Blocks

To make our examples comparable and easier to work with, let’s define two Scala traits that every CSV library will implement, namely the CommaSeparatedValuesWriter and the CommaSeparatedValuesReader.

Specifically, the CommaSeparatedValuesWriter defines a method that yields a convenient CSV file digest:

trait CommaSeparatedValuesReader {
  def read(file: File): CSVReadDigest
}

The CSVReadDigest, which is the return type of the read function, contains headers and rows as properties:

case class CSVReadDigest(headers: Seq[String], rows: Seq[Seq[String]])

Equally with the CommaSeparatedValuesWriter, the CommaSeparatedValuesWrites defines a method that writes headers and rows to a given file:

trait CommaSeparatedValuesWriter {
  def write(
    file: File,
    headers: Seq[String],
    rows: Seq[Seq[String]]
  ): Try[Unit]
}

3. PrintWriter and BufferedReader

Before we import any external libraries, it’s useful to showcase how we can write and read a CSV file using readily available Java tools, the PrintWriter and BufferedReader.

3.1. Write

The SimpleCSVWriter prints the file contents by iterating through the input data and appending them to the underlying PrintWriter:

class SimpleCSVWriter extends CommaSeparatedValuesWriter {
  override def write(
    file: File,
    headers: Seq[String],
    rows: Seq[Seq[String]]
  ): Try[Unit] = Try {
    val writer = new PrintWriter(file)
    writer.println(headers.mkString(","))
    rows.foreach(row => writer.println(row.mkString(",")))
    writer.close()
  }
}

3.2. Read

The SimpleCSVReader reads a file’s contents by leveraging the BufferedReader interface. Additionally, the file contents are exhausted by recursively calling the readLinesRecursively method, which invokes the BufferedReader readline method for each line until the end of the file is reached:

class SimpleCSVReader extends CommaSeparatedValuesReader {
  override def read(file: File): CSVReadDigest = {
    val in = new InputStreamReader(new FileInputStream(file))
    val bufferedReader = new BufferedReader(in)
    @tailrec
    def readLinesRecursively(
      currentBufferedReader: BufferedReader,
      result: Seq[Seq[String]]
    ): Seq[Seq[String]] = {
      currentBufferedReader.readLine() match {
        case null => result
        case line =>
          readLinesRecursively(
            currentBufferedReader,
            result :+ line.split(",").toSeq
          )
      }
    }
    val csvLines = readLinesRecursively(bufferedReader, List())
    bufferedReader.close()
    CSVReadDigest(
      csvLines.head,
      csvLines.tail
    )
  }
}

4. Scala CSV

Scala CSV is a Scala library that provides traits with methods that accept and return Scala data structures making CSV handling less cumbersome since no Java-to-Scala conversions are needed.

4.1. Write

The library’s trait that provides CSV read capabilities is the CSVWriter. The ScalaCSVWriter wraps the CSVWriter which trivializes the task of writing lines to a CSV file by exposing functions that accept Seq[Any] arguments:

class ScalaCSVWriter extends CommaSeparatedValuesWriter {
  override def write(
    file: File,
    headers: Seq[String],
    rows: Seq[Seq[String]]
  ): Try[Unit] = Try {
    val writer = CSVWriter.open(file)
    writer.writeRow(headers)
    writer.writeAll(rows)
    writer.close()
  }
}

4.2. Read

Likewise, reading a file with ScalaCSV CSVReader is straightforward. The reader’s method all returns the file as a List[List[String]] hence the ScalaCSVReader wrapper implementation is quite short:

class ScalaCSVReader extends CommaSeparatedValuesReader {
  override def read(file: File): CSVReadDigest = {
    val reader = CSVReader.open(file)
    val all = reader.all()
    reader.close()
    CSVReadDigest(all.head, all.tail)
  }
}

5. OpenCSV

OpenCSV is a popular and widely used Java library for reading and writing CSV files thus we’ll proceed to create our own Scala wrapper implementations to showcase it.

5.1. Write

To use the OpenCSV CSVWriter’s interface writeAll method, our input must be first transformed to a Java Iterable. Let’s write the OpenCSVWriter:

class OpenCSVWriter extends CommaSeparatedValuesWriter {
  override def write(
    file: File,
    headers: Seq[String],
    rows: Seq[Seq[String]]
  ): Try[Unit] = Try(
    new CSVWriter(new BufferedWriter(new FileWriter(file)))
  ).flatMap((csvWriter: CSVWriter) =>
    Try {
      csvWriter.writeAll(
        (headers +: rows).map(_.toArray).asJava,
        false
      )
      csvWriter.close()
    }
  )
}

Furthermore, the OpenCSV CSVWriter interface includes methods that accept java.sql.ResultSet arguments, thus making it extremely useful when dealing with data fetched from databases.

5.2. Read

Similar to the SimpleCSVReader, the OpenCSVReader uses recursion to read a file’s contents. The recursive method readLinesRecursively returns the CSV rows by using the readNext CSVReader‘s function as an iterator:

class OpenCSVReader extends CommaSeparatedValuesReader {
  override def read(file: File): CSVReadDigest = {
    val reader = new CSVReader(
      new InputStreamReader(new FileInputStream(file))
    )
    @tailrec
    def readLinesRecursively(
      currentReader: CSVReader,
      result: Seq[Seq[String]]
    ): Seq[Seq[String]] = {
      currentReader.readNext() match {
        case null => result
        case line => readLinesRecursively(currentReader, result :+ line.toSeq)
      }
    }
    val csvLines = readLinesRecursively(reader, List())
    reader.close()
    CSVReadDigest(
      csvLines.head,
      csvLines.tail
    )
  }
}

6. Apache Commons CSV

The Apache Commons CSV library enables us to read and write CSV files in various formats. It’s been evolving since 2014 and is widely used, mainly by Java projects.

6.1. Write

In contrast with the other implementations we’ve provided in our examples, the Apache Commons CSVWriter‘s format is configured using a second argument, the CSVFormat. The CSVFormat builder allows us to choose a format and configure it by overriding any property that suits us. In our example, namely the ApacheCommonsCSVWriter, we override the headers property of the default CSVFormat:

class ApacheCommonsCSVWriter extends CommaSeparatedValuesWriter {
  override def write(
    file: File,
    headers: Seq[String],
    rows: Seq[Seq[String]]
  ): Try[Unit] = Try {
    val csvFormat = CSVFormat.DEFAULT
      .builder()
      .setHeader(headers: _*)
      .build()
    val out = new FileWriter(file)
    val printer = new CSVPrinter(out, csvFormat)
    rows.foreach(row => printer.printRecord(row: _*))
    printer.close()
  }
}

6.2. Read

The CSVFormat is also used for CSV parsing but in a different way. Let’s notice that in our implementation, the empty setHeader method call is the Apache Commons CSV way of telling the parser to automatically parse the headers from the first line of the file. The call to the parse function of the CSVFormat yields the CSVParser interface, which provides a variety of methods for reading the lines of the input file. So, let’s write our example without forgetting that the returned objects need Java-to-Scala transformations:

class ApacheCommonsCSVReader extends CommaSeparatedValuesReader {
  override def read(file: File): CSVReadDigest = {
    val in = new InputStreamReader(new FileInputStream(file))
    val csvParser = CSVFormat.DEFAULT
      .builder()
      .setHeader()
      .build()
      .parse(in)
    val result = CSVReadDigest(
      csvParser.getHeaderNames.asScala.toSeq,
      csvParser.getRecords.asScala.map(r => r.values().toSeq).toSeq
    )
    csvParser.close()
    result
  }
}

7. CSV Delimiters

Before we conclude our short tutorial, it’s crucial to mention the most frequent issue encountered in a CSV dataset is the comma presence in the delimited values leading to parse errors and wrong results.

A good compromise is the choice of a delimiter that is very unlikely to be present in our data such as a special character or a complex sequence of characters. Alternatively, some use quotes but then quotes must be escaped as well.

To conclude, we suggest that we should have a good knowledge of the data contents of our files before choosing a delimiter for our dataset.

8. Conclusion

In this short tutorial, we demonstrated how to read and write CSV files with Scala.

Additionally, we included some examples with Scala and Java libraries for good measure. In the end, we emphasized the common delimiter issue that developers usually have to provide a solution for.

     

Related Stories

• Different Ways to Format Numbers in Scala
• IOApp in Cats Effect
• Remove the First n Digits of a Number in Scala

 

1. Overview

Gatling is a load-testing framework mainly used to test REST APIs. It’s designed to be easy to use and maintain. This is why the testing scenarios can be written using code and configuration files. It’s a free tool, but it also offers an enterprise edition.

Gatling is an open-source tool written in Scala, but lately, it also supports Java sources to make it easier for the Java community to use. The API is almost the same between Java and Scala, so it’s easy to use if we’re familiar with any of those languages. But because Gatling was originally shipped with Scala, we might run into the need to maintain or extend Scala implementations more often.

In this tutorial, we’ll explore Gatling using Scala. We’ll see how to easily write scenarios to performance test our REST application, no matter the application’s implementation language. Also, the basic concepts of Gatling are discussed, so we won’t go into much detail about them.

2. Create Requests in Gatling Using Scala

In order to test a REST API, we need to define the REST requests in Gatling. But first, we need to implement the endpoint against which we want to run performance tests.

2.1. The REST Service API

Testing different REST operations in Gatling is done in a pretty similar way. So, for our examples, we’re going to use a REST API with a simple GET request, but POST, DELETE, etc. should be similar.

We use Scala Play, but any framework and any language can be used instead. We’ll be testing the TodoListController, which has an endpoint todo(), returning a simple 200 response. The path to reach this endpoint is /todo.

2.2. The Galting Request in Scala

On the Gatling side, we need to use HttpRequestBuilder to create the HTTP request to test this endpoint:

val request: HttpRequestBuilder = http(requestName)
  .get(requestPath)
  .check(status.is(expectedResponseStatus))
  .check(bodyString.optional.saveAs("sBodyString"))

The minimum parameters needed for this GET request in Gatling using Scala are:

• the request name, to describe the request, like request_todo_endpoint
• the request path, which should be the path of the specific request, like /todo
• the checks, where we can run some checks to validate the response or save some response fields in variables for other usages

In our solution, we’ve added the response body in the sBodyString variable to be able to log it in error cases. In this case, we log some information to make debugging easier.

2.3. Enrich the Request With Debugging Information Using Session

In order to be able to better debug issues, we can use the Session object to wrap the HttpRequestBuilder. This way we can check the status of a request and if there are failures, we can add some logs and more:

object ChainRequestsProvider {
    def simpleRequest(requestName: String, requestPath: String, expectedResponseStatus: Int): ChainBuilder = {
        val request: HttpRequestBuilder = http(requestName)
          .get(requestPath)
          .check(status.is(expectedResponseStatus))
          .check(bodyString.optional.saveAs("sBodyString"))
        exec(session => session.markAsSucceeded)
          .exec(request).doIf(_.isFailed) {
              exec { session =>
                  println("***Failure on [" + requestPath + "] endpoint:")
                  print("Gatling Session Data: ")
                  println(session.attributes.get("sBodyString"))
                  session
              }
          }
    }
}

At the beginning of each request, we set the session as successful. This is because, in Gatling using Scala, the session maintains its status even after some different requests. So we reset the session at the beginning of each request. Then we check if it’s failed, to validate if the current request check failed. Finally, we log the data we need for debugging, if we get a failure on the request.

Notably, we don’t execute the request yet. We’ve only created a ChainBuilder object that we’ll use later. We’re still missing some configurations in order to execute the request, like the base URL, the test load, etc. Moreover, this object is only a part of the broader test simulation.

3. Create Scenarios in Gatling Using Scala

As mentioned earlier, the HttpRequestBuilder or ChainBuilder object we create to test a specific endpoint is still missing some configuration. We need to set a base URL and the test load configuration. To do this in Gatling using Scala, we use the scenario() method:

object ScenariosProvider {
    private val httpProtocol = http.baseUrl("http://localhost:9000").disableCaching.disableFollowRedirect
    def getScenario(scenarioName: String, request: ChainBuilder, tps: Double, rampUpSeconds: Integer, durationSeconds: Integer): PopulationBuilder = {
        scenario(scenarioName)
          .exec(request)
          .inject(
              rampUsersPerSec(0).to(tps).during(rampUpSeconds),
              constantUsersPerSec(tps).during(durationSeconds-rampUpSeconds-rampUpSeconds),
              rampUsersPerSec(tps).to(0).during(rampUpSeconds)
          )
          .protocols(httpProtocol)
    }
}

The scenario() method expects a minimum configuration of:

• the scenario name, which is a name to recognize this specific scenario, like request_todo_endpoint
• the request to execute, that is some request builder, like HttpRequestBuilder or ChainBuilder
• the inject parameters, which are the injected users numbers per time unit
• the protocol is the configuration for the HTTP scheme, like https, the base URL, caching options, etc

Regarding the injected users, Gatling in Scala provides multiple ways to set the load. The load, in transactions per second (tps) can be set in a constant manner or by specifying ramp periods to increase or decrease the load steadily. We also have to set the period of the tps and finally combine different injectors.

In this scenario, we set a ramp-up period to get the users from 0 to some value in a specific period of time, then keep the load steady for some duration, and in the final configured time ramp down to 0.

The returned object is the PopulationBuilder. We’ll use it in the next step, when we create the simulation, along with other PopulationBuilder objects for other endpoints or configurations.

4. Create and Execute Simulations in Gatling Using Scala

Bringing everything together, for load tests in Gatling using Scala, we need to define a Simulation with all of the different scenarios we want to test in parallel. The simulation can include testing different endpoints, testing with different loads, testing failing scenarios, and more:

class PeakLoadSimulation extends Simulation {
    setUp(
        getScenario("getExistingEndpoint", simpleRequest("request_todo_endpoint", "/todo", 200), 50, 10, 60),
        getScenario("nonExistingEndpoint", simpleRequest("request_wrong_endpoint", "/not-todo", 200), 5, 10, 60)
    )
}

The simulation we create, PeakLoadSimulation, needs to extend the Simulation class. Then, in the setUp() method, we define the scenarios as PopulationBuilder objects. We can add as many scenarios as we want. All of the different scenarios are to be executed in parallel.

When executing the simulation against the REST service API we defined earlier, we should get a successful execution:

Here, we can see the generated report that contains the request count, response times, and the XXth percentile of response times. The request count validates that the requests we made to the existing endpoint are ten times more than the ones we did against the non-existing one.

5. Add Assertions in Gatling Using Scala

When running the simulations, we can monitor the outcome in different ways. We could be testing if the server will handle the load off or if it will start throwing OutOfMemory errors. We could also be getting info out of dashboards from the metrics the service exposes.

Gatling using Scala provides ways to validate the execution outcomes from the testing framework side, using assertions against the scenarios. Those assertions can be per scenario and test different metrics, like the successful response rate, the response time, etc:

class PeakLoadSimulation extends Simulation {
    setUp(
        getScenario("getExistingEndpoint", simpleRequest("request_todo_endpoint", "/todo", 200), 50, 10, 60),
        getScenario("nonExistingEndpoint", simpleRequest("request_wrong_endpoint", "/not-todo", 200), 5, 10, 60)
    ).assertions(
        details("request_todo_endpoint").successfulRequests.percent.gt(99.99),
        details("request_todo_endpoint").responseTime.percentile4.lt(20),
        details("request_todo_endpoint").requestsPerSec.gt(40),
        details("request_wrong_endpoint").successfulRequests.percent.lt(1),
        details("request_wrong_endpoint").responseTime.percentile4.lt(20)
    )
}

We enrich our simulation by adding four assertions for the two scenarios. Each scenario checks at the end of the execution that the successful rate is the expected one and the response time of 99% of the responses (percentile4) is less than 20 milliseconds, which makes sense when testing servers on localhost. Last, we add the requestsPerSec, to verify that the load we want per scenario is the expected one.

The successfulRequests field should accept as successful responses the ones that pass the status check we set in the simpleRequest method. For example, if we change the missing endpoint scenario to expect 404, simpleRequest(“request_wrong_endpoint”, “/not-todo”, 404), then the successfulRequests percent should also be changed to validate that is greater than 99.

Notably, the value in the details() method matches the name we give to the HttpRequestBuilder object as requestName.

When we run the updated simulation, with the assertions, we should again get a successful execution:

This time, we see a similar report, but the outcome of the assertions is also included. First goes the requestName, then goes the description of the assertion in plain text, and last comes the outcome, true or false, followed by some extra info.

6. Conclusion

In this article, we went through Gatling using Scala, by discussing all the components needed to create a load test scenario against a REST endpoint. We looked at the basic configuration needed to create simple tests. To make the concepts clear, we demonstrated each concept with a Simulation that tests a couple of endpoints.

     

Related Stories

• eventually Block In ScalaTest
• Ignore Tests Conditionally in ScalaTest
• Test Aspects in ZIO Test

 

1. Introduction

Cats Effect is a popular library in Scala ecosystem and functional programming for managing side effects in functional programming. It provides an easy-to-use API for building asynchronous, concurrent applications. In this tutorial, we’ll explore the IOApp, a trait that serves as the entry point for simplifying the way we can write Cats Effect applications.

2. Creating a Simple Application

To start implementing applications with the Cats Effect library, we first need to add the following dependency in our build.sbt file:

libraryDependencies += "org.typelevel" %% "cats-effect" % "3.5.7"

Now we are ready to implement our application. We’ll create an application that only prints a line in our console. For this, we’ll create an object as our entry point and extend the IOApp.Simple trait. Next, we’ll define the abstract method run of the trait:

object SimpleIOApp extends IOApp.Simple {
  val run: IO[Unit] = IO.println("Running with simple app")
}

The run method takes no arguments and returns an effect of type Unit. It is intended for very simple applications and will always exit with a successful exit code. The Cats Effect runtime provides all the necessary configuration, such as an execution context, and also manages threads and the whole lifecycle of the application. We can use val instead of def for our run method because the IO effect is lazily evaluated during runtime invocation.

3. Creating an Application With Arguments

Most applications accept arguments, use custom exit codes to notify the users for not expected behavior, and may use custom resources to run. The default IOApp trait helps us implement applications with all of these features.

We’ll now create an application that will accept command-line arguments. We only need to extend the IOApp trait and then define the run method. The arguments are passed as a pure List instead of an Array. The return type is IO[ExitCode] meaning that we can define our own exit codes to indicate success or failure:

object ArgumentIOApp extends IOApp {
  def run(args: List[String]): IO[ExitCode] =
    IO.println(s"Running with args: ${args.mkString(",")}").as(ExitCode.Success)
}

In this example, we accept any number of arguments and print them in the console, returning a success code. We can further enhance our application and handle the case when no arguments have been provided. We’ll then return a custom error code:

object ArgumentIOApp extends IOApp {
  def run(args: List[String]): IO[ExitCode] =
    if (args.nonEmpty) {
      IO.println(s"Running with args: ${args.mkString(",")}")
        .as(ExitCode.Success)
    } else {
      IO.println("No args provided. Aborting").as(ExitCode(2))
    }
}

Now, we handle the case where we didn’t provide any arguments and abort the program with exit code 2, which indicates that a major error occurred. The library provides a value for an error exit code ExitCode.Error, which maps to exit code 1, but we can use any other valid number we want.

4. Defining Custom Runtime Configuration

So far, we’ve created applications that utilize the runtime configuration that the IOApp provides by default. But there are also cases in which we want to modify it and use a custom one. By default, the IOApp uses the global execution context, but we may stumble upon a case where another implementation suits our needs better. We can achieve that by defining our custom execution context and using it to evaluate computations. Let’s create another application that will use a work-stealing thread pool:

object ContextIOApp extends IOApp {
  private val customExecutionContext: ExecutionContext =
    ExecutionContext.fromExecutor(Executors.newWorkStealingPool(1))
  override def run(args: List[String]): IO[ExitCode] = {
    val computation = IO {
      println(s"Running on thread: ${Thread.currentThread().getName}")
    }
    computation.evalOn(customExecutionContext).as(ExitCode.Success)
  }
}

The only thing we changed is that we’ve defined our new execution context and that we’ve used the evalOn method, upon the IO computation, so it is evaluated with our custom execution context. Pretty straightforward.

5. Using Resources

Since everything is already in place by the trait, we can also use resources in our application. We’ll demonstrate that by creating again an execution context, but handling it as a resource this time:

object ResourceIOApp extends IOApp {
  val customExecutionContext: Resource[IO, ExecutionContext] =
    Resource.make(
      IO(ExecutionContext.fromExecutorService(Executors.newWorkStealingPool(1)))
    ) { ec =>
      IO.println("Shutting down execution contest").as(ec.shutdown())
    }
  def run(args: List[String]): IO[ExitCode] = customExecutionContext
    .use { ec =>
      IO.println(s"Running on thread: ${Thread.currentThread().getName}")
    }
    .as(ExitCode.Success)
}

We’re using the Resource monad to wrap our custom execution context and safely shut it down afterward. Then, with the use method, we make our application run and use it.

6. Interrupting Applications

The Cats Effect library uses fibers for concurrency. Our IOApp can interact with the main fiber and cancel it in case it receives an interrupt signal. This means that it’ll also propagate to all the other fibers and invoke any finalizers we’ve defined, so our application can gracefully shut down. This mechanism, however, has one major side effect. If a finalizer runs forever, the main fiber can’t shut down and it’ll also run forever, and we need to issue a SIGKILL signal for cleanup.

7. Conclusion

In this article, we’ve explored how we can define simple and more complex applications easily with the IOApp trait of the Cats Effect library. We’ve demonstrated how to use custom error codes, our custom execution context and resources, as well as how the application can be interrupted.

     

Related Stories

• Effective Error Handling With EitherT in Scala
• CSV Files in Scala
• Introduction to Chimney

 

1. Introduction

In Scala 3, context functions are functions with only implicit parameters or, more precisely, context parameters. This means the compiler will input the correct arguments (taking them from the “context”, aka the scope) rather than explicitly writing them in the code.

In this tutorial, we’ll examine context functions and how to use them.

2. The Syntax

In Scala 3, we write the type of a context function using ?=>. This is different from “ordinary” functions, where we use the “arrow” sign: =>.

As we saw above, context functions do not have any explicit parameters. Therefore, to use one of the arguments in the implementation, we’ll have to use summon[T], which is similar to implicitly[T] in Scala 2. In brief, summon[T] looks for an implicit value of type T in the scope. In Scala 3, we can provide such values using given.

Let’s see an example of a context function to increment a number:

val increment: Int ?=> Int = summon[Int] + 1

We define a context function simply by specifying its type and body, just as we’d do with a “regular” function. The Int ?=> Int notation is a shortcut for ContextFunction1[Int, Int], as much as Int => Int is syntactic sugar for Function1[Int, Int].

For context functions, however, the type is mandatory. If we omitted Int ?=> Int, the compilation would fail:

No given instance of type Int was found for parameter x of method summon in object Predef
val increment = summon[Int] + 1

To invoke our function, we have to provide an implicit parameter either explicitly or via the implicit scope:

// Implicit parameter provided via the scope
given Int = 1
println(s"Result scope: $increment")
// Implicit parameter provided explicitly to the call
println(s"Result explicit: ${increment(using 5)}")

In the first case, we used given to put an Int into the implicit scope. Here, we can invoke increment simply by calling it. In the second case, we explicitly provided the parameter with using at the call site. If we run the example above, we’ll see two lines in the output:

Result scope: 2
Result explicit: 6

2.1. Multiple Context Parameters

Context functions can have more than a single parameter. Let’s see an example:

val repeatString: String ?=> Int ?=> String = summon[String].repeat(summon[Int])
println(s"Result repeatString: ${repeatString(using "Baeldung")(using 3)}")

In the example above, we defined a function with two context parameters, a String and an Int, and invoked it just as before:

Result repeatString: BaeldungBaeldungBaeldung

3. An Example: Modeling Kotlin Scope Functions

Let’s now see a more complex example of a context function. We’ll use them to model one of Kotlin’s scope functions, also.

In Kotlin, also is a function taking the context object as a parameter, with name it, and returning the context object itself. In brief, whenever we see also in the code we can think of a function that uses a given object and returns it, without modifying it.

Let’s take a look at Scala’s implementation:

opaque type WrappedAlso[T] = T
def it[T](using a: WrappedAlso[T]): T = a
extension [T](x: T)
  def also(f: WrappedAlso[T] ?=> Unit): T =
    f(using x)
    x

First, we modeled an opaque type, named WrappedAlso, which essentially wraps a value of type T. This is always advised with context functions to prevent unwanted givens from being picked from the scope.

Secondly, we implemented a constructor to avoid explicitly summoning a parameter of type T. To comply with Kotlin’s naming convention, we call this method it.

Lastly, we defined an extension method, called also, doing all the magic. It inputs a context function type WrappedAlso[T] ?=> Unit, which means we’ll have to provide a context function using a value of type T. also invokes the function f with x as an explicit parameter, discards its result, and returns the original value x.

Since WrappedAlso is an opaque type alias, its values won’t need boxing. Furthermore, since also is added as an extension method, its argument does not need boxing either.

Let’s see how we can use our new function:

val numbers = List(1, 2, 3, 4, 5)
numbers
  .also(println(s"The list before adding 6: $it"))
  .appended(6)
  .also(println(s"The list after adding 6: $it"))

In the example above, we defined a list of numbers and made two calls to also, before and after adding an element. In the function body, we can refer to the context parameter with it, as much as we’d do in Kotlin.

If we hadn’t defined the it constructor, we would have had to refer to it with an explicit summon[List[Int]], which is far less readable.

Running the example produces the expected output:

The list before adding 6: List(1, 2, 3, 4, 5)
The list after adding 6: List(1, 2, 3, 4, 5, 6)

Lastly, any changes to the list in the body of the also function will not get propagated:

numbers
    .also(it appended 7)
    .also(println(s"The list after adding 7: $it"))

The example above produces The list after adding 7: List(1, 2, 3, 4, 5) as an output.

4. Conclusion

In this article, we learned about context functions are in Scala 3. We started with an overview of the syntax, and then we deepened our knowledge with functions with more than one parameter. Finally, we looked at a practical example, showing how simple it is to model idiomatic Kotlin scope functions.

     

Related Stories

• TASTy Files in Scala 3

 

1. Overview

In this tutorial, we’ll compare the usage of vals and defs within trait definitions. Using a val rather than defs within traits can appear like a small difference on the surface, but can become problematic when extending the trait. We’ll cover the concept of a trait and explore some concrete examples of what can go wrong when extending a trait containing a val.

2. Abstraction in Traits

A trait, similar to an interface in Java, should be used to define an abstract blueprint, which other abstract classes or a concrete class definition can extend. As such, we don’t want a trait to contain any implementation details that the extending class should provide. This includes providing functionality that restricts the extending class.

For example, if we provided a SqlConnection field for a trait that provides access to a database. When we come to create an implementation for an in-memory or NoSql database, we can’t because we’re locked into providing a SqlConnection. This implementation detail should only be provided within the extending class.

The same applies to the variable type of a field. By defining a field as a val, we enforce that the the field is instantiated at the point the class is created. This could be inappropriate for particular implementations of the trait. To ensure we’re not restricting the implementation of an extending class, we should use the most open variable type of def.

3. Restricting Fields

Now we understand the concept of a trait and why we should favor defs fields over vals from a conceptual point of view. Let’s look at some concrete examples to see the problems in practice:

trait CurrentTimePrinter {
  val currentTime: LocalDateTime
  def printCurrentTime(): Unit
}
object TimePrinterImpl extends CurrentTimePrinter {
  override val currentTime: LocalDateTime = LocalDateTime.now()
  override def printCurrentTime(): Unit = println(currentTime)
}

In this example, we define a trait called CurrentTimePrinter with a field called currentTime. Then, a function called printCurrentTime() needs to be implemented in the extending class. Below the trait, we create an object that implements CurrentTimePrinter and overrides the fields defined in the trait. If we run this code and call printCurrentTime, the same LocalDateTime value will be printed each time as currentTime is a val.

Let’s try and change it to a def, so LocalDateTime.now() is called each time to get an up-to-date value:

object TimePrinterImpl extends CurrentTimePrinter {
  override def currentTime: LocalDateTime = LocalDateTime.now()
  override def printCurrentTime(): Unit = println(currentTime)
}

If we try to compile this code now, we’re given an error:

overriding value currentTime in trait CurrentTimePrinter of type java.time.LocalDateTime;
method currentTime needs to be a stable, immutable value

This occurs because our trait defines currentTime as an immutable value by setting it as a val. Therefore, we have to provide an immutable value when writing its implementation. This forces the implementing class to use a val for currentTime. Which, in our use case, isn’t appropriate. We can fix this by setting currentTime to a def in CurrentTimePrinter:

trait CurrentTimePrinterWithDef {
  def currentTime: LocalDateTime
  def printCurrentTime(): Unit
}
object TimePrinterWithDefImpl extends CurrentTimePrinterWithDef {
  override def currentTime: LocalDateTime = LocalDateTime.now()
  override def printCurrentTime(): Unit = println(currentTime)
}

This time, we can call printCurrentTime() and it will print a newly calculated LocalDateTime for each call.

By setting currentTime as a def, we’ve allowed the implementing class the flexibility to choose the field type. We could override it as a def, val lazy val, or even a var if we wanted to. Using a def in the trait makes the field as open as possible for the overriding class to choose what the implementation should do.

4. Initialization Order

The next issue we can encounter when using vals in traits comes when using multiple inheritance. Let’s create some traits and classes to explore this issue:

trait TraitA {
  val multiplier: Int
  val result: Int = multiplier * 7
}
trait TraitB {
  val multiplier: Int = 10
}
object AB extends TraitA with TraitB
object BA extends TraitB with TraitA

Here, we’ve created two traits. TraitA has an abstract field of multiplier and then a concrete field of result, which multiplies the multiplier by 7. Next, we have TraitB, which has a concrete member of multiplier, initialized to 10. Finally, we have two object definitions, AB, which extends TraitA with TraitB. Then, BA which does the opposite – TraitB with TraitA.

Let’s call the result on both these objects and see what happens:

AB.result // AB.result = 0
BA.result // BA.result = 70

As we can see, these two results are very different and unexpected for just reversing the order in which we extend the traits. This happens because traits are initialized in the order they’re listed. So, in AB, when the result is calculated in TraitA, the multiplier hasn’t been set to 10, as TraitB hasn’t been initialized yet. So, the multiplier defaults to zero, and the result is zero. Whereas, in BA, TraitB is initialized first, so when result is calculated in TraitA, multiplier has been set to 10, therefore we get 70.

As we can see, this effect could produce unexpected results in our codebases. Fortunately, we can mitigate against this by setting result as a def in TraitA:

trait TraitA {
  val multiplier: Int
  def result: Int = multiplier * 7
}
trait TraitB {
  val multiplier: Int = 10
}
object AB extends TraitA with TraitB
object BA extends TraitB with TraitA

Now, let’s call result on both objects again:

AB.result // AB.result = 70
BA.result // BA.result = 70

This time, both results returned 70. This is because result is now a def, which is calculated each time we call it. Meaning that the object and all the traits it extends have been initialized at the point we call the result.

5. Conclusion

In this article, we’ve discussed the good practice of separating implementation details from the abstractions we set within trait definitions. Next, we explored the issues that can occur when we extend a trait containing a val using the example of printing the currentTime. Finally, we saw the unexpected results that can occur in our codebase caused by the initialization order in multiple inheritance using traits with vals.

     

Related Stories

• Stackable Trait Pattern in Scala
• Meaning of _root_ In Scala Import Clause

 

1. Introduction

As a functional programming language, Scala provides powerful abstractions for working with collections, monads, and other structures. Among these abstractions, map() and flatMap() are two of the most frequently used higher-order functions. While they might seem similar at first glance, they serve different purposes and behave differently depending on the context.

In this tutorial, let’s explore the difference between those functions.

2. map()

The map() function transforms each collection element or monadic structures like Option, Future, etc. by applying a given function. The result is a new structure containing the transformed elements.

Let’s look at the syntax of the map() function:

def map[B](f: A => B): F[B]

As we can see, the map() function takes a function that transforms a value of type A into a value of type B.

Let’s explore a few example usages from Scala’s standard library types, such as List and Option:

// on List
val numbers = List(1,2,3,4,5)
val doubled = numbers.map(_ * 2) // List(2, 4, 6, 8, 10)
// on Option
val intOpt = Option(2)
val doubledOpt = intOpt.map(_ * 2) // Option(4)

Similarly, many classes from both the standard library and third-party libraries implement the map() function, as it’s one of the most commonly used operations.

Typically, map() is used when working with the successful side of a monadic structure, such as the Some case in Option or the Right case in Either. This allows transformations without altering the underlying structure. Failure cases such as None and Left are not transformed. However, this is more of a convention than a strict rule on the monadic implementation

3. flatMap()

The flatMap() function can also transform the elements of a collection or a monadic context. However, it’s applicable when the transformation function returns a structure of the same type. It flattens the resulting nested structure into a single structure.

Let’s look at the syntax:

def flatMap[B](f: A => F[B]): F[B]

Here, we can see that the transformation function f in flatMap() returns F[B], whereas in map(), it returns B.

Let’s look at an example by using List:

val sentences = List("Hello world", "Scala is awesome", "Functional programming is fun")
val words = sentences.flatMap(sentence => sentence.split(" "))
assert(words == List("Hello", "world", "Scala", "is", "awesome", "Functional", "programming", "is", "fun"))

In this example, the split method returns an array of words, and flatMap() is used to flatten the resulting lists of words into a single list. If we had used map() instead of flatMap(), the result would have been a nested list, where each element is a list of words for each sentence. To flatten this structure, we would then need to use flatten(), which can be avoided by directly using flatMap(), as it handles both the transformation and flattening in a single step.

Like map(), flatMap() also operates on the successful side of a monad.

When multiple nested flatMap() calls are involved, the code can become too difficult to read. In such cases, for-comprehension provides a more readable alternative. Internally, for-comprehensions are syntactic sugar for chaining flatMap() operations.

Let’s look at another example using Option chaining:

val supportedOs = Option("Linux")
val input = Option("Linux")
supportedOs.flatMap { os =>
  input.flatMap { in =>
    if (in == os) then Some("Succes") else None
  }
}

We can rewrite this to a simpler version using for-comprehension:

val supportedOs = Option("Linux")
val input = Option("Linux")
for {
  os <- supportedOs
  in <- input
  if in == os
} yield "Success"

This improves readability when dealing with multiple flatMap() chains.

4. Summary

Let’s summarize what we discussed about map and flatMap

5. Conclusion

In this tutorial, we explored the differences between map() and flatMap() functions. These two are the most fundamental operations in any functional programming language. A clear understanding of these functions is essential for utilizing the full power of functional programming.

     

Related Stories

• Convert a Scala List to a Tuple
• Remove an Element From a Scala Collection
• Calculate the Average of a List in Scala

 

1. Introduction

The Play framework comes with Twirl as the default template engine. Twirl is Scala-based and easy to learn for developers who have worked with Scala and HTML.

In this tutorial, we’ll demonstrate how to create and submit forms using Twirl within a Play application.

2. Form Definition and Simple Example

Forms are Scala objects defined in the application code and passed through template parameters to the Twirl templates to be compiled and rendered. The apply methods of the Form helper object are a convenient way to create a Form object:

val form: Form[SimpleForm] = Form(
  mapping(
    "i" -> number,
    "active" -> boolean,
    "msg" -> text
  )(SimpleForm.apply)(SimpleForm.unapply)
)

Now that we’ve created a form with three fields, including a number, a boolean, and a text field, it’s time to use it within a template:

@(simpleForm: Form[SimpleForm])(implicit messages: Messages)
@form(action = routes.FormController.formPost()) {
    @inputText(simpleForm("i"))
    @inputText(simpleForm("active"))
    @inputText(simpleForm("msg"))
    
}

Subsequently, a controller method will expose the template:

def simpleForm: Action[AnyContent] = Action { implicit request =>
  Ok(views.html.Baeldung.FormTemplate(SimpleForm.form))
}

Finally, the controller’s formPost function handles the form submission:

def formPost(): Action[AnyContent] = Action { implicit request =>
  val form = SimpleForm.form.bindFromRequest().get
  Ok(form.toString)
}

Now that all the building blocks are in place, let’s fetch the form to inspect the HTML code:

$ curl localhost:9000/simple-form | awk '!/^[[:space:]]*$/'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   732  100   732    0     0  75046      0 --:--:-- --:--:-- --:--:-- 81333
<form action="/simple-form" method="POST" >
<dl class=" " id="i_field">
    <dt><label for="i">i</label></dt>
    <dd>
    <input type="text" id="i" name="i" value="" />
</dd>
        <dd class="info">Numeric</dd>
</dl>
<dl class=" " id="active_field">
    <dt><label for="active">active</label></dt>
    <dd>
    <input type="text" id="active" name="active" value="" />
</dd>
        <dd class="info">format.boolean</dd>
</dl>
<dl class=" " id="msg_field">
    <dt><label for="msg">msg</label></dt>
    <dd>
    <input type="text" id="msg" name="msg" value="" />
</dd>
</dl>
    <input type="submit" value="Submit">
</form>

It’s important to note that the awk post-processing we’ve added removes unnecessary empty lines added by the template engine and it’s used to shorten the curl command’s output. In other words, the awk part of the command is completely optional and can be omitted.

3. Field Types

Out of the box, Twirl forms support various field types such as number, boolean, date, sqldate, localdate, uuid, and more. Specifically, the Forms helper object contains the mapping code for these types.

To demonstrate some of the various supported field types, we’ve created the MultipleFieldsForm:

val form: Form[MultipleFieldsForm] = Form(
  mapping(
    "i" -> number,
    "pwd" -> text,
    "active" -> boolean,
    "msg" -> text,
    "date"-> date,
    "uuid" -> uuid,
    "favMovie" -> text,
    "favDrink" -> text
  )(MultipleFieldsForm.apply)(MultipleFieldsForm.unapply)
)
object Movies {
  val list = List(
    "pulpFiction" -> "Pulp Fiction",
    "inception" -> "Inception",
    "theMatrix" -> "The Matrix",
    "titanic" -> "Titanic",
  )
}
object Drinks {
  val list = List(
    "vodka" -> "Vodka",
    "tequila" -> "Tequila",
    "whisky" -> "Whisky",
    "wine" -> "Wine",
    "beer" -> "Beer"
  )
}

By using template input helpers, the generated form prevents user input errors by adding type validations:

@(multipleFieldsForm: Form[MultipleFieldsForm])(implicit messages: Messages)
@form(action = routes.FormController.multipleFieldsFormPost()) {
    @inputText(multipleFieldsForm("i"))
    @inputPassword(multipleFieldsForm("pwd"))
    @checkbox(multipleFieldsForm("active"))
    @inputText(multipleFieldsForm("msg"))
    @inputDate(multipleFieldsForm("date"))
    @inputText(multipleFieldsForm("uuid"))
    @inputRadioGroup(multipleFieldsForm("favMovie"), MultipleFieldsForm.Movies.list)
    @select(multipleFieldsForm("favDrink"), MultipleFieldsForm.Drinks.list)
    <input type="submit" value="Submit">
}

Twirl also supports custom fields, as long as a Mapping implementation is present. For this purpose, we’ll introduce the UnitMeasurement case class:

case class UnitMeasurement(quantity: Int, unit: String)

In other words, we’ll add support for fields like “1 pound” and “3 meters”. Initially, a Mapping of this custom type is required, namely the unitMeasurementMapping:

def unitMeasurementMapping: Mapping[UnitMeasurement] =
  Forms.of[UnitMeasurement]

Another addition we have to include to make our custom type work is the implicit Formatter since the Forms.of method requires it:

implicit def binder: Formatter[UnitMeasurement] =
  new Formatter[UnitMeasurement] {
    override def bind(
      key: String,
      data: Map[String, String]
    ): Either[Seq[FormError], UnitMeasurement] = Formats.parsing(
      d => UnitMeasurement.fromString(d),
      "The format is (\\d*)(\\s)(\\D*)- example: \"1 pound\"",
      Nil
    )(key, data)
    override def unbind(
      key: String,
      value: UnitMeasurement
    ): Map[String, String] = Map(key -> s"${value.quantity} ${value.unit}")
  }

Essentially, the Formatter[UnitMeasurement] trait defines the “UnitMeasurement to String” and “String to UnitMeasurement” transformations. Another important building block is the helper method Formats.parsing that provides every Form-related functionality, requiring only a “String to T” arrow function.

The UnitMeasurement.fromString method implements the actual transformation:

object UnitMeasurement {
  implicit val unitMeasurementFormWrites: Writes[UnitMeasurement] =
    Json.writes[UnitMeasurement]
  private val pattern = "(\\d*)(\\s)(\\D*)".r
  def fromString(str: String): UnitMeasurement = {
    val matches = pattern.findAllIn(str)
    if (matches.hasNext) {
      val List(number, space, quantity) = matches.subgroups
      UnitMeasurement(number.toInt, quantity)
    } else {
      throw new RuntimeException(s"Incorrect data: $str")
    }
  }
}

Now, we’re ready to use this new custom Mapping in a form definition:

val form: Form[ComplexFormCustomField] = Form(
  mapping(
    "i" -> number,
    "active" -> boolean,
    "msg" -> text,
    "measurement" -> Measure.unitMeasurementMapping
  )(ComplexFormCustomField.apply)(ComplexFormCustomField.unapply)
)

4. Field Constraints

While Twirl provides field-type validation out of the box, additional constraints can be applied to the form fields. The Forms helper object provides a variety of helper functions that return constrained mappings:

val form: Form[InputFormWithConstraints] = Form(
  mapping(
    "i" -> number(min = 10, max = 20),
    "msg" -> text(minLength = 3, maxLength = 12),
    "msgOpt" -> optional(text),
    "email" -> email,
    "birthday" -> date
  )(InputFormWithConstraints.apply)(InputFormWithConstraints.unapply)
)

Another neat feature is that we’re able to define custom constraints by leveraging the verifying method of the Mapping trait:

val form: Form[InputFormWithConstraints] = Form(
  mapping(
    "email" -> email.verifying(minLength(15)),
    "birthday" -> date.verifying(
      "Not 18 years old",
      d => d.toInstant.isBefore(Instant.now().minus(18, ChronoUnit.YEARS))
    )
  )(InputFormWithCustomConstraints.apply)(
    InputFormWithCustomConstraints.unapply
  )
)

In the above example, an email is valid only if it’s at least 15 characters long. Also, a birthday date is considered valid if the date is older than 18 years.

5. Validation

Field constraints enable us to provide meaningful feedback to the users regarding unsuccessful form submissions. To do so, the controller methods that handle form submission requests need to be written differently:

def formWithConstraintsPost(): Action[AnyContent] = Action { implicit request =>
  InputFormWithConstraints.form
    .bindFromRequest()
    .fold(
      { formWithError =>
        BadRequest(views.html.Baeldung.FormTemplateWithConstraints(formWithError))
      },
      { data => Ok(Json.toJson(data)) }
    )
}

In case of erroneous form submissions, the formWithConstraintsPost method wraps the template initialized with the rejected form in a BadRequest. Let’s post an empty form to inspect the server response:

$ curl localhost:9000/form-with-constraints -d "" | awk '!/^[[:space:]]*$/'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1655  100  1655    0     0   183k      0 --:--:-- --:--:-- --:--:--  202k
<form action="/form-with-constraints" method="POST" >
<dl class=" error" id="i_field">
    <dt><label for="i">i</label></dt>
    <dd>
    <input type="text" id="i" name="i" value="" />
</dd>
        <dd class="error">This field is required</dd>
        <dd class="info">Minimum value: 10</dd>
        <dd class="info">Maximum value: 20</dd>
        <dd class="info">Numeric</dd>
</dl>
<dl class=" error" id="msg_field">
    <dt><label for="msg">msg</label></dt>
    <dd>
    <input type="text" id="msg" name="msg" value="" />
</dd>
        <dd class="error">This field is required</dd>
        <dd class="info">Minimum length: 3</dd>
        <dd class="info">Maximum length: 12</dd>
</dl>
<dl class=" " id="msgOpt_field">
    <dt><label for="msgOpt">msgOpt</label></dt>
    <dd>
    <input type="text" id="msgOpt" name="msgOpt" value="" />
</dd>
</dl>
<dl class=" error" id="email_field">
    <dt><label for="email">email</label></dt>
    <dd>
    <input type="text" id="email" name="email" value="" />
</dd>
        <dd class="error">This field is required</dd>
        <dd class="info">Email</dd>
</dl>
<dl class=" error" id="birthday_field">
    <dt><label for="birthday">birthday</label></dt>
    <dd>
    <input type="date" id="birthday" name="birthday" value="" />
</dd>
        <dd class="error">This field is required</dd>
        <dd class="info">Date ('yyyy-MM-dd')</dd>
</dl>
    <input type="submit" value="Submit">
</form>

As expected, the HTML response is our form with error element additions based on the field constraints we defined earlier.

6. Conclusion

In this article, we’ve described how to create, render, submit, and validate HTML forms with Play framework. The Twirl engine is a solid choice for projects using Scala and Play if there’s no alternative way of handling forms.

      

1. Introduction

ScalaTest is one of the most popular and widely used testing frameworks in Scala. It offers various tools to write expressive and concise tests, and one of its standout features is the eventually block. This utility is especially useful when dealing with asynchronous or eventually consistent systems, such as microservices, distributed databases, or message queues.

In this tutorial, let’s look at how to use the eventually block effectively.

2. Set Up

First, let’s add the ScalaTest library dependency to our build.sbt file:

libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.19" % Test

Now, let’s create a simple test. For this example, we’ll simulate an asynchronous operation where a value is updated after a delay. This pattern resembles scenarios such as database write operations, where updates take some time to complete:

class AsyncOperation(waitTime: Int) {
  private var database: Map[String, String] = Map.empty
  def writeToDatabase(key: String, value: String): Unit = {
    new Thread(() => {
      Thread.sleep(waitTime) // Simulate a delay in writing to the database
      database += (key -> value)
    }).start()
  }
  def readFromDatabase(key: String): Option[String] = database.get(key)
}

The writeToDatabase() method simulates a delayed async write, while readFromDatabase() checks the result. Let’s write a simple test to verify this code:

it should "get the result from the database after save" in {
  val op = AsyncOperation(100)
  op.writeToDatabase("key", "value")
  op.readFromDatabase("key") shouldBe Some("value")
}

This test fails because the read operation is attempted before the write operation has finished.

Using ScalaTest’s eventually block, we’ll verify that the data appears in the database after the write operation is completed.

3. Eventually Block

The eventually block in ScalaTest repeatedly retries the provided code until the assertion passes or the specified timeout is reached. This is particularly useful for testing asynchronous operations, eventual consistency, or delayed responses, such as database writes, API calls, or message queue processing.

We can incorporate the eventually block by either extending the test class with the Eventually trait or importing it directly. In this example, we’ll mix in the Eventually trait to our test class:

class EventuallyUnitTest extends AnyFlatSpec with Matchers with Eventually {
  it should "retry the save to database multiple times in eventually with default timeout" in {
    val op = AsyncOperation(100)
    op.writeToDatabase("key", "value")
    eventually {
      op.readFromDatabase("key") shouldBe Some("value")
    }
  }
}

This code is similar to the failed test described in the previous section but with one difference: the read operation is now wrapped within the eventually block. As a result, the eventually block retries the read operation multiple times until the assertion succeeds or the timeout is reached. By default, it uses a timeout of 150 milliseconds and a retry interval of 15 milliseconds. Since the write operation takes 100 milliseconds, the default timeout provides sufficient time for the read operation to succeed, allowing the test to pass.

Additionally, the eventually block uses a backoff algorithm to optimize retries. This helps to avoid overloading the system by gradually increasing the interval between each retry.

4. Configuring the Timeout

Let’s see what happens if the write operation takes more than 150 milliseconds. Let’s update the test to use more time for write operation:

val op = AsyncOperation(300)
op.writeToDatabase("key", "value")
eventually {
  op.readFromDatabase("key") shouldBe Some("value")
}

Running the test with this code fails, displaying the message:

The code passed to eventually never returned normally.
Attempted 5 times over 154.83845499999998 milliseconds. 
Last failure message: None was not equal to Some("value").

This shows that even after 5 retries over approximately 150 milliseconds, the read operation didn’t return the expected value. It’s important to note that due to task scheduling and thread availability, the actual timeout may slightly exceed the configured 150 milliseconds.

In cases where we know the operation requires more time, we can adjust the timeout by providing a custom implicit value of type PatienceConfig using the given keyword, which the eventually block will use:

given patienceConfig: PatienceConfig =
  PatienceConfig(
    timeout = scaled(Span(2, Seconds)),
    interval = scaled(Span(5, Millis))
  )

This configuration sets custom values for the timeout and interval. Using scaled with the new defaults is optional but recommended, as it allows the values to adjust dynamically when tests run on systems with varying performance. With this updated configuration, the test now passes as the timeout is increased to 2 seconds.

Additionally, instead of using the implicit configuration, we can also pass the timeout and interval explicitly in the eventually block:

eventually(timeout = Timeout(Span(2, Seconds)), interval = Interval(Span(50, Millis))) {
  op.readFromDatabase("new-key") shouldBe Some("value")
}

This makes the timeout clearer than just using an implicit variable.

5. Conclusion

In this article, we explored the eventually block in ScalaTest and its usefulness in verifying asynchronous operations, particularly in integration tests. We also discussed how to configure timeout values to meet specific requirements.

     

Related Stories

• Ignore Tests Conditionally in ScalaTest
• Comparing Double Values in ScalaTest
• Testing Akka Streams

 

1. Introduction

In a previous article, we saw what Akka Streams are and how to test them. In this article, we’ll dive into error handling. Typically, when an operator in a stream fails, the entire stream gets torn down. In that case, all the downstream operators get informed about the failure and won’t get run, whereas upstream operators see a cancellation.

There are three main ways to avoid complete stream failure: Recovering from the failure, restarting the stream, or supervising it.

In this tutorial, we’ll briefly examine all three ways to handle errors in Akka Streams.

2. Recovering from Failures

Stream recovery allows us to emit one final element downstream instead of the error to avoid complete stream failure. We can do this with the recover operator.

Let’s see an example:

val parseWithRecover: Flow[String, Either[String, (Int, Int)], NotUsed] =
    Flow[String]
      .map { pair =>
        val parts = pair.split(",")
        Right((parts(0).toInt, parts(1).toInt))
      }
      .recover({ case e: ArrayIndexOutOfBoundsException =>
        Left(e.getMessage)
      })

In the code snippet above, we added recover to the parse Flow we saw in the Introduction to Akka-Streams in Scala article. In particular, we changed the return type of the Flow to be Either[String, (Int, Int)]. This way, we can react to ArrayIndexOutOfBoundsExceptions, returning an error message.

Let’s see a simple test for the new Flow:

"The \"parseWithRecover\" flow" should "parse recover from a parsing error" in {
    val (pub, sub) = TestSource[String]()
      .via(parseWithRecover)
      .toMat(TestSink[Either[String, (Int, Int)]]())(Keep.both)
      .run()
    pub.sendNext("1,1")
    pub.sendNext("145146")
    pub.sendComplete()
    sub.requestNext(Right(1, 1))
    sub.requestNext(Left("Index 1 out of bounds for length 1"))
    sub.expectComplete()
  }

2.1. Adding Retries

The recover operator only lets us emit one element instead of a failed one. recoverWithRetries does the same thing up to a specified number of times. This is useful when dealing with temporary errors, such as network hiccups.

3. Restarting the Stream After a Failure

In Akka Streams, we can restart a single stream operator: Source, Flow, or Sink. We can do that using the RestartSource, RestartFlow, and RestartSink constructs, respectively, to implement an exponential backoff restart strategy. This means that when the given operator fails, Akka will restart it, waiting an increasing amount of time between two subsequent restarts.

This pattern lets us add restart support on a very fine grade, enabling it for single-stream operators. It is also very useful for mitigating the effects of transient failures, such as resources not being available.

Let’s see an example, using the stream elements implemented in the “Introduction to Akka-Streams in Scala” article and adding backoff restarts to the source:

private val backoffSettings = RestartSettings(
  minBackoff = 3 seconds,
  maxBackoff = 30 seconds,
  randomFactor = 0.2
).withMaxRestarts(3, 5.minutes)
RestartSource
  .withBackoff(backoffSettings) { () => source }
  .via(parse)
  .via(compare)
  .runWith(sink)
  .andThen {
    case Failure(exception) => println(exception)
    case Success((correct, total)) =>
      println(s"$correct/$total correct answers")
  }(system.dispatcher)
  .onComplete(_ => system.terminate())(system.dispatcher)

In the example above, we defined a backoff strategy with the RestartSettings class. In particular, we set the initial restart delay to 3 seconds (minBackoff), allowing the delay to grow until 30 seconds (maxBackoff), with a randomFactor of 0.2. The random factor simply randomizes the backoff calculation; the higher the value, the more random the backoff delay.

Lastly, we allowed the stream to restart at most three times or for five minutes.

We then add backoff capabilities to source by wrapping it inside RestartSource#withBackoff, which inputs the backoff settings and a lazily-evaluated source. If we run the example above, we get the following output:

[baeldung-akka.actor.default-dispatcher-6] INFO akka.stream.scaladsl.RestartWithBackoffSource - Restarting stream due to completion [1]
[baeldung-akka.actor.default-dispatcher-7] INFO akka.stream.scaladsl.RestartWithBackoffSource - Restarting stream due to completion [2]
[baeldung-akka.actor.default-dispatcher-6] INFO akka.stream.scaladsl.RestartWithBackoffSource - Restarting stream due to completion [3]
16/24 correct answers

The output shows the stream restarted three times, as expected. This is because RestartSource#withBackoff restarts the stream on success and failure.

If we wanted to restart it only on failure, we’d have to use RestartSource#onFailuresWithBackoff.

A RestartSource will not complete or fail until maxRestarts is reached, as it handles completion and failure by restarting the underlying Source. For RestartFlow and RestartSink, the restart might cause some messages to get lost.

4. Supervising a Stream

Supervision is a more advanced error resolution strategy, inspired by the error handling of Akka actors.

However, it does not make sense in all scenarios. For instance, if an operator interacts with an external service over the network, a back-off strategy is more suited if the connection drops. Therefore, not all operators support stream supervision out of the box and Akka Stream does not supervise streams automatically.

On the contrary, streams handling user data might benefit from supervision, as it gives us more control over what to do in case of failure.

We can choose one of three supervision strategies:

• Stop: Where the stream is completed with failure. This is the default
• Resume: Where the element causing the failure is dropped without terminating the stream
• Restart: Where the element causing the failure is dropped and the stream continues after restarting the operator, effectively creating a new operator instance. In this case, all the internal state accumulated in the stream gets lost, and the stream will not replay the elements that flowed in the past

Defining a supervision strategy requires accessing the stream’s underlying runnable graph. Let’s see an example:

val runnableGraph = TestSource[String]()
  .via(parseWithRecover)
  .toMat(TestSink[Either[String, (Int, Int)]]())(Keep.both)
val decider: Supervision.Decider = {
  case _: ArrayIndexOutOfBoundsException => Supervision.Resume
  case _                                 => Supervision.Stop
}
val graphWithResumeSupervision =
      runnableGraph.withAttributes(ActorAttributes.supervisionStrategy(decider))
val (pub, sub) = graphWithResumeSupervision.run()
pub.sendNext("1,1")
pub.sendNext("145146")
pub.sendNext("1,2")
pub.sendComplete()
sub.requestNext(Right(1, 1))
sub.requestNext(Right(1, 2))
sub.expectComplete()

In the example above, we first defined a supervision strategy using the Supervision.Decider class. In particular, we chose to resume the stream only in the case of ArrayIndexOutOfBoundsExceptions, stopping it for all the other errors. Secondly, we decorated the stream graph with the withAttributes method, and then we ran it.

In particular, we used the test kit to send three elements to the stream, with the second crafted to throw an ArrayIndexOutOfBoundsException. Nonetheless, the stream only emitted two elements before completion, proving the second was indeed dropped.

5. Logging Stream Elements

Logging is not a form of error handling per se, but it is certainly a good practice to signal that something went wrong during the execution of a stream. We can use the log operator in Akka Streams to that end:

val parseWithLogging: Flow[String, (Int, Int), NotUsed] =
    Flow[String]
      .map { pair =>
        val parts = pair.split(",")
        (parts(0).toInt, parts(1).toInt)
      }
      .log(name = "Baeldung stream")
      .addAttributes(
        Attributes.logLevels(
          onElement = Attributes.LogLevels.Info,
        )
      )

To have Akka Stream log something, we add a logging backend to our project. For example, adding slf4j-simple to the build.sbt file:

libraryDependencies ++= Seq(
  "org.slf4j" % "slf4j-api" % "2.0.16",
  "org.slf4j" % "slf4j-simple" % "2.0.16"
)

By default, the log operator logs the element of a stream at the debug level and any exceptions at the error level. However, logging levels are configurable via Attributes#logLevels(). If we run the example in the “Introduction to Akka-Streams in Scala” article, using parseWithLogging instead of parse, we’ll get the following output:

[baeldung-akka.actor.default-dispatcher-6] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
[baeldung-akka.actor.default-dispatcher-7] INFO akka.stream.Materializer - [Baeldung stream] Element: (5,10)
[baeldung-akka.actor.default-dispatcher-7] INFO akka.stream.Materializer - [Baeldung stream] Element: (15,15)
[baeldung-akka.actor.default-dispatcher-7] INFO akka.stream.Materializer - [Baeldung stream] Element: (78,79)
[baeldung-akka.actor.default-dispatcher-7] INFO akka.stream.Materializer - [Baeldung stream] Element: (12,12)
[baeldung-akka.actor.default-dispatcher-7] INFO akka.stream.Materializer - [Baeldung stream] Element: (0,0)
[baeldung-akka.actor.default-dispatcher-7] INFO akka.stream.Materializer - [Baeldung stream] Element: (456,456)

Where each element is logged, along with the name of the stream.

Stream logging is a broad topic. For the scope of this tutorial, it is enough to know that Akka Streams lets us decide what to log and to what level.

6. Conclusion

In this article, we saw many ways to deal with errors in Akka Streams. First, we analyzed recoveries, allowing us to define a strategy to replace faulty elements. Secondly, we moved to restart the components of a stream, a strategy suitable to cope with temporary errors. Thirdly, we uncovered the power of supervision to define fine-grained error handling. Lastly, we took a quick look at logging individual stream elements or errors as they happen.

     

Related Stories

• Testing Akka Streams
• Introduction to Apache Pekko
• Reading CSV Files Using Akka-Streams

 

1. Overview

In this tutorial, we’ll demonstrate how to remove the first n digits from a number in Scala using basic string manipulation techniques.

2. Converting the Number to a String

To begin, we need to ensure our number can be manipulated as a string. This step is essential, as we will be removing characters based on their positions.

Here’s a straightforward approach:

def removeFirstNDigits(num: Int, n: Int): Int = {
  num.toString.drop(n).toInt
}

In this method, we convert the integer to a string and use the String.drop() method to remove the first n characters. However, we should consider what happens if n exceeds the length of the number.

3. Handling Edge Cases

To account for potential issues, we can modify our function to return 0 if n is greater than or equal to the length of the number:

def removeFirstNDigitsLimit(num: Int, n: Int): Int = {
  val numString = num.toString
  if (n >= numString.length) 
    0 
  else 
    numString.drop(n).toInt
}

This version checks the length of the string representation of the number and safely handles the scenario where n is too large.n

4. Example Usage

Let’s look at some examples to see how our function behaves:

removeFirstNDigitsLimit(123456, 2) shouldBe 3456
removeFirstNDigitsLimit(123456, 5) shouldBe 6
removeFirstNDigitsLimit(123456, 6) shouldBe 0

These examples illustrate how our function effectively removes the specified number of digits.

5. Using Pattern Matching

An alternative approach could involve pattern matching to enhance readability and expressiveness:

def removeFirstNDigitsPatternMatching(num: Int, n: Int): Int = {
  val numString = num.toString
  numString match {
    case str if n >= str.length => 0
    case str => str.drop(n).toInt
  }
}

This method utilizes pattern matching to succinctly handle the edge case, making the code more elegant while retaining its functionality.

6. Negative Numbers

When dealing with negative numbers, we need to account for the presence of the negative sign. In our previous examples, the negative sign (-) is part of the number but does not get removed when using string manipulation. This means that if we naively apply drop(n) to a negative number, the negative sign will still be preserved in the result, potentially leading to incorrect behavior.

To handle negative numbers correctly, we can add a check to separate the sign from the magnitude of the number. Here’s how we can modify our function to deal with negative numbers:

def removeFirstNDigitsNegative(num: Int, n: Int): Int = {
  val numString = num.toString
  if (num < 0) {
    // For negative numbers, remove the first n digits but keep the negative sign
    val absNumString = numString.drop(1) // Drop the negative sign
    if (n >= absNumString.length) 0
    else ("-" + absNumString.drop(n)).toInt
  } else {
    // For positive numbers, apply the same logic as before
    if (n >= numString.length) 0
    else numString.drop(n).toInt
  }
}

Let’s see an example with this function:

removeFirstNDigitsNegative(-123456, 2) shouldBe -3456

7. Conclusion

In this article, we learned how to remove the first n digits from a number in Scala.

We explored different approaches, starting from a basic implementation to more robust solutions that handle edge cases. By using string manipulation and pattern matching, we effectively solved the problem while keeping the code clean and efficient.

     

Related Stories

• CSV Files in Scala
• Round a Number to the Nearest or Next Multiple of 10 in Scala
• Ways to Round Off Decimal Numbers in Scala