I am a bit ashamed to admit this but until yesterday, I had no idea that I can add validation to a REST API by using the @Valid and the @RequestBody annotations.

This was not working in Spring MVC 3.0 and for some reason I had not noticed that the support for this was added in Spring MVC 3.1.

I never liked the old approach because I had to

1. Inject the Validator and MessageSource beans to my controller so that I can validate the request and fetch the localized error messages if the validation fails.
2. Call the validation method in every controller method which input must be validated.
3. Move the validation logic into a common base class which is extended by the controller classes.

When I noticed that I don’t have to do these things anymore, I decided to write this blog post and share my findings with all of you.

Let’s start by taking a look at the DTO class used in this blog post. The source code of the CommentDTO class looks as follows:

Let’s move on and find out how we can add validation to a REST API with Spring MVC 3.1.

Spring MVC 3.1 Is a Good Start

We can add validation to our REST API by following these steps:

1. Implement a controller method and ensure that its input is validated.
2. Implement the logic which handles validation errors.

Both of the steps are described in the following subsections.

Implementing the Controller

We can implement our controller by following these steps:

1. Create a class called CommentController and annotate this class with the @Controller annotation.
2. Add an add() method to the CommentController class which takes the added comment as a method parameter.
3. Annotate the method with @RequestMapping and @ResponseBody annotations.
4. Apply the @Valid and @RequestBody annotations to the method parameter.
5. Return the added comment.

The source code of the CommentController class looks as follows:

We have now added a new method to our controller and added validation to it. When the validation fails, the MethodArgumentNotValidException is thrown. Let’s find out how we can return a meaningful response to the user of our API when the validation fails.

Handling Validation Errors

We can implement the logic which handles the validation errors by following these steps:

1. Implement the data transfer objects which contains the information returned to the user of our REST API.
2. Implement the exception handler method.

These steps are described with more details in the following.

Creating the Data Transfer Objects

First, we have to create the data transfer objects which contains the information returned to the user of our REST API. We can do this by following these steps:

1. Create a DTO which contains the information of a single validation error.
2. Create a DTO which wraps those validation errors together.

Let’s get started.

The source code of the first DTO looks as follows:

The implementation of the second DTO is rather simple. It contains a list of FieldErrorDTO objects and a method which is used to add new field errors to the list. The source code of the ValidationErrorDTO looks as follows:

The following listing provides an example Json document which is send back to the user of our API when the validation fails:

Let’s see how we can implement the exception handler method which creates a new ValidationErrorDTO object and returns created object.

Implementing the Exception Handler Method

We can add the exception handler method to our controller by following these steps:

1. Add a MessageSource field to the CommentController class. The message source is used to fetch localized error message for validation errors.
2. Inject the MessageSource bean by using constructor injection.
3. Add a processValidationError() method to the CommentController class. This method returns ValidationErrorDTO object and takes a MethodArgumentNotValidException object as a method parameter.
4. Annotate the method with the @ExceptionHandler annotation and ensure that the method is called when the MethodArgumentNotValidException is thrown.
5. Annotate the method with the @ResponseStatus annotation and ensure that the HTTP status code 400 (bad request) is returned.
6. Annotate the method with the @ResponseBody annotation.
7. Implement the method.

Let’s take a closer look at the implementation of the processValidationError() method. We can implement this method by following these steps:

1. Get a list of FieldError objects and process them.
2. Process the field errors one field error at the time.
3. Try to resolve a localized error message by using the MessageSource object, current locale and the error code of the processed field error.
4. Return the resolved error message. If the error message is not found from the properties file, return the most accurate field error code.
5. Add a new field error by calling the addFieldError() method of the ValidationErrorDTO class. Pass the name of the field and the resolved error message as method parameters.
6. Return the created ValidationErrorDTO object after each field error has been processed.

The source code of the CommentController class looks as follows:

That is it. Let’s spend a moment to evaluate what we have just done.

We Are Almost There

We have now added validation to our REST API with Spring MVC 3.1. This implementation has one major benefit over the old approach:

We can trigger the validation process by using the @Valid annotation.

However, the methods annotated with the @ExceptionHandler annotation will be triggered only when the configured exception is thrown from the controller class which contains the exception handler method. This means that if our application has more than one controller, we have to create a common base class for our controllers and move the logic which handles the validation errors to that class. This might not sound like a big deal but we should prefer composition over inheritance.

Spring MVC 3.2 provides the tools which we can use to remove the need of inheritance from our controllers. Let’s move on and find out how this is done.

Spring MVC 3.2 to the Rescue

Spring MVC 3.2 introduced a new @ControllerAdvice annotation which we can use to implement an exception handler component that processes the exceptions thrown by our controllers. We can implement this component by following these steps:

1. Remove the logic which handles validation errors from the CommentController class.
2. Create a new exception handler class and move the logic which processes validation errors to the created class.

These steps are explained with more details in the following subsections.

Removing Exception Handling Logic from Our Controller

We can remove the exception handling logic from our controller by following these steps:

1. Remove the MessageSource field from the CommentController class.
2. Remove the constructor from our controller class.
3. Remove the processValidationError() method and the private methods from our controller class.

The source code of the CommentController class looks as follows:

Our is next step is to create the exception handler component. Let’s see how this is done.

Creating the Exception Handler Component

We can create the exception handler component by following these steps:

1. Create a class called RestErrorHandler and annotate it with the @ControllerAdvice annotation.
2. Add a MessageSource field to the RestErrorHandler class.
3. Inject the MessageSource bean by using constructor injection.
4. Add the processValidationError() method and the required private methods to the RestErrorHandler class.

The source code of the RestErrorHandler class looks as follows:

We Are Finally There

Thanks to Spring MVC 3.2, we have now implemented an elegant solution where the validation is triggered by the @Valid annotation, and the exception handling logic is moved to a separate class. I think that we can call it a day and enjoy the results of our work.

Summary

This blog post has taught us that

• If we want to add validation to a REST API when we are using Spring 3.0, we have to implement the validation logic ourself.
• Spring 3.1 made it possible to add validation to a REST API by using the @Valid annotation. However, we have to create a common base class which contains the exception handling logic. Each controller which requires validation must extend this base class.
• When we are using Spring 3.2, we can trigger the validation process by using the @Valid annotation and extract the exception handling logic into a separate class.

The example application of this blog are available at Github (Spring 3.1 and Spring 3.2)

In the earlier parts of my Spring Data Solr tutorial, we have implemented a simple search function which is used to search the information of todo entries. The current implementation of our search function shows all search results in a single page. This is not a viable solution for most real life applications because the number of search results can be so large that search function is no longer usable.

This blog post provides us the solution to that problem by describing how we can paginate the query results or our search function with Spring Data Solr.

This blog post is divided into five sections:

• The first section describes how we can request the correct page manually and talks about the different return types of query methods.
• The second section describes how we can obtain the search result count by adding a custom method to our repository.
• The third section describes how we can paginate the search results of query methods.
• The fourth section teaches us to paginate the search results of dynamic queries.
• The fifth and last section describes how we can configure and use a technique called web pagination.

Note: These blog posts provides additional information which helps us to understand the concepts of described in this blog post:

• Running Solr with Maven
• Spring Data Solr Tutorial: Introduction to Solr
• Spring Data Solr Tutorial: Configuration
• Spring Data Solr Tutorial: Query Methods
• Spring Data Solr Tutorial: Adding Custom Methods to a Single Repository
• Spring Data Solr Tutorial: Dynamic Queries
• Spring Data Solr Tutorial: Sorting

Let’s get started.

Few Minutes of Theory

Before we will start making modifications to our example application, we will take a short look at the theory behind pagination. This section is divided in two subsections which are described in the following:

• The first section describes how we can specify the pagination options of our query.
• The second section describes the different return types of a query method.

Let’s move on.

Specifying the Wanted Page

The used pagination options are specified by using the PageRequest class which implements the Pageable interface.

The typical pagination requirements are given in the following:

• Get the query results belonging to a single page.
• Get the query results belonging to a single page when the query results are sorted by using the value of a single field.
• Get the query results belonging to a single page when the query results are sorted by using the values of multiple fields and the sort order of different fields is the same.
• Get the query results belonging to a single page when the query results are sorted by using the values of multiple fields and the sort order of different fields is not the same.

Let’s find out how we can create the PageRequest objects which fulfill the given requirements.

First, we must create a PageRequest object which specifies that we want to get the query results belonging to a single page. We can create the PageRequest object by using the following code:

Second, we must create a PageRequest object which specifies that we want to get the results belonging to a single page when query results are sorted by using the value of a single field. We can create the PageRequest object by using the following code:

Third, we must create a PageRequest object which specifies that we want to get the results belonging to a single page when the query results are sorted by using multiple fields and the sort order of different fields is same. We can create the PageRequest object by using the following code:

Fourth, we must create a PageRequest object which specifies that want to get the query results belonging to a single page when the query results are sorted by using multiple fields and the sort order of different fields is not same. We can create this object by using the following code:

We now know how we can create new PageRequest objects. Let’s move on and talk about the different return types of query methods.

Deciding the Return Type of a Query Method

When a query method uses pagination, it can have two return types. These return types are described in the following (We will assume that the name of our model class is TodoDocument):

• When we are interested about the pagination metadata, the return type of our query method must be Page<TodoDocument> (Get more information about the Page interface which declares the methods used to obtain the pagination metadata).
• When we are not interested about the pagination metadata, the return type of our query method should be List<TodoDocument>.

Getting the Search Result Count

Before we can start paginating the search results of our queries, we have to implement a function which is used to get the number of todo entries which matches with given search criteria. This number is required so that we can implement the pagination logic to the frontend.

We can implement this function by following these steps:

1. Add a custom method to our repository. This method is used to return the search result count.
2. Create a new service method which uses our custom repository method.

These steps are described with more details in the following subsections.

Adding a Custom Method to Our Repository

At the moment it is not possible to create a count query without adding a custom method to a repository. We can do this by following these steps:

1. Create a custom repository interface.
2. Implement the custom repository interface.
3. Modify the actual repository interface.

Let’s move on and find out how this is done.

Creating a Custom Repository Interface

We can create a custom repository interface by following these steps:

1. Create an interface called CustomTodoDocumentRepository.
2. Add a count() method to the created interface. This method takes the used search term as a method parameter.

The source code of the CustomTodoDocumentRepository interface looks as follows:

Implementing the Custom Repository Interface

We can implement the custom repository interface by following these steps:

1. Create a class called TodoDocumentRepositoryImpl and implement the CustomTodoDocumentRepository interface.
2. Annotate the class with the @Repository annotation.
3. Add SolrTemplate field to the class and annotate the field with the @Resource annotation.
4. Implement the count() method.

Let’s take a closer look at the implementation of the count() method. We can implement this method by following these steps:

1. Get words of the given search term.
2. Construct the used search criteria by calling the private constructSearchConditions() method and pass the words of the search term as a method parameter.
3. Create the executed query by creating a new SimpleQuery object and pass the created Criteria object as a constructor parameter.
4. Get the search result count by calling the count() method of the SolrTemplate class and pass the created SimpleQuery object as a method parameter.
5. Return the search result count.

The source code of the TodoDocumentRepositoryImpl class looks as follows:

Modifying the Actual Repository Interface

We can make the custom count() method visible to the users of our repository by extending the CustomTodoRepositoryInterface. The source code of the TodoDocumentRepository looks as follows:

Using the Custom Repository Method

We can use the created repository method by following these steps:

1. Modify the TodoIndexService interface.
2. Implement the modified interface.

These steps are described with more details in the following.

Note: We have to make other changes as well but I will not describe these changes here since they are not related to Spring Data Solr.

Modifying the Service Interface

We have to modify the TodoIndexService interface by adding a new countSearchResults() method to it. This method takes the used search term as a method parameter and returns the search result count. The source code of the TodoIndexService interface looks as follows:

Implementing the Modified Interface

We can implement the countSearchResults() method by following these steps:

1. Add the countSearchResults() method to the RepositoryTodoIndexService class.
2. Obtain the search result count by calling the custom repository method and return the search result count.

The relevant part of the RepositoryTodoIndexService class looks as follows:

Paginate the Query Results of Query Methods

When we are creating our queries by using query methods, we can paginate the query results by following these steps:

1. Add a new Pageable parameter to the query method. This parameter specifies the details of the fetched page.
2. Modify the service layer by adding a new Pageable parameter to the search() method of the TodoIndexService interface.

Let’s get started.

Modifying the Repository Interface

We can add pagination support to our repository by adding a Pageable parameter to the query method which is used to build the executed query. Let’s take a look at the declarations of our query methods.

Query Generation from Method Name

When the executed query is created by using the query generation from method name strategy, we have to add a Pageable parameter to the findByTitleContainsOrDescriptionContains() method of the TodoDocumentRepository interface. These source code of our repository interface looks as follows:

Named Queries

When are using named queries, we have to add a Pageable parameter to the findByNamedQuery() method of the TodoDocumentRepository interface. The source code of the TodoDocumentRepository interface looks as follows:

@Query Annotation

When the executed query is created by using the @Query annotation, we have to add a Pageable parameter to the findByQueryAnnotation() method of the TodoDocumentRepository interface. The source code of our repository interface looks as follows:

Modifying the Service Layer

We have to the following modifications to the service layer of our example application:

1. Add a Pageable parameter to the search() method of the TodoIndexService interface.
2. Implement the new search() method.

Note: We have to make other changes as well but I will not describe these changes here since they are not related to Spring Data Solr.

The source code of the TodoIndexService interface looks as follows:

We can use the modified query methods by making the following changes to the search() method of the RepositoryIndexService class:

1. Get the paginated query results by calling the query method of our repository and pass the used search term and the Pageable object as method parameters.
2. Return the query results.

Let’s move and take a look at different implementations of the search() method.

Query Generation from Method Name

When we are building our queries by using the query generation from method name strategy, we can get query results belonging to a specific page by using the findByTitleContainsOrDescriptionContains() method of the TodoDocumentRepository interface.

The relevant part of the RepositoryTodoIndexService class looks as follows:

Named Queries

When we are using named query for building the executed query, we can get the search results belonging to a specific page by using the findByNamedQuery() method of the TodoDocumentRepository interface.

The relevant part of the RepositoryTodoIndexService class looks as follows:

@Query Annotation

When we are building our query by using the @Query annotation, we can get the search results which belongs to a specific page by calling the findByQueryAnnotation() method of the TodoDocumentRepository interface.

The source code of the RepositoryTodoIndexService class looks as follows:

Paginating the Query Results of Dynamic Queries

We can paginate the query results of dynamic queries by following these steps:

1. Add a Pageable parameter to the search() method of our custom repository.
2. Modify the service layer by adding a Pageable parameter to the search() method of the TodoIndexService interface.

These steps are described with more details in the following subsections.

Changing the Custom Repository

We have to add pagination support to our custom repository. We can do this by following these steps:

1. Modify the custom repository interface by adding a Pageable parameter to its search() method.
2. Change the implementation of the search() method by adding pagination support to it.

Let’s move on and find out how this is done.

Changing the Custom Repository Interface

We have to add a Pageable parameter to the search() method declared in the CustomTodoDocumentRepository interface. The source code of our custom repository interface looks as follows:

Implementing the Custom Repository Method

Our next step is to add pagination support to the implementation of the search() method. We can implement the search() method of the TodoDocumentRepositoryImpl class by following these steps:

1. Get the words of the search term.
2. Construct the used search criteria by calling the private createSearchConditions() method and passing the words of the search term as a method parameter.
3. Create the executed query by creating a new SimpleQuery object and pass the created Criteria object as a constructor parameter.
4. Set the pagination options of the query by calling the setPageRequest() method of the SimpleQuery class. Pass the Pageable object as a method parameter.
5. Get the search results by calling the queryForPage() method of the SolrTemplate class. Pass the created query and the type of the expected return objects as method parameters.
6. Return the search results by calling the getContent() method of the Page interface.

The source code of the TodoDocumentRepositoryImpl class looks as follows:

Using the Custom Repository

Before we can use the modified repository method, we have to make the following changes to the service layer of our example application:

1. Add a Pageable parameter to the search() method of the TodoIndexService interface.
2. Implement the search() method.

These steps are described with more details in following.

Note: We have to make other changes as well but I will not describe these changes here since they are not related to Spring Data Solr.

Modifying the Service Interface

We have to add a Pageable parameter to the search() method of the TodoIndexService interface. The source code of the TodoIndexService looks as follows:

Implementing the Service Interface

When we are building our by using the criteria API of Spring Data Solr, we can get the query results by calling the search() method of our custom repository and passing the user search term and the Pageable object as method parameters.

The source code of the RepositoryTodoIndexService class looks as follows:

Using Web Pagination

One question is still left unanswered. That is question is:

Where the pagination options used to paginate the query results of our queries are specified?

We will create the pagination options of our queries by using a technique called web pagination. This technique is based on a custom argument resolver class called PageableArgumentResolver. This class parses pagination information from HTTP request and makes it possible to add a Pageable method parameter to controller methods.

This section describes how we can configure and use this technique in our example application. It is divided into three subsections:

• The first subsection describes how we can configure the PageableArgumentResolver class.
• The second subsection describes how we can use it.
• The last subsection talks about the pros and cons of web pagination.

Let’s find out how we can use this technique in our example application.

Configuration

This subsection describes how we can configure the PageableArgumentResolver class which is to used extract pagination options from HTTP requests. Let’s find out how we do this by using Java configuration and XML configuration.

Java Configuration

We can add a custom argument argument resolver by making the following changes to the ExampleApplicationContext class:

1. Override the addArgumentResolvers() method of the WebMvcConfigurerAdapter class.
2. Implement the addArgumentResolvers() method by creating new PageableArgumentResolver object and adding the created object to the list of argument resolvers which is given as a method parameter.

The relevant part of the ExampleApplicationContext class looks as follows:

XML Configuration

We can configure a custom argument resolver by making the following changes to the exampleApplicationContext.xml file:

1. Use the argument-resolvers element of the mvc namespace for configuring the custom argument resolvers.
2. Configure the PageableArgumentResolver bean inside the argument-resolvers element.

The relevant part of the exampleApplicationContext.xml file looks as follows:

Usage

After we have configured the PageableArgumentResolver class by using one of methods which were described earlier, we can add Pageable method parameters to our controller methods. The search() method the TodoController class is a good example of this. The relevant part of its source code looks as follows:

However, adding the Pageable argument to the controller method is not enough. We still have to add the pagination options to the HTTP request. This is done by adding special request parameters to the request. These request parameters are described in the following:

• The page.page request parameter specifies the requested page.
• The page.size request parameter specifies the page size.
• The page.sort request parameter specifies the property which is used to sort the query results.
• The page.sort.dir request parameter specifies the sort order.

Let’s spend a moment and think about the pros and cons of web pagination.

Pros and Cons

The web pagination has both pros and cons which we should be aware of before making the decision to use it in our applications. Let’s find out what these are.

Pros

Using web pagination has one major benefit:

It is an easy and simple to transfer pagination options from the web layer to the repository layer. All we have to do is to configure a custom argument resolver, add a Pageable parameter to a controller method and send the pagination options by using specific request parameters. This is a lot simpler than processing pagination options in our code and manually creating a PageRequest object.

Cons

The cons of using web pagination are described in the following:

• Web pagination creates a dependency between the web layer and Spring Data. This means that the implementation details of the repository layer leaks into to the upper layers of our application. Although purists will probably claim that this is huge mistake, I don’t share their opinion. I think that abstractions should make our life easier, not harder. We must also remember that the law of leaky abstractions states that all-non trivial abstractions, to some degree, are leaky.
• One real disadvantage of web pagination is that we can use it only if our search results are sorted by using a single field. Although this is perfectly fine for most use cases, there are situations when this becomes a problem. If this happens, we have to process pagination options manually.

Summary

We have now added the pagination of search results to our example application. This tutorial has taught us the following things:

• We learned to create new PageRequest objects.
• We learned that we can select the return type of our query method from two different options.
• We learned to paginate the query results of query methods and dynamic queries.
• We know how we can use web pagination, and we are aware of its pros and cons.

The next part of my Spring Data Solr Tutorial describes how we can add custom methods to all Spring Data Solr repositories.

P.S. The example applications of this blog post are available at Github (query methods and dynamic queries).

The previous parts of my Spring MVC Test tutorial have described how we can write integration tests for a REST API. Although the techniques described in those blog posts are useful, the problem is that our assertions were not very elegant. Our assertions were basically ensuring that the body of the HTTP response contained the “right” strings.

This approach has two problems:

• It is not very readable, especially if the returned JSON is large. Since our tests should act as a documentation for our code, this is a huge problem.
• It is very hard to write tests which ensure that the ordering of collections is correct without sacrificing readability.

Luckily for us, there is a better way to do this. JsonPath is to JSON what XPath is to XML. It provides an easy and readable way to extract parts of a JSON document. This blog post describes how we can write assertions by using the Spring MVC Test and the Java implementation of JsonPath.

Note: These blog posts describe the structure of our example application and teaches us to write integration tests to it without using the JsonPath library (This blog post skips the details described in these blog posts):

• Integration Testing of Spring MVC Applications: REST API, Part One
• Integration Testing of Spring MVC Applications: REST API, Part Two
• Integration Testing of Spring MVC Applications: Security
• Integration Testing of Spring MVC Applications: Migrating to Spring 3.2

Let’s get started and find out how we can get the required dependencies with Maven.

Getting Required Dependencies with Maven

We can get the required dependencies with Maven by following these steps:

1. Declare the Hamcrest dependency (version 1.3) in the pom.xml file.
2. Declare the JUnit dependency (version 4.11) in the pom.xml file and exclude the hamcrest dependencies.
3. Declare the Spring Test (version 3.2.2.RELEASE) dependency in the pom.xml file.
4. Add JsonPath dependencies (version 0.8.1) to the pom.xml file.

The relevant dependency declarations looks as follows:

Let’s move and find out how we can write assertions by using JsonPath.

Writing Integration Tests

We can write integration tests by executing the tested operation and making assertions against the returned JSON document. We can create a new assertion by following these steps:

1. Create a JsonPath expession which fetches the preferred part from the returned JSON document (Get more information about the JsonPath notation).
2. Make an assertion against the fetched part by using a Hamcrest matcher.
3. Use the jsonPath() method of MockMvcResultMatchers class to verify that the assertion is true and pass the objects created in phases one and two as method parameters.

Enough with theory. Let’s move on and find out how we can write assertions against JSON documents. The following subsections describe how we can write assertions against JSON documents which contains either the information of a single object or the information of multiple objects.

Single Object

This subsection describes how we can ensure that the JSON document which contains the information of a single object is correct. As an example we will write an integration test for a controller method which is used to delete the information of an existing todo entry. After the todo entry has been deleted successfully, its information is returned back to the client. The returned JSON looks as follows:

We can write the integration test by following these steps:

1. Use the @ExpectedDatabase annotation to ensure that the todo entry is deleted.
2. Perform a DELETE request to url ‘/api/todo/1′. Set the logged in user.
3. Verify that the returned HTTP status code is 200.
4. Verify that the content type of the response is ‘application/json’ and its character set is ‘UTF-8′.
5. Get the id of the deleted todo entry by using the JsonPath expression $.id and verify that the id is 1.
6. Get the description of the deleted todo entry by using the JsonPath expression $.description and verify that the description is is “Lorem ipsum”.
7. Get the title of the deleted todo entry by using the JsonPath expression $.title and verify that the title is “Foo”.

The source code of our integration test looks as follows:

Collection of Objects

This subsection describes how we can write assertions which ensure that the JSON document containing a collection of objects is correct. We will take a look at two different situations:

• The objects are always returned in the same order.
• The objects are returned in a random order.

Let’s continue our journey.

Objects Returned in the Same Order

When the user wants to get all todo entries which are stored in the database, the entries are always returned in the same order. The returned JSON looks as follows:

We can write our integration test by following these steps:

1. Use the @ExpectedDatabase annotation to verify that no changes is made to the database.
2. Perform a GET request to url ‘/api/todo’. Set the logged in user.
3. Verify that the returned HTTP status code is 200.
4. Verify that the content type of the response is ‘application/json’ and its character set ‘UTF-8′.
5. Fetch the collection of todo entries by using JsonPath expression $ and ensure that two todo entries are returned.
6. Use the JsonPath expressions $[0].id, $[0].description and $[0].title to get the id, description and title of the first todo entry. Verify that its information is correct.
7. Use the JsonPath expressions $[1].id, $[1].description and $[1].title to get the id, description and title of the second todo entry. Verify that its information is correct.

The source code of our integration test looks as follows:

Objects Returned in a Random Order

When a validation of an added or updated todo entry fails, our example application returns the field errors back to the client of our REST API. The problem is that we cannot guarantee the order in which the fields are validated. This means that the field errors are returned in a random order. One JSON document which contains the returned field errors looks as follows:

We can write an write an integration test, which verifies that field errors are returned when a new todo entry which contains invalid information is added, by following these steps:

1. Use the @ExpectedDatabase annotation to verify that no changes are made to the database.
2. Create the title and description of the todo entry. Ensure that both the title and description are too long.
3. Create a new TodoDTO object and set its title and description.
4. Perform a POST request to the url ‘/api/todo’. Set the content type of the request to ‘application/json’. Set the character set of the request to ‘UTF-8′. Transform the created object into a correct format and send it in the body of the request. Set the logged in user.
5. Verify that the content type of the response is ‘application/json’ and its character set is ‘UTF-8′.
6. Fetch the field errors by using the JsonPath expression $.fieldErrors and ensure that two field errors are returned.
7. Use the JsonPath expression $.fieldErrors[*].path to fetch all available paths. Ensure that field errors about title and description fields are available.
8. Use the JsonPath expression $.fieldErrors[*].message to fetch all available error messages. Ensure that error messages concerning title and description fields are returned.

The source code of our integration test looks as follows:

Summary

We have now written integration tests to a REST API by using Spring MVC Test and JsonPath. This blog post has taught us four things:

• We learned how we can get the required JsonPath dependencies with Maven.
• We learned how we can write assertions against the JSON representation of a single object.
• We learned how we can write assertions against the JSON representation of a collection of objects.
• We learned that writing assertions with JsonPath improves the readability of our tests.

The example application of this blog post is available at Github.

P.S. You might want to read the other parts of my Spring MVC Test tutorial.

When we are implementing a word search function, we typically want to sort the search results in descending order by using the relevancy of each search result. This is also the default behaviour of Solr.

However, there are situations when it makes to sense to specify the sort order manually. One such situation is an implementation of a “regular” search function which was discussed in the previous part of my Spring Data Solr tutorial.

This blog post describes how we can sort our query results with Spring Data Solr. To be more specific, we have to modify the search function of our example application to sort the search results in descending order by using the value of the id field.

This blog post is divided into three sections:

• The first section describes how we can specify the sorting options used in our queries.
• The second section describes how we can sort our query results when we are building our queries by using query methods.
• The third section teaches us to sort the query results of dynamic queries.

Let’s get started.

Note: These blog posts provide additional information which helps us to understand the concepts described in this blog post:

• Running Solr with Maven
• Spring Data Solr Tutorial: Introduction to Solr
• Spring Data Solr Tutorial: Configuration
• Spring Data Solr Tutorial: Query Methods
• Spring Data Solr Tutorial: Adding Custom Methods to a Single Repository
• Spring Data Solr Tutorial: Dynamic Queries

Specifying the Sort Options of a Query

The sort options of a query are specified by using the Sort class. The typical requirements for sorting query results are given in the following:

• Sort the query results by using the value of a single field.
• Sort the query results by using the values of multiple fields when the sort order of different fields is the same.
• Sort the query results by using the values of multiple fields when sort order of different fields is not the same.

Let’s take a look how we can create a Sort object which fulfills the given requirements.

First, we must create a Sort object which specifies that query results are sorted by using a single field. Lets assume that we want to sort the query results in ascending order by using the id field. We can create the Sort object by using the following code:

Second, we must create a Sort object which states that query results are sorted by using the values of multiple fields when the sort order of different fields is the same. Lets assume that we have to sort the query results in descending order by using the id and description fields. We can create the Sort object by using the following code:

Third, we want to sort the query results by using the values of multiple fields when the sort order of different fields is not the same. Lets assume that we want to sort the query results in descending order by using the description field and in ascending order by using the id field. We can create the Sort object by using the following code:

We now know how we can create new Sort objects. Let’s move on and put this theory into practice.

Sorting the Query Results of Query Methods

When we are building our queries by using query methods, we can sort the query results by following these steps:

1. Add a new Sort parameter to the query method. This method parameter specifies the used sort options.
2. Create a new Sort object in the service layer and pass it as a method parameter when the query method is called.

Let’s move on and find out how this is done.

Modifying the Repository Interface

We can sort the query results of our query by adding a new Sort parameter to the our query method. This method parameter specifies the sort options of the executed query. Let’s move on and take a look at the declarations of our query methods.

Query Generation From Method Name

When the executed query is created by using the query generation from method name strategy, we have to add a Sort parameter to the findByTitleContainsOrDescriptionContains() method of the TodoDocumentRepository interface. The source code of our repository interface looks as follows:

Named Queries

When the executed query is created by using named queries, we have to add a Sort parameter to the findByNamedQuery() method of the TodoDocumentRepository interface. The source code of our repository interface looks as follows:

Note: This approach does not work if we are using Spring Data Solr RC1 because of a known bug. We have to either use the build snapshot dependency or wait for the release of RC2.

The @Query Annotation

When the executed query is created by using the @Query annotation, we have to add a Sort parameter to the findByQueryAnnotation() method of the TodoDocumentRepository interface. The source code of our repository interface looks as follows:

Note: This approach does not work if we are using Spring Data Solr RC1 because of a known bug. We have to either use the build snapshot dependency or wait for the release of RC2.

Using the Query Method

We can use the modified query method by making the following changes to the search() method of the RepositoryIndexService class:

1. Create a private sortByIdDesc() method which specifies that the query results are sorted in descending order by using the id of the document.
2. Get the sorted query results by calling the query method declared in the TodoDocumentRepository interface.
3. Return the query results.

Let’s move on and take a look at the different implementations of the search() method.

Query Generation From Method Name

When we are building our queries by using the query generation from method name strategy, we can get the query results by using the findByTitleContainsOrDescriptionContains() method of the TodoDocumentRepository interface.

The source code of the relevant part of RepositoryTodoIndexService class looks as follows:

Named Queries

When we are building our queries by using named queries, we can get the query results by using the findByNamedQuery() method of the TodoDocumentRepository interface.

The relevant part of the RepositoryTodoIndexService looks as follows:

The @Query annotation

When we are building our queries by using the @Query annotation, we can get the query results by using the findByQueryAnnotation() method of the TodoDocumentRepository interface.

The relevant part of the RepositoryTodoIndexService class looks as follows:

Sorting the Query Results of Dynamic Queries

Because dynamic queries are created by adding a custom method to a repository interface, the steps required to sort the query results of a dynamic query have no effect to the service layer of our example application.

We can sort the query results of dynamic queries by making the following changes to the implementation of our custom repository interface:

1. Add a private sortByIdDesc() method to the TodoDocumentRepositoryImpl class. This method returns a Sort object which specifies that the query results are sorted in descending order by using the id of the document.
2. Modify the search() method of the TodoDocumentRepositoryImpl class. Set the sort options to the executed query by using the addSort() method of the Query interface and pass the created Sort object as a method parameter.

The relevant part of the TodoDocumentRepositoryImpl class looks as follows:

Summary

We have now learned how we can sort query results with Spring Data Solr. This tutorial has taught us three things:

• We know that we can specify the used sort options by using the Sort class.
• We learned that we can sort the query result of query methods by adding a new method parameter to the query method.
• We learned that we can set the sort options to a dynamic query by using the addSort() method of the Query interface.

The next part of my Spring Data Solr tutorial describes how we can paginate the query results of our queries.

P.S. The example applications of this blog posts are available at Github (query methods and dynamic queries).

Solr is often referred as a search server which we can use when we are implementing full-text search functions. However, it is often wise to leverage the performance of Solr when we are implementing a search function which takes its input from a search form.

In this scenario, the executed search query depends from the received input. This means that the number of query parameters depends from the input entered to the search form. In other words, the executed search query is dynamic.

The previous part of my Spring Data Solr tutorial taught us how we can add custom methods to a single repository. It is time put this information in to use and find out how we can create dynamic queries with Spring Data Solr.

Let’s get started.

Note: These blog entries provide additional information which helps us to understand the concepts described in this blog post:

• Running Solr with Maven
• Spring Data Solr Tutorial: Introduction to Solr
• Spring Data Solr Tutorial: Configuration
• Spring Data Solr Tutorial: CRUD (Almost)
• Spring Data Solr Tutorial: Adding Custom Methods to a Single Repository

Creating Dynamic Queries

This section describes how we can create dynamic queries with Spring Data Solr. It is divided into two subsections which are described in the following:

• The first subsection describes the basics which we need to know before we can start working on the actual search function.
• The second subsection describes how we can implement the search function of our example application by adding a custom method to our Spring Data Solr repository.

Learning the Basics

Before we can start implementing the search function of our example application, we need to know how we can create queries “manually” by using Spring Data Solr. We can create a query “manually” by following these steps:

1. Create the search criteria.
2. Create the query which holds the used search criteria.
3. Execute the created query.

These steps are described with more details in the following.

Creating the Search Criteria

First, we have to create the search criteria for our query. We can do this by using the criteria classes which are described in the following:

• The org.springframework.data.solr.core.query.SimpleStringCriteria class is a basic criteria class which is used to specify the executed query by using already formatted query string. The query string specified in this class is executed as is. Thus, this class cannot be used to build dynamic queries.
• The org.springframework.data.solr.core.query.Criteria is a criteria class which is used to build dynamic queries. It has a fluent API which supports the chaining of multiple Criteria objects.

Creating the Executed Query

Second, we have to create the executed query. The query classes of Spring Data Solr are described in the following:

• The org.springframework.data.solr.core.query.SimpleQuery class is a query class which supports both pagination and grouping.
• The org.springframework.data.solr.core.query.SimpleFacetQuery class is a query class supporting faceted search.
• The org.springframework.data.solr.core.query.SimpleFilterQuery class is query class which supports filter queries.

Executing the Created Query

Third, we have to execute the created query. The SolrTemplate class implements several methods which we can use for this purpose. These methods are described in the following:

• The long count(final SolrDataQuery query) method returns the number of documents found with the query given as a method parameter.
• The UpdateResponse delete(SolrDataQuery query) method deletes the documents which match with the query given as a method parameter and returns an UpdateResponse object.
• The T queryForObject(Query query, Class<T> clazz) method returns a single document which matches with the query given as a method parameter.
• The FacetPage<T> queryForFacetPage(FacetQuery query, Class<T> clazz) method executes a facet query against Solr index and returns the query results as a FacetPage object.
• The Page<T> queryForPage(Query query, Class<T> clazz) method executes the query against Solr index and returns the query results as an implementation of the Page interface.

Let’s move on and put this theory into practice.

Implementing the Search Function

The requirements of our search function are following:

• The search function must return all todo entries which name or description contains some word of the given search term. In other words, if the search term is “Foo Bar”, our search function must return todo entries which title or description contains either “Foo” or “Bar”.
• The search must be case insensitive.

Because our search function is not static, we have to create it by using a dynamic query. We can create dynamic queries with Spring Data Solr by adding custom method to our Spring Data Solr repository. In other words, we have to follow these steps:

1. Create a custom interface which declares the added method.
2. Implement the created interface.
3. Modify the repository interface to extend the created interface.

These steps are described with more details in the following.

Creating the Custom Interface

First, we have to create a custom interface which declares our custom search method. We can do this by following these steps:

1. Create an interface called CustomTodoDocumentRepository.
2. Declare the search() method. This method takes the used search term as a method parameter and returns a list of TodoDocument objects.

The source code of the CustomTodoDocumentRepository interface looks as follows:

Implementing the Created Interface

Second, we have to implement the custom interface which we created earlier. We can do this by following these steps:

1. Create a class called TodoDocumentRepositoryImpl and implement the CustomTodoDocumentRepository interface.
2. Annotate the class with the @Repository annotation.
3. Add SolrTemplate field to the class and annotate it with the @Resource annotation.
4. Implement the search() method.

The implementation of the search() method requires a more detailed description which is given here. We can implement the search() method by following these steps:

1. Get the words of the search term.
2. Construct the used search criteria by calling the private createSearchConditions() method and passing the words of the search term as a method parameter. This method creates the used search criteria by using the API of the Criteria class.
3. Create the executed query by creating a new SimpleQuery object and pass the created Criteria object as a constructor parameter.
4. Get the search results by calling the queryForPage() method of the SolrTemplate class. Pass the created query and the type of the expected result objects as method parameters.
5. Return the search results by calling the getContent() method of the Page interface.

The source code of the TodoDocumentRepositoryImpl class looks as follows:

Modifying the Repository Interface

Third, we have to make our custom search() method visible to the users of our repository. We can do this by extending the CustomTodoDocumentRepository interface. The source code of the TodoDocumentRepository interface looks as follows:

We have now added a custom search() method to our Spring Data Solr repository. Let’s find out how we can use this method.

Using the Custom Method

We can use the custom method by modifying the search() method of the RepositoryTodoIndexService class. The new implementation of this method is very simple. It gets the search results by calling the search() method of our Spring Data Solr repository and returns the search results.

The source code of the RepositoryTodoIndexService class looks as follows:

Summary

We have now implemented a dynamic search function with Spring Data Solr. Although our search function was rather simple, we should now be able to implement more complex queries as well.

This tutorial has taught us two things:

• We learned how we can create queries “manually” by using Spring Data Solr.
• We learned that we have to implement dynamic search methods by adding custom method to a single repository.

The next part of my Spring Data Solr tutorial describes how we can sort our query results.

P.S. The example application of this blog post is available at Github.

The previous part of my Spring Data Solr tutorial taught us how we can create static queries by using query methods. The natural next step would be describe how we can create dynamic queries with Spring Data Solr. However, before we can move on to that subject, we have to understand how we can add custom methods to a single repository.

This blog entry will help us to understand how that is done.

During this blog entry we will modify our example application to update the information of a todo entry to the Solr index by using a technique called partial update.

Let’s start by taking a closer look at Solr’s partial update feature.

Note: These blog entries provide additional information which helps us the understand the concepts described in this blog entry:

• Running Solr with Maven
• Spring Data Solr Tutorial: Introduction to Solr
• Spring Data Solr Tutorial: Configuration
• Spring Data Solr Tutorial CRUD (Almost)

What is Partial Update?

The partial update feature was introduced in Solr 4.0 and it gives us the possibility to select the fields which are updated. This can be very useful if it is slow to index the content of the whole document.

However, the partial update function has its limitations. If we want to use the partial update function, we have to store all fields of the document which increases the size of the Solr index. The reason for this is that it is not possible to do a partial update to Lucene index. Lucene always deletes the old document before indexing the new one. This means that if the fields which are not updated are not stored, the values of these fields are lost when a partial update is done to a document.

It is our job to decide which one is more important to us: speed or the size of the index.

We can get more information about partial update by checking out the following resources:

• Solr 4.0: Partial documents update
• Solr – Partial Update of Documents @ StackOverflow
• Solr – update a new field to existing document @ StackOverflow

Let’s move and learn how we can add custom methods to a single Spring Data Solr repository.

Adding Custom Methods a Single Repository

We can add custom methods to a single repository by following these steps:

1. Create a custom interface which declares the custom methods.
2. Implement the custom interface.
3. Modify the repository interface to extend the custom interface.

These steps are described with more details in the following subsections.

Creating the Custom Interface

First, we have to create an interface and declare the custom methods in it. We can do this by following these steps:

1. Create an interface called PartialUpdateRepository.
2. Declare the custom methods.

Because we have to declare only one custom method which is used to update the information of a todo entry, the source code of the PartialUpdateRepository interface looks as follows:

Implementing the Custom Interface

Second, we have to implement the PartialUpdateRepository interface. The repository infrastructure tries to auto detect the classes which implements the custom repository interfaces by using the following rules:

• The implementation of a custom repository interface must be found from the same package than the custom interface.
• The name of the class which implements a custom repository interface must be created by using the following formula: [The name of the actual repository interface][The repository implementation postfix].

The default value of the repository implementation postfix is ‘Impl’. We can overwrite the default value by using one of the following methods:

• If we are using Java configuration, we can configure the used postfix by setting the preferred postfix as the value of the repositoryImplementationPostfix attribute of the @EnableSolrRepositories annotation.
• If we are using XML configuration, we can configure the used postfix by setting the preferred postfix as the value of the repository-impl-postfix attribute of the repositories namespace element.

The example application of this blog entry uses the default configuration. Thus, we can implement the PartialUpdateRepository interface by following these steps:

1. Create a class called TodoDocumentRepositoryImpl.
2. Annotate the class with the @Repository annotation.
3. Add SolrTemplate field to the class and annotate this field with the @Resource annotation.
4. Implement the update() method.

Let’s take a closer look at the implementation of the update() method. We can implement this method by following these steps:

1. Create a new PartialUpdate object. Set the name of the document’s id field and its value as constructor arguments.
2. Set the names and values of updated fields to the created object.
3. Do a partial update by calling the saveBean() method of the SolrTemplate class.
4. Commit changes by calling the commit() method of the SolrTemplate class.

The source code of the TodoRepositoryImpl class looks as follows:

Modifying the Repository Interface

We can make the custom update() method visible to the users of our repository by extending the PartialUpdateRepository interface. The source code of TodoDocumentRepository interface looks as follows:

Let’s move on and find out how we can use our new repository method.

Using the Custom Method

We can use the custom update() method by making the following changes to our example application:

1. Add update() method to the TodoIndexService interface.
2. Implement the update() method.
3. Modify the update() method of the RepositoryTodoService class to use the new method.

These steps are described with more details in the following subsections.

Adding New Method to TodoIndexService Interface

As we remember, the TodoIndexRepository interface declares methods which are used to add information to the Solr index, search information from it and remove documents from the index.

We have to add a new method to this interface. This method is called update() and it takes the updated Todo object as a method parameter. The source code of the TodoIndexRepository interface looks as follows:

Implementing the Added Method

We can implement the update() method of the TodoIndexService interface by following these steps:

1. Add the update() method to the RepositoryIndexService class and annotate the method with the @Transactional annotation. This ensures that our Spring Data Solr repository will participate in Spring managed transactions.
2. Call the update() repository method and pass the updated Todo object as a method parameter.

The source code of the RepositoryTodoIndexService class looks as follows:

Modifying the RepositoryTodoService Class

Our last step is to modify the update() method of the RepositoryTodoService class to use the new update() method which is declared in the TodoIndexService interface. The relevant parts of the RepositoryTodoService class looks as follows:

Summary

We have now added a custom method to a single Spring Data Solr repository and implemented an update function which uses the partial update feature of Solr. This tutorial has taught us two things:

• We know how we can add custom methods to a single Spring Data Solr repository.
• We know that we can use partial update only if all fields of our document are stored (The value of the stored attribute is true).

The next part of my Spring Data Solr tutorial describes how we can use the skills learned from this blog entry for creating dynamic queries with Spring Data Solr.

P.S. The example application of this blog entry is available at Github.

We have learned how we can configure Spring Data Solr. We have also learned how we can add new documents to the Solr index, update the information of existing documents and delete documents from the Solr index. Now it is time to move forward and learn how we can search information from the Solr index by using Spring Data Solr.

The requirements of our search function are given in the following:

• The search function must return all todo entries which title or description contains the given search term.
• The search must be case insensitive.

We can implement the search function by following these steps:

1. Create a query method.
2. Use the created query method.

Let’s move on and find out how we can implement the search function by using query methods.

Note: These blog entries provide additional information which helps us the understand the concepts described in this blog entry:

• Running Solr with Maven
• Spring Data Solr Tutorial: Introduction to Solr
• Spring Data Solr Tutorial: Configuration
• Spring Data Solr Tutorial CRUD (Almost)

Creating the Query Method

Query methods are methods which are

1. added to the repository interface.
2. used to specify the search query which is executed when the query method is called.
3. used to build static queries (queries which have always the same amount of query parameters).

We can create query methods with Spring Data Solr by using the following techniques:

• Query generation from the method name
• Named queries
• @Query annotation

These techniques are described with more details in the following subsections.

Query Generation from the Method Name

The query generation from the method name is a query generation strategy where the executed query is parsed from the name of the query method.

1. The name of the query method must start with a special prefix which identifies the query method. These prefixes are: find, findBy, get, getBy, read and readBy. This prefix is stripped from the method name when the executed query is parsed.
2. Property expressions are used to refer to the properties of our document class.
3. Special keywords are used together with property expressions to specify the operators used in the created query. These keywords are added to the name of the query method after a property expression.
4. We can combine property expressions by adding either And or Or keyword between them.
5. The parameter count of the query method must be equal with the number of property expressions used in its name.

We can get more information about the property expressions and the repository keywords by reading the following resources:

• Spring Data Solr Reference Manual: Query Creation
• Spring Data Solr Reference Manual: Repository Query Keywords

As we remember, our search function must return all todo entries which title or description contains the given search term. Also, our document class has two properties which we are using in this query. These properties are called title and description. We can create the method name which fulfils the requirements of our search function by following these steps:

1. Add the findBy prefix to start of the method name.
2. Add the property expression of the title property after the prefix.
3. Add the Contains keyword after the property expression.
4. Add the Or keyword to the method name.
5. Add the property expression of the description property after the Or keyword.
6. Add the Contains keyword to the method name.
7. Add two method parameters to our query method. The first parameter matched against the title property and second one is matched against the description property.

The source code of the TodoDocumentRepository interface looks as follows:

Note: This query method will not work if the search term contains more than one word.

Named Queries

Named queries are queries which are declared in a separate properties file and wired to the correct query method. The rules concerning the properties file used to declare the named queries are described in the following:

• The default location of the properties file is META-INF/solr-named-queries.properties but we can configure the location by using the namedQueriesLocation property of the @EnableSolrRepositories annotation.
• The key of each named query is created by using the following formula: [The name of the document class].[The name of the named query].

Named queries which are configured in the properties files are matched with the query methods of our repository interface by using the following rules:

• The name of query method which executes the named query must be same than the name of the named query.
• If the name of the query method is not the same than the name of the named query, the query method must be annotated with the @Query annotation and the name of the named query must be configured by using the name property of the @Query annotation.

We can implement query method with named queries by following these steps:

1. Specify the named query.
2. Create the query method.

These steps are described with more details in the following.

Specifying the Named Query

We can create a named query which fulfils the requirements of our search function by following these steps:

1. Create the properties file which contains the named queries.
2. Create a key for the named query by using the formula described earlier. Since the name of our document class is TodoDocument, the key of our named query is TodoDocument.findByNamedQuery.
3. Create the named query by using the Lucene query parser syntax. Because our query must return documents which title or description contains the given search term, our query is: title:*?0* OR description:*?0* (The ?0 is replaced with the value of the query method’s first parameter).

The content of the META-INF/solr-named-query.properties file looks as follows:

Creating the Query Method

We can create the query method which uses the created named query by following these steps:

1. Add findByNamedQuery() method to the TodoDocumentRepository interface. This method returns a list of TodoDocument objects and takes a single String parameter called searchTerm.
2. Annotate the method with the @Query annotation and set the value of its name property to ‘TodoDocument.findByNamedQuery’. This step is not required since the name of the query method is the same than the name of the named query but I wanted to demonstrate the usage of the @Query annotation here.

The source code of the TodoDocumentRepository interface looks as follows:

@Query Annotation

The @Query annotation can be used to specify the query which is executed when a query method is called. We can create the query method which fulfils the requirements of our search function by following these steps:

1. Add findByQueryAnnotation() method to the TodoDocumentRepository interface. This method returns a list of TodoDocument objects and it has a single String parameter called searchTerm.
2. Annotate the method with the @Query annotation.
3. Set the executed query as the value of the @Query annotation. We can create the query by using the Lucene query parser syntax. Because our query must return documents which title or description contains the given search term, the correct query is: title:*?0* OR description:*?0* (The ?0 is replaced with the value of the query method’s first parameter).

The source code of the TodoDocumentRepository interface looks as follows:

Using the Created Query Method

We can use the created query method by following these steps:

1. Declare the search() method in the TodoIndexService interface.
2. Add the implementation of the search() method to the RepositoryTodoIndexService class.

These steps are described with more details in the following subsections.

Declaring the Search Method

Our first step is to declare the search method in the TodoIndexService interface. The relevant part of the TodoIndexService interface looks as follows:

Implementing the Search Method

Our second step is to implement the search() method. Our implementation is rather simple. It obtains a list of TodoDocument objects by calling the correct method of the TodoDocumentRepository interface and returns that list.

The name of the query method depends from the technique used to create the query method. The different implementations are described in the following.

Query Generation from the Method Name

When we are generating the query from the name of the query method, our implementation obtains a list of TodoDocument objects by calling the findByTitleContainsOrDescriptionContains() method of the TodoDocumentRepository interface and returns that list.

The relevant part of the RepositoryTodoIndexService class looks as follows:

Named Queries

If we are using named queries, our implementation gets a list of TodoDocument objects by calling the findByNamedQuery() method of the TodoDocumentRepository interface and returns that list.

The relevant part of the RepositoryTodoIndexService class looks as follows:

@Query Annotation

When we are using the @Query annotation, our implementation obtains a list of TodoDocument objects by calling the findByQueryAnnotation() method of the TodoDocumentRepository interface and returns that list.

The relevant part of the RepositoryTodoIndexService class looks as follows:

Selecting the Correct Query Creation Technique

The obvious question is:

What is the best way to add query methods to our Spring Data Solr repositories?

We can use the following guidelines when we are deciding the correct query creation technique for our query method:

• If the created query is very simple, query generation from the method name is often the best choice. The problem of this approach is that implementing “complex” queries with this approach leads into long and ugly method names.
• It is a good idea to keep our queries near our query methods. The benefit of using the @Query annotation is that we can see the executed query and our query method by reading the source code of our repository interface.
• If we want to separate the executed queries from our repository interface, we should use named queries. The problem of this approach is that we have to check the executed query from the properties file which is quite cumbersome.

Summary

We have now learned how we can create static queries with Spring Data Solr. This blog entry has taught us two things:

• We know what query method methods are and how we can create them.
• We are familiar with the different query creation techniques and we know when to use them.

The example application which demonstrates the concepts described in this blog entry is available at Github. In the next part of this tutorial we will learn how we can add custom functionality to a single repository.

In the previous part of my Spring Data Solr tutorial, we learned how we can configure Spring Data Solr. Now it is time to take a step forward and learn how we can manage the information stored in our Solr instance. This blog entry describes how we add new documents to the Solr index, update the information of existing documents and delete documents from the index.

We can make the necessary modifications to our example application by following these steps:

1. Create a document class which contains the information stored in the Solr index.
2. Create a repository interface for our Spring Data Solr repository.
3. Create a service which uses the created repository.
4. Use the created service.

These steps are described with more details in the following sections.

Note: These blog entries provide additional information which helps us to understand the concepts described in this blog entry:

• Running Solr with Maven
• Spring Data Solr Tutorial: Introduction to Solr
• Spring Data Solr Tutorial: Configuration

Creating the Document Class

The first step is to create a document class which contains the information added to the Solr index. A document class is basically just a POJO which is implemented by following these rules:

• The @Field annotation is used to create a link between the fields of the POJO and the fields of the Solr document.
• If the name of the bean’s field is not equal to the name of the document’s field, the name of the document’s field must be given as a value of the @Field annotation.
• The @Field annotation can be applied either to a field or setter method.
• Spring Data Solr assumes by default that the name of the document’s id field is ‘id’. We can override this setting by annotating the id field with the @Id annotation.
• Spring Data Solr (version 1.0.0.RC1) requires that the type of the document’s id is String.

More information:

• Solrj @ Solr Wiki

Let’s move on and create our document class.

In the first part of my Spring Data Solr tutorial, we learned that we have to store the id, description and title of each todo entry to the Solr index.

Thus, we can create a document class for todo entries by following these steps:

1. Create a class called TodoDocument.
2. Add the id field to the TodoDocument class and annotate the field with the @Field annotation. Annotate the field with the @Id annotation (This is not required since the name of the id field is ‘id’ but I wanted to demonstrate its usage here).
3. Add the description field to the TodoDocument class and annotate this field with the @Field annotation.
4. Add the title field to the TodoDocument and annotate this field with the @Field annotation.
5. Create getter methods to the fields of the TodoDocument class.
6. Create a static inner class which is used to build new TodoDocument objects.
7. Add a static getBuilder() method to the TodoDocument class. The implementation of this method returns a new TodoDocument.Builder object.

The source code of the TodoDocument class looks as follows:

Creating the Repository Interface

The base interface of Spring Data Solr repositories is the SolrCrudRepository<T, ID> interface and each repository interface must extend this interface.

When we extend the SolrCrudRepository<T, ID> interface, we must give two type parameters which are described in the following:

• The T type parameter means the type of our document class.
• The ID type parameter means the type of the document’s id. Spring Data Solr (version 1.0.0.RC1) requires that the id of a document is String.

We can create the repository interface by following these steps:

1. Create an interface called TodoDocumentRepository.
2. Extend the SolrCrudRepository interface and give the type of our document class and its id as type parameters.

The source code of the TodoDocumentRepository interface looks as follows:

Creating the Service

Our next step is to create the service which uses the created Solr repository. We can create this service by following these steps:

1. Create a service interface.
2. Implement the created interface.

These steps are described with more details in the following.

Creating the Service Interface

Our service interface declares two methods which are described in the following:

• The void addToIndex(Todo todoEntry) method adds a todo entry to the index.
• The void deleteFromIndex(Long id) method deletes a todo entry from the index.

Note: We can use the addToIndex() method for adding new todo entries to the Solr index and updating the information of existing todo entries. If an existing document has the same id than the new one, the old document is deleted and the information of the new document is saved to the Solr index (See SchemaXML @ Solr Wiki for more details).

The source code of the TodoIndexService interface looks as follows:

Implementing the Created Interface

We cam implement the service interface by following these steps:

1. Create a skeleton implementation of our service class.
2. Implement the method used to add documents to the Solr index.
3. Implement the method used to delete documents from the Solr index.

These steps are described with more details in the following.

Creating a Skeleton Implementation of the Service Class

We can create a skeleton implementation of our service interface by following these steps:

1. Create a class called RepositoryTodoIndexService and annotate this class with the @Service annotation. This annotation marks this class as a service and ensures that the class will be detected during the classpath scanning.
2. Add a TodoDocumentRepository field to the RepositoryTodoIndexService class and annotate that field with the @Resource annotation. This annotation instructs the Spring IoC container to inject the actual repository implementation the service’s repository field.

The source code of our dummy service implementation looks as follows:

Adding Documents to the Solr Index

We can create the method which adds new documents to the Solr index by following these steps:

1. Add the addToIndex() method to the RepositoryTodoIndexService class and annotate this method with the @Transactional annotation. This ensures that our Spring Data Solr repository will participate in Spring managed transactions.
2. Create a new TodoDocument object by using the builder pattern. Set the id, title and description of the created document.
3. Add the document to the Solr index by calling the save() method of the TodoDocumentRepository interface.

The source code of the created method looks as follows:

Deleting Documents from the Solr Index

We can create a method which deletes documents from the Solr index by following these steps:

1. Add the deleteFromIndex() method to the RepositoryTodoDocumentService class and annotate this method with the @Transactional annotation. This ensures that our Spring Data Solr repository will participate in Spring managed transactions.
2. Delete document from the Solr index by calling the delete() method of the TodoDocumentRepository interface.

The source code of the created method looks as follows:

Using the Created Service

Our last step is to use the service which we created earlier. We can do this by making the following modifications to the RepositoryTodoService class:

1. Add the TodoIndexService field to the the RepositoryTodoService class and annotate this field with the @Resource annotation. This annotation instructs the Spring IoC container to inject the created RepositoryTodoIndexService object to the service’s indexService field.
2. Call the addToIndex() method of the TodoIndexService interface in the add() method of the RepositoryTodoService class.
3. Call the deleteFromIndex() method of the TodoIndexService interface in the deleteById() method of the RepositoryTodoService class.
4. Call the addToIndex() method of the TodoIndexService interface in the update() method of the RepositoryTodoService class.

The source code of the RepositoryTodoService looks as follows:

Summary

We have successfully created an application which adds documents to the Solr index and deletes documents from it. This blog entry has taught us the following things:

• We learned how we can create document classes.
• We learned that we can create Spring Data Solr repositories by extending the SolrCrudRepository interface.
• We learned that Spring Data Solr assumes by default that the name of the document’s id field is ‘id’. However, we can override this setting by annotating the id field with the @Id annotation.
• We learned that at the moment Spring Data Solr (version 1.0.0.RC1) expects that the id of a document is String.
• We learned how we can add documents to the Solr index and delete documents from it.
• We learned that Spring Data Solr repositories can participate in Spring managed transactions.

The next part of my Spring Data Solr tutorial describes how we can search information from the Solr index by using query methods.

P.S. The example application of this blog entry is available at Github.

In the previous part of my Spring Data Solr tutorial, we learned that Solr provides a REST-like HTTP API which can be used to add information to Solr index and execute queries against indexed data. The problem is that running a separate Solr instance in a development environment is a bit cumbersome.

However, not all hope is lost because Solr provides two alternative server implementations which we can use in our applications. These implementations are described in the following:

• The embedded Solr server connects directly to Solr core. We can use this server for development purposes but we must also remember that using it in production environment is not recommended. However, using the embedded Solr server is still a viable option in the development environment.
• The HTTP Solr server connects to an external Solr server by using HTTP. This is the recommended way of using the Solr search server and that is why we should always use it in the production environment.

This blog entry describes how we can get the required dependencies with Maven. We also learn to configure the Spring Data Solr to use the embedded Solr server in the development environment and the HTTP Solr server in the production environment.

Note: These blog entries provides additional information which helps us to understand the concepts described in this blog entry:

• Running Solr with Maven
• Spring Data Solr Tutorial: Introduction to Solr

Let’s get started.

Getting the Required Dependencies with Maven

We can get the required dependencies with Maven by following these steps:

1. Add the Spring Milestone Maven repository to the POM file.
2. Add the required dependencies to the pom.xml file.

Both of these steps are described with more details in the following.

Adding the Spring Milestone Maven Repository to the POM File

We can add the Spring milestone Maven repository to our POM file by adding the following XML to the pom.xml file:

Adding the Required Dependencies to the POM File

We can add the required dependencies to the POM file by following these steps:

1. Add the Spring Data Solr dependency (version 1.0.0.RC1) to the dependencies section of our POM file.
2. Add the Solr core dependency (version 4.1.0) to the dependencies section of our POM file and exclude the SLF4J JDK14 binding. Because Solr core is required by the embedded Solr server, we can skip this step if we are not using the embedded Solr server.

We can complete these steps by adding the following XML to the dependencies section of the POM file:

Configuring Spring Data Solr

This section describes how we can configure Spring Data Solr to use different Solr servers in the development and production environment. We will use the embedded Solr server in the development environment and the HTTP Solr server in the production environment.

We can configure Spring Data Solr by following these steps:

1. Create a properties file.
2. Configure the embedded Solr server.
3. Configure the HTTP Solr server.
4. Set the active bean definition profile.

These steps are described with more details in the following subsections.

Creating the Properties File

The name of our properties file is application.properties and we will use it to configure two properties which are described in the following:

• The solr.server.url property specifies the url of the used Solr server. The value of this property is used to configure the HTTP Solr server which is used in the production environment.
• The solr.solr.home configures the home directory of Solr. The value of this property is used to configure the home directory of the embedded Solr server which is used in the development environment.

The content of the application.properties file looks as follows:

Configuring the Embedded Solr Server

This subsection describes how we can configure Spring Data Solr to use the embedded Solr server in the development environment.

Java Configuration

We can create a configuration class which configures the embedded Solr server by following these steps:

1. Create a class called EmbeddedSolrContext and annotate that class with the @Configuration annotation.
2. Enable Spring Data Solr repositories by annotating that class with the @EnableSolrRepositories annotation and configuring the root package of our Solr repositories.
3. Annotate the created class with the @Profile annotation and set its value to ‘dev’. This means that this configuration class is bypassed unless the ‘dev’ profile have been activated.
4. Annotate the class with the @PropertySource annotation and set its value to ‘classpath:application.properties’. This configures the location of our property file and adds a PropertySource to Spring’s Environment.
5. Add an Environment field to the class and annotate that field with the @Resource annotation. The injected Environment is used to access the properties which we added to our properties file.
6. Create a method called solrServerFactoryBean() and annotate this method with the @Bean annotation. The implementation of this method creates a new EmbeddedSolrServerFactoryBean object, sets the value of the Solr home and returns the created object.
7. Create a method called solrTemplate() and annotate this method with the @Bean annotation. The implementation of this method creates a new SolrTemplate object and passes the used SolrServer implementation as a constructor argument.

The source code of the EmbeddedSolrContext class looks as follows:

XML Configuration

We can create an XML configuration file for the embedded Solr server by following these steps:

1. Configure the used properties file by using the property-placeholder element of the context namespace.
2. Enable Solr repositories and configure the base package of our Solr repositories by using the repositories element of the solr namespace.
3. Create a bean configuration for the development profile.
4. Configure the embedded Solr server bean by using the embedded-solr-server element of the solr namespace. Set the value of the Solr home.
5. Configure the Solr template bean. Set the configured embedded Solr server bean as constructor argument.

The contents of the exampleApplicationContext-solr.xml file looks as follows:

Configuring the Http Solr Server

This subsection describes how we can configure Spring Data Solr to use the HTTP Solr server in the production environment.

Java Configuration

We can create a configuration class which configures the HTTP Solr server by following these steps:

1. Create a class called HttpSolrContext and annotate that class with the @Configuration annotation.
2. Enable Spring Data Solr repositories by annotating that class with the @EnableSolrRepositories annotation and configuring the root package of our Solr repositories.
3. Annotate the created class with a @Profile annotation and set its value to ‘prod’. This means that this configuration class is bypassed unless the ‘prod’ profile have been activated.
4. Annotate the class with the @PropertySource annotation and set its value to ‘classpath:application.properties’. This configures the location of our property file and adds a PropertySource to Spring’s Environment.
5. Add an Environment field to the class and annotate that field with the @Resource annotation. The injected Environment is used to access the properties which we added to our properties file.
6. Create a method called solrServerFactoryBean() and annotate this method with the @Bean annotation. The implementation of this method create a new HttpSolrServerFactoryBean object, sets the value of the Solr server url and returns the created object.
7. Create a method called solrTemplate() and annotate this method with the @Bean annotation. The implementation of this method creates a new SolrTemplate object and passes the used SolrServer implementation as a constructor argument.

The source code of the HttpSolrContext class looks as follows:

XML Configuration

We can create an XML configuration file for the HTTP Solr server by following these steps:

1. Configure the used properties file by using the property-placeholder element of the context namespace.
2. Enable Solr repositories and configure the base package of our Solr repositories by using the repositories element of the solr namespace.
3. Create a bean configuration for the production profile.
4. Configure the HTTP Solr server bean by using the solr-server element of the solr namespace. Set the url of the Solr server.
5. Configure the Solr template bean. Set the configured HTTP Solr server bean as a constructor argument.

The content of the exampleApplicationContext-solr.xml file looks as follows:

Setting the Active Bean Definition Profile

We can select the active bean definition profile by setting the value of the spring.profiles.active system variable. The allowed values of this system variable (in the context of our example application) are described in the following:

• We can configure our application to run in the development profile by setting the value of the spring.profiles.active system variable to ‘dev’.
• When we want configure our application to run in the production profile, we have to set the of the spring.profiles.active system variable to ‘prod’.

We can configure our example application to support both profiles by following these steps:

1. Add required profiles to the POM file.
2. Create the profile specific properties files for system properties.
3. Configure the Jetty Maven plugin.

These steps are described with more details in the following.

Adding the Required Profiles to the POM File

We can add the required profiles to our Maven build by following these steps:

1. Create a profile for development environment. Set the id of this profile to ‘dev’ and set the value of the build.profile.id property to ‘dev’.
2. Create a profile for the production environment. Set the id of this profile to ‘prod’ and set the value of the build.profile.id property to ‘prod’.

The configuration of our Maven profiles looks as follows:

Creating the Profile Specific Properties Files for System Properties

The profile specific properties files are located in the sub directories of the profiles directory. The name of each sub directory matches with the values of the build.profile.id properties configured in the pom.xml file.

We can create the profile specific properties files for system properties by following these steps:

1. Create a properties file called system.properties to the profiles/dev directory. This properties file contains the system properties of the development profile.
2. Create a properties file called system.properties to the profiles/prod directory. This properties file contains the system properties of the production profile.

The content of the properties file used to configure the system properties of the development profile looks as follows:

The content of the properties file used to configure the system properties of the production profile looks as follows:

Configuring the Jetty Maven Plugin

We can configure the Jetty Maven plugin by following these steps:

1. Add the plugin declaration of the Jetty Maven plugin to the plugins section of our Pom file.
2. Configure the stopKey and stopPort of the Jetty Maven plugin.
3. Configure the location of the properties file containing the used system properties.

The configuration of the Jetty Maven plugin looks as follows:

Summary

We have now successfully obtained the required dependencies with Maven and configured Spring Data Solr. This blog entry has taught us four things:

• We learned to get the required dependencies with Maven.
• We know that we should use the embedded Solr server only in the development environment and learned how we can configure Spring Data Solr to use it.
• We learned that we should always use the HTTP Solr server in the production environment and know how we can configure Spring Data Solr to use it.
• We know how we can use the bean definition profiles of Spring Framework for creating different configurations for development and production environment.

The next part of my Spring Data Solr tutorial describes how we can add new document to Solr index, update the information of an existing documents and delete documents from the Solr index.

PS. The example application of this blog entry is available at Github.

Most of the applications must have a some kind of a search function. The problem is that search functions are often huge resource hogs and they can kill the performance of our application by causing heavy load to the database. That is why transferring that load to an external search server is a great idea.

This is the first part of my Spring Data Solr tutorial. During this tutorial we will implement a search function to a todo application which is the example application of my Spring MVC Test tutorial.

The requirements of our search function are simple. It must return a list of todo entries which title or description contains the used search term. The search result page must also provide a link to the page used to view the information of a todo entry.

Before we can start the implementation of our search function, we need to take a look at the Solr search server. This blog entry provides us the basic information about Solr and it is divided into three parts:

• The first part gives us a brief introduction to Solr and its data model.
• The second part describes how we can create a schema for our Solr instance.
• The last part describes how we can use the REST-like HTTP API provided by Solr.

Let’s get started.

A Brief Introduction to Solr

Let’s start by getting a brief introduction to the Solr search server. This introduction is very thin and it provides only the information we need to know in order to understand the implementation of our search function.

Also, even though Solr is heavily dependent from Lucene, this blog entry makes no difference between them.

This section describes

• The data model of Solr search server.
• What happens when new documents are added to Solr.
• What happens when a search query is executed against the indexed data.

The Data Model

An index consists of documents which are essentially a collection of fields. If we compare this data model to the data model of a relational database, we notice the following similarities:

• An index is roughly the same than a database table.
• A document is similar than a row of a database table.
• A field means the same than a column of a database table.

Each field of a document can be either indexed, stored or both. The meaning of these terms is described in the following:

• An indexed field is a field which is searchable and sortable. Indexed fields are not returned in the search results.
• A stored field is field which value returned in the search results.
• If a field is both indexed and stored, the field is both searchable and sortable. Its value is also returned in the search results.

Adding Information to the Index

When a new document is added to Solr, indexed and stored fields are processed in a different way. This difference is described in the following:

• Indexed fields undergo an analysis phase which typically breaks the text into words and apply different transformations to it. The results of this analysis phase are saved to Solr index.
• The values of stored fields are saved as is.

Searching Information from the Index

The search function can be divided into three steps which are described in the following:

• Typically the search query undergo a similar analysis phase than the indexed fields. The goal of this is to ensure that the search query matches with the content of the index.
• Solr uses its index to perform the search.
• The matching documents are returned in the requested format. Each document contains the values of its stored fields.

Creating the Schema

The schema is used to configure the following things:

• The fields of a document.
• How the fields of document are processed when a new document is added to the index.
• How the fields are dealt with when a search is executed against the index.

The schema is configured in a file called schema.xml, and we can create a schema for our application by following these steps:

1. Configure the used field types.
2. Configure the fields of our document.
3. Configure the copy fields.
4. Configure the unique key field of our document.

These steps are described with more details in the following subsections. The skeleton schema of our Solr instance looks as follows:

Note: This section describes the schema of the example application of my blog entry called Running Solr with Maven.

Configuring the Field Types

A field type specifies the following things:

• The data type of the field.
• How the information is analyzed when it is added to the index.
• How the information is processed when information is searched from the index.

We can configure a field type by using the fieldType element. Its attributes which are used in our schema are described in the following:

• The name attribute states the name of the field type. It is basically an alias which is used declare the type of a field.
• The class attribute declares the class that implements the field type in question.
• The sortMissingLast attribute specifies how the sorting works when the value of this field is missing. If the value of this attribute is set to ‘true’, the documents which don’t have value in the field in question are returned last.
• The positionIncrementGap attribute declares the amount of empty space which is put between multiple fields of the same document. The value of this attribute is used in multivalued fields and its idea is to prevent false matches across different fields.
• The precisionStep attribute is used in ranged queries of numeric fields. We can get more information about this by reading the API documentation of the NumericRangeQuery class.

We can configure the field types of our schema by following these steps:

1. Configure the Field Type for Long Fields
2. Configure the Field Type for String Fields
3. Configure the Field Type for Fields Containing Text

These steps are described with more details in the following.

Configuring the Field Type for Long Fields

Let’s start by configuring a simple field type for long fields which are not analyzed on the index or search phase. We can configure this field type by following these steps:

1. Set the name of the field type to ‘long’.
2. Set the name of the implementing class to ‘solr.TrieLongField’.
3. Set the value of the precisionStep attribute to zero.
4. Set the value of the positionIncrementGap attribute to zero.

Our field type declaration looks as follows:

Configuring the Field Type for String Fields

The next step is to configure a simple field type for string fields which are not analyzed on the index or search phase. We can configure this field type by following these steps:

1. Set the name of the field type to ‘string’.
2. Set the name of the implementing class to ‘solr.StrField’.
3. Set the value of the sortMissingLast attribute to ‘true’.

The declaration of our string field looks as follows:

Configuring the Field Type for Fields Containing Text

The last step is to configure the text_general field type. We can do this by following these steps:

1. Create a new field type. Set the name of the field type to ‘text_general’. Set the name of the implementing class to ‘solr.TextField’. Set the value of the positionIncrementGap to 100.
2. Create a new analyzer which is run at the index time. Configure that the text is splitted into words by using the word break rules of the Unicode Text Segmentation algorithm. Create a filter which removes words found from stopwords.txt file from the text. Transform text to lower case.
3. Create a new analyzer which is run at the query phase. Configure that the search query is splitted into words by using the word break rules of the Unicode Text Segmentation algorithm. Create a filter which removes words found from stopwords.txt file from the search query. Ensure that the synonyms found from synonyms.txt are applied. Transform the search query to lower case.

The declaration of the text_general field type looks as follows:

We can get more information about the analysis phase by reading the following documents:

• Analyzers, Tokenizers and Token Filters.

Configuring the Fields of Our Document

We can add new fields to our document by adding field elements to the schema.xml file. The field element has many attributes but at this point we need to understand the meaning of following attributes:

• The name attribute specifies the name of the field.
• The indexed attribute (true/false) specifies if the field is added to the search index. Only indexed fields are searchable and sortable.
• The stored attribute(true/false) specifies if the field should be returned in the search results.
• The multiValued (true/false) specifies if the field can appear in the document more than once.
• The type type attribute specifies the type of the field.
• The required (true/false) specifies whether the field is required or not.

In order to make maximize the performance of our Solr instance, we must follow the following guidelines:

• We should not store fields which are not required in the search results.
• We should not index the fields which are not used by our search function.

We can get more information about the optimal field configuration by reading the following documents:

• Field Options by Use Case
• Solr Performance Factors

We are now ready to configure the actual fields of our schema. Let’s first talk a bit about the information which we need show on the search result page. This information is described in the following:

• We need the id of the todo entry which is used to create a link to the view todo entry page.
• We need the title of the todo entry which is used as an anchor text of the created link.

When we know that our application must be able to search the contents of the title and description of a todo entry, we can add the required fields to our schema by following these steps:

1. Add a required field called ‘id’ to the schema and set its type to ‘string’. Ensure that this field is both indexed and stored. Set the value of the multiValued attribute to false.
2. Add a required field called ‘title’ to the schema and its type to ‘text_general’. Configure this field to be both indexed and stored. Set the value of the multivalued attribute to false.
3. Add a field called ‘description’ the schema sets its type to ‘text_general’. Configure this field to be indexed but not stored. Set the value of the multiValued attribute to false.
4. Add a field called ‘text’ and set its type ‘text_general’. Configure this field to be indexed but not stored. Set the value of its multiValue attribute to true. This field is a field which stores the content of all other indexed text fields.
5. Add a field called ‘_version_’ and sets its type ‘long’. Configure this field to be indexed and stored.

Our field declarations looks as follows: