Byte Introduction

Get started with Spring Data

Skills:

MongoDB

Objective

Get started with MongoDB using Spring Data.

Background/Recap

Spring and Spring Boot are vast topics that you can spend weeks together to get a grasp of. You will be learning one portion of it here - How to access and manipulate data in Mongodb through the Spring Data project. If you don’t know anything about Spring / Spring Boot, please go through the Spring Boot byte before attempting this one.


Spring

In layman’s terms, it’s an application framework that helps you build Java applications with all bells and whistles really fast.


Spring Data

Spring Data is an umbrella project that contains many sub-projects, each of which intends to provide a Spring based programming model for data access through a variety of data sources. One of these sub-projects is for MongoDB. Few other sub-project examples are those that deal with Redis, Couchbase, and so on.


References

Context for this Byte

The context for this byte is a typical forum which has users, and these users all make posts on the forum. For the sake of this exercise, the content of the posts is randomized.

image alt text

You are given the forum data and must use Spring Data and its modules to achieve the milestones given.

Primary goals

  1. Understand how Spring Data for Mongo DB works

  2. Perform queries using MongoRepository

  3. Understand the difference between MongoRepository and MongoTemplate

Objective

Get started with MongoDB using Spring Data.

Background/Recap

Spring and Spring Boot are vast topics that you can spend weeks together to get a grasp of. You will be learning one portion of it here - How to access and manipulate data in Mongodb through the Spring Data project. If you don’t know anything about Spring / Spring Boot, please go through the Spring Boot byte before attempting this one.


Spring

In layman’s terms, it’s an application framework that helps you build Java applications with all bells and whistles really fast.


Spring Data

Spring Data is an umbrella project that contains many sub-projects, each of which intends to provide a Spring based programming model for data access through a variety of data sources. One of these sub-projects is for MongoDB. Few other sub-project examples are those that deal with Redis, Couchbase, and so on.


References

Context for this Byte

The context for this byte is a typical forum which has users, and these users all make posts on the forum. For the sake of this exercise, the content of the posts is randomized.

image alt text

You are given the forum data and must use Spring Data and its modules to achieve the milestones given.

Primary goals

  1. Understand how Spring Data for Mongo DB works

  2. Perform queries using MongoRepository

  3. Understand the difference between MongoRepository and MongoTemplate

Download the source code

Create ~/workspace/bytes/ directory and cd to it


mkdir -p ~/workspace/bytes/

cd ~/workspace/bytes/

Download the source code to the ~/workspace/bytes/ directory using one of the following commands:


git clone https://gitlab.crio.do/crio_bytes/me_byte_springdata.git

git clone git@gitlab.crio.do:crio_bytes/me_byte_springdata.git

Load the data

The repository has a data.json file which is a data dump you need to load into MongoDB. To do this, use these commands:


cd ~/workspace/bytes/me_byte_springdata

mongoimport --db=forumdb --type=json --file=data.json --collection=users --jsonArray

Tip

You can poke around the tables and documents using the mongo shell if you like. If you want to know more about using MongoDB, you can also take up the MongoDB Byte.

Test the server

Can you move to your repo directory and start the server now using the following command? It may take 1-2 minutes when you run it the first time as a lot of dependencies get downloaded.


cd ~/workspace/bytes/me_byte_springdata

./gradlew bootRun

It will print the forum stats if you now visit http://localhost:8081/stats on your browser preview. It can be found on the left side at the bottom, the icon is shown in the screenshot. Clicking it opens an embedded browser window as shown in the screenshot as well.

image alt textimage alt text

or if you prefer the terminal, open the embedded terminal and use curl. You should get a HTML response.


# type in terminal

curl http://localhost:8081/stats

You are now running a REST API server which will respond back to your API requests.

Before you begin

It is recommended that you become familiar with the file and folder structure before you begin. Spend some time opening files and simply reading the code. It is not necessary to understand what is happening. It is to be able to remember the files, classes and perhaps method names that are used. Let’s take a look at some of the important files:

  • MainApplication.java: The main file, serves as the entry point to the application. It also has the code which starts the spring backend server.

  • ForumController.java: Controller file that intercepts the incoming requests, processes them and sends it onwards to the service layer.

  • ForumService.java and ForumServiceImpl.java: Files in the service layer. These two files simply forward the request to the forum’s repository layer.

  • ForumRepositoryService.java and ForumRepositoryServiceImpl.java: Repository layer files that issue queries to MongoDB. It also takes care of converting objects from entity objects to DTO objects.

  • UserRepository.java: It is the class through which interaction with the database occurs as you will soon see.

What just happened at the end of the previous milestone?

The forum controller prints the "Hello World" output but also a statistic on the number of users in the forum. Dig into the controller class and see where this number came from.

Open the file - src/main/java/com/crio/springdatabyte/controller/ForumController.java and look for this snippet of code.


@GetMapping("/stats")
public ResponseEntity<String> stats() {
 Stats stats = service.getForumStats();
 String message = "<p>Hello!</p>"
     + "<div><b>Forum stats</b></div>"
     + "<div>---------------</div>"
     + "<div>Number of users: " + stats.getNumUsers() + "</div>";

 return ResponseEntity.ok().body(message);
}

What is happening is that when we request the /stats url, it is intercepted by the method above and the output is sent to the browser. So the natural question to ask is where are the stats coming from?

The line responsible for that is service.getForumStats(). A quick peek into ForumServiceImpl.java will lead you to ForumRepositoryServiceImpl.java. This is where you can see the code that talks to the database.



@Override

public Stats getForumStats() {

 List<UserEntity> users = userRepository.findAll();


 return new Stats(users.size());

}


This time, the line of interest is userRepository.findAll().


And once more, you can take a quick peek into the UserRepository.java file to see the implementation, except for one problem… There’s no implementation!


Unlike the other times where there was an interface and a corresponding implementation, for the repository class, there is only an interface. Where does the implementation come from?


Spring Data provides it. The interface acts as a marker to help you capture the types you need to work with. And the method name is parsed by Spring to generate the query internally. If you observe the interface, it extends MongoRepository<UserEntity, Integer>.


The "types" here are UserEntity and Integer. UserEntity is the entity to manage (The Entity is a presentation of the row or document in the table or collection in your database). Integer is the id of the managed entity object, in this case it is an integer. Also note, the entity class must be marked with @Document for MongoDB to signify to Spring that it must manage it.


The other big advantage is that this repository pattern is an abstraction for different databases. This means, you can use a technology specific abstraction for different databases. The one you are using is MongoRepository, but you can use JpaRepository for relational databases such as MySQL.


The method name you have invoked is the findAll() method Spring Data knows you have invoked the findAll() method, from the mongo repository class, then it generates the appropriate query automatically and sends it to the database that the server connects to using the details in application.properties file.


Phew! That’s a lot of things that are happening under the hood.


That is precisely the advantage of using this pattern. Spring manages the hard work of query generation, connecting and disconnecting to databases and so on so that you can simply invoke one method and continue to focus on the business logic.


So much for the introduction. Next, you will see how to handle querying using this powerful abstraction.

References

Curious Cats

  • Is the method name findAll() fixed? What if you use findEverything(), can you try it out and see if it works? Why or why not? Can you search the documentation for CrudRepository and see how it relates to MongoRepository?

  • Entities managed for Mongo DB must use @Document. A "document" is a term specific for MongoDB, so does this mean a data source backed by a different technology uses a different annotation? Can you try to figure out what this would be for MySQL?

Querying

Previously, you looked at getting all users in the forum and got the count. What if you wanted to select only some users?


Since it has been established that the querying occurs by observing the method name and parameters, it stands to reason that if we tweak these two things, Spring will be able to generate different queries.


Let’s look at the code in the ForumRepositoryServiceImpl.java once again.


@Override
public List<Post> getPostsByUser(String username) {
 List<UserEntity> userEntityList = userRepository.findByUsername(username);

 List<Post> posts = new ArrayList<>();

 for(UserEntity userEntity : userEntityList) {
   List<PostModel> postModelEntityList = userEntity.getPosts();
   for(PostModel postModelEntity : postModelEntityList) {
     posts.add(modelMapper.map(postModelEntity, Post.class));
   }
 }

 return posts;
}

Look at the method this time. From the UserRepository interface, the signature is:


List<UserEntity> findByUsername(String username)


Let’s break this down:

  • The return type is a List.

  • The method name is different. The query parsing mechanism strips the prefix findBy and parses the rest of the method name. The By is a delimiter by which the rest of the criteria for the underlying query is determined. Generally, this criteria will be the entity properties. In this case, this is "username".

  • To use multiple properties, operators to concatenate the properties are used, such as And as well as Or.

  • So far, it has been established that username is the criteria, but you need a value for this username upon which the User collection can be scanned. This value is provided in the method parameter.

  • With that, Spring has enough information to generate the query and send it to MongoDB.

Once the user document is retrieved, posts are extracted out of it and returned to the service layer.

Tip

If you want to observe the output, you can request the url from the browser preview feature as you did to get the forum stats in module 1. What is the request url? Hint: Look at the controller methods and try to figure it out from there.

Curious Cats

  • What if the entity property is longer in words? For example, let’s assume there the entity has properties: birthPlace which signifies city of birth, birth which is for date of birth and place which is for current city of residence. How will you put this in a method that uses all three as criteria? Is the method name findByBirthPlaceBirthPlace correct?

  • Hint: All the method parsing algorithm needs is a way to differentiate between the criteria. The default is to use camel case as a delimiter, which means you have to use a different delimiter in the above method name. How to do this?

  • The repository method returns List<UserEntity>, but the username is generally unique in a forum. How would you change the method to return a single user document? Can you try it out?

Advanced querying

In this form, the method name is not parsed. Instead, you add a @Query annotation to the method and specify a MongoDB json query string.

For example, in the previous task, you used


List<UserEntity> findByUsername(String username);

as the method name which needed to be parsed. Using the query annotation route, its form can be changed to something that looks like this:


@Query("{username: ?0}")

List<UserEntity> findUsers(String username);


Let’s break this down:

  • The first thing to be observed is that the method name does not play a part here in generating the query here.

  • The username in the json query string is the entity property, and the ?0 is the value you want to give to the criteria. This value is taken from the method parameter. The "0" here indicates that it is the first parameter in the query method that should be used as the value. Similarly, you would expect to write ?1, ?2 and so on should more method parameters by used.

With the @Query annotation, you gain more power as it is more expressive but the tradeoff for this is that you lose the ease and convenience of the query methods from before. The other tradeoff is that there is a tighter coupling between the query and the underlying technology of the data store as we must know the query semantics of the store.


Let’s take a look at the implementation of the use case where this is utilized.

Getting all posts which have "cricket"

The objective is to get users whose posts contain the string "cricket" in their post content. To do this, you need to search for the string inside an array of posts for every user document. This is not straightforward to do using a query method. But we can turn to the expressive power of @Query annotation to solve this problem.

The code in the repository service is as follows:


@Override
public List<User> getUsersWithPostText(String text) {
  List<UserEntity> userEntityList = userRepository.findUsersWithPostText(text);

  List<User> userList = new ArrayList<>();
  for(UserEntity userEntity : userEntityList) {
    userList.add(modelMapper.map(userEntity, User.class));
  }

  return userList;
}

And the repository method with the annotation is:


@Query("{'posts.content': {$regex: ?0}}")
List<UserEntity> findUsersWithPostText(String content);

Of course, you are not limited to regex operators. You are only limited by the query capabilities offered by mongo. Thus the method annotation form offers pretty much the full expressive power of querying directly on MongoDB.

Curious Cats

  • How many posts contain the word "ipl"?

  • Can you try searching for posts that have a substring and order it by the time of post creation?

Why another form?

The third form of querying the database that Spring Data offers is MongoTemplate. With each increasing step you have taken, you have traded convenience for expressiveness and flexibility. MongoTemplate is the most powerful form, allowing you to construct dynamic, ad-hoc queries which offer granular control over the other forms.

Usage

Let’s go through a previous use case - get users by username.


// ... other code


@Autowired

MongoTemplate mongoTemplate


// ... other code

Query query = new Query();
query.addCriteria(Criteria.where("username").is(username));

// ...

mongoTemplate.findOne(query, User.class);


Let’s break down this usage:

  • There are three objects here: Query object, Criteria object, MongoTemplate object. The Query and Criteria objects have a fluent style API which makes it easy to chain together criteria and queries programmatically. This offers one big advantage over the other forms: You can construct queries dynamically. Previously, you were effectively limited to "hard-coded" queries which means if you query had to be changed based on some system condition, it was not possible unless there was already another query method that could be selected. With this chainable API, you can construct queries in an ad-hoc fashion.

  • Once the query is created, mongoTemplate can be used to execute the query. Notice the findOne method. If you remember your mongo queries, it should be familiar. There are a whole host of methods MongoTemplate has to help you beside this one.

Which one to use?

The answer as always is: It depends. If your queries do not change much, or are relatively straightforward you may prefer query methods. If you want full flexibility to write your own ad-hoc queries, MongoTemplate may be more preferable. You could also go for a mixed approach through proper layering and decoupling, where you could start off with the relatively simple repository query methods and then start adopting MongoTemplate as and when more expressive queries are required.

Curious Cats

  • Try to change the method implementation of getPostsByUser in the repository service layer to use MongoTemplate.