1. Overview
With the large amount of data being logged, it’s important to mask sensitive details of the users while logging. In the new GDPR-present world, among many concerns, we must give special attention to logging sensitive data of individuals.
In this tutorial, we’ll see how to mask sensitive data in logs with Logback. Overall, this approach isn’t the real way to solve the problem – it’s kind of a last line of defense for our log files.
2. Logback
Logback is one of the most widely used logging frameworks in the Java Community. It’s a replacement for its predecessor, Log4j. It offers a faster implementation than Log4j, and it provides more options for configuration and more flexibility in archiving old log files.
Sensitive data is any information that is meant to be protected from unauthorized access. This can include anything from personally identifiable information (PII), such as Social Security numbers, to banking information, login credentials, address, email, and others.
We’ll mask the sensitive data that belongs to users while logging within our application.
3. Masking Data
Let’s say we log user details in the context of a web request. We need to mask the sensitive data related to users. Let’s assume our application receives the following request or response that we logged:
{
"user_id":"87656",
"ssn":"786445563",
"address":"22 Street",
"city":"Chicago",
"Country":"U.S.",
"ip_address":"192.168.1.1",
"email_id":"[email protected]"
}
Here, we can see that we have sensitive data like ssn, address, ip_address, and email_id. Hence, we have to mask this data while logging.
We’ll mask the logs centrally by configuring masking rules for all log entries produced by Logback. In order to do that, we have to implement a custom ch.qos.logback.classic.PatternLayout.
3.1. PatternLayout
The idea behind the configuration is to extend every Logback appender we need with a custom layout. In our case, we’ll write a MaskingPatternLayout class as an implementation of PatternLayout. Each mask pattern represents the regular expression that matches one type of sensitive data.
Let’s build the MaskingPatternLayout class:
public class MaskingPatternLayout extends PatternLayout {
private Pattern multilinePattern;
private List<String> maskPatterns = new ArrayList<>();
public void addMaskPattern(String maskPattern) {
maskPatterns.add(maskPattern);
multilinePattern = Pattern.compile(maskPatterns.stream().collect(Collectors.joining("|")), Pattern.MULTILINE);
}
@Override
public String doLayout(ILoggingEvent event) {
return maskMessage(super.doLayout(event));
}
private String maskMessage(String message) {
if (multilinePattern == null) {
return message;
}
StringBuilder sb = new StringBuilder(message);
Matcher matcher = multilinePattern.matcher(sb);
while (matcher.find()) {
IntStream.rangeClosed(1, matcher.groupCount()).forEach(group -> {
if (matcher.group(group) != null) {
IntStream.range(matcher.start(group), matcher.end(group)).forEach(i -> sb.setCharAt(i, '*'));
}
});
}
return sb.toString();
}
}
The implementation of PatternLayout.doLayout()Â is responsible for masking matched data in each log message of our application if it matches one of the configured patterns.
The maskPatterns list from logback.xml constructs a multiline pattern. Unfortunately, the Logback engine does not support constructor injection. If it comes as a list of properties, addMaskPattern is invoked for every config entry. So, we have to compile the pattern every time we add a new regex to the list.
3.2. Configuration
In general, we can use regex patterns for masking sensitive user details.
For example, for the SSN, we can use a regex like:
\"SSN\"\s*:\s*\"(.*)\"
And for the address, we can use:
\"address\"\s*:\s*\"(.*?)\"Â
Furthermore, for the IP address data pattern (192.169.0.1), we can use the regex:
(\d+\.\d+\.\d+\.\d+)
Finally, for email, we can write:
(\w+@\w+\.\w+)
Now, we’ll add these regex patterns in maskPattern tags inside our logback.xml file:
<configuration>
<appender name="mask" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="com.baeldung.logback.MaskingPatternLayout">
<maskPattern>\"SSN\"\s*:\s*\"(.*?)\"</maskPattern> <!-- SSN JSON pattern -->
<maskPattern>\"address\"\s*:\s*\"(.*?)\"</maskPattern> <!-- Address JSON pattern -->
<maskPattern>(\d+\.\d+\.\d+\.\d+)</maskPattern> <!-- Ip address IPv4 pattern -->
<maskPattern>(\w+@\w+\.\w+)</maskPattern> <!-- Email pattern -->
<pattern>%-5p [%d{ISO8601,UTC}] [%thread] %c: %m%n%rootException</pattern>
</layout>
</encoder>
</appender>
</ configuration>
3.3. Execution
Now, we’ll create the JSON for the above example and use logger.info() to log the details:
Map<String, String> user = new HashMap<String, String>();
user.put("user_id", "87656");
user.put("SSN", "786445563");
user.put("address", "22 Street");
user.put("city", "Chicago");
user.put("Country", "U.S.");
user.put("ip_address", "192.168.1.1");
user.put("email_id", "[email protected]");
JSONObject userDetails = new JSONObject(user);
logger.info("User JSON: {}", userDetails);
After executing this, we can see the output:
INFO [2021-06-01 16:04:12,059] [main] com.baeldung.logback.MaskingPatternLayoutExample: User JSON:
{"email_id":"*******************","address":"*********","user_id":"87656","city":"Chicago","Country":"U.S.", "ip_address":"***********","SSN":"*********"}
Here, we can see that the user JSON in our logger has been masked:
{
"user_id":"87656",
"ssn":"*********",
"address":"*********",
"city":"Chicago",
"Country":"U.S.",
"ip_address":"*********",
"email_id":"*****************"
}
With this approach, we can only mask those data in log files for which we’ve defined regular expressions in maskPattern in logback.xml.
4. Conclusion
In this tutorial, we covered how to use the PatternLayout feature to mask sensitive data in application logs with Logback. Also, we saw how to add regex patterns in logback.xml for masking specific data.
As usual, code snippets are available over on GitHub.
