Utilities

Utilities are generic objects which are frequently used and help simplify the code.

Logger Database Integration Password Hashing Random Generator Server Environment Time and Date Email Integration

Utilities Package

Ctrl-click app and select New - Package

Enter utilities in the name field and press Enter.

Logger

Logging functionality is a must for any application. It provides a clear view on what application is doing, greatly simplifies the debugging process, and records any mishaps when the application is running in the production environment.

By default, Play Framework includes logging functionality for the application and the framework. We will add a minor customization to reduce the Play and third-party logs to a minimum, while having a clear print out of the application logic.

Custom Logger

Using IntelliJ, expand conf directory and open logback.xml file.

To simplify the log file access, change the file appender directory to logs/application.log.

Add a new logger entry above the play and application loggers. Name the new logger sample and set the level to DEBUG. Change play logger level to WARN and application logger level to INFO.

For your reference, several logging levels are available:

  • OFF - Used to turn off logging, not as a message classification.
  • ERROR - Runtime errors, or unexpected conditions.
  • WARN - Use of deprecated APIs, poor use of API, ‘almost’ errors, other runtime situations that are undesirable or unexpected, but not necessarily “wrong”.
  • INFO - Interesting runtime events such as application startup and shutdown.
  • DEBUG - Detailed information on the flow through the system.
  • TRACE - Most detailed information.

As you develop the application, experiment with different levels for the play, application, and sample loggers to see how much information you would like to see. At LineDrop, we mostly keep the codebase logging set to DEBUG, occasionally changing it to TRACE to observe low level operations.

Uncomment ASYNCFILE entry in the root node to enable writing of the logs to a file.

Example: LineDrop Play/Scala Web Application - Logback

Configuration

Using IntelliJ, expand conf directory and open application.conf.

Add a new entry to the end of the file:

Example: LineDrop Play/Scala Web Application - Configuration

Log Utility

Log utility will generate an instance of the Play logger and assign it the name read from configuration.

Ctrl-click utilities and select New - Scala Class.

Enter Log in the name field and double-click Object.

If prompted, click Add to add the file to version control.

Browse to the project repository to view the file:

LineDrop Play/Scala Web Application - Log Utility

Copy and paste the contents into the new file.

The get method reads the name of the logger from the configuration and returns an instance of a Play Logger with that name.

Commit and Push Changes

Select VCS - Commit from the top menu.

Review the files and directories and enter the commit message.

Click the dropdown arrow on the Commit button and select Commit and Push.

Click Push to confirm.

Database Integration

Web application needs a way to efficiently store and retrieve object data such as subscribers, users, sessions, keys. At LineDrop, we chose to store our data using MongoDB due to high performance on large datasets and the ease of implementation.

MongoDB is a NoSQL database; it doesn’t have tables or relationships between tables. Instead, the data is stored in collections of JSON-like documents containing key/value pairs. Documents in the same collection may have different structures.

Document example:

Launching MongoDB

Open a Terminal window and launch the database engine.

mongod

Minimize the Terminal windows to keep the database running.

Library Dependencies

Using IntelliJ, open build.sbt.

Add MongoDB Scala driver library dependencies. For example:

Please see MongoDB Scala Driver for the latest version.

Click Import Changes when prompted.

Wait for the dependencies to load.

Example: LineDrop Play/Scala Web Application - Build.sbt

Configuration

Storing database connection parameters in the application configuration file helps to avoid “hard-coding” the variables and makes it easy to change the server location as well as the name of the database.

Open conf/application.conf and add the following entry:

Server key points to where the database engine is running. In this case, the database is running on the same machine as the application.

Database key is the name of the database to be created and used by the application. In this case, the database is named sample.

Database Utility

Ctrl-click utilities and select New - Scala Class.

Enter Database in the name field and double-click Object.

If prompted, click Add to add the file to version control.

Browse to the project repository to view the file:

Example: LineDrop Play/Scala Web Application - Database Utility

Copy and paste the contents into the new file replacing the original content.

Database utility is a generic object that provides all of the project’s models access to the data. It is unaware of the models and is designed to read, write, update, and delete data from a given collection.

MongoDB Scala driver prints out quite a bit of information in the terminal making it difficult to read the application output. So, as the first line of code inside the Database object, we set the specific org.mongodb.driver logger to report errors only.

The object reads the application configuration and loads the server connection string into config_client variable and the database name into config_database.

The object gets an instance of the customer logger and assigns it to the log variable.

Config_client, config_database, and log are private variables, meaning they are only accessible within the object.

Insert Method

The insert method inserts a single document into a collection.

The method accepts a document and a collection name as parameters. It opens a client connection to the database server, then accesses the database and the collection.

The method uses the driver’s insertOne method which is a threaded operation. An observer is registered with the tread to report back once the insert operation is completed.

Critical observer methods are overridden to capture the result.

  • onNext: current document has been successfully inserted
  • onError: an error has occurred
  • onComplete: the thread has finished

Client is explicitly closed when the thread is finished or an error has occurred. We discovered from experience that if the client is not explicitly closed, the connection is kept open too long creating a potential memory leak.

The method returns no value. Once it is called, it simply runs its thread to completion.

Find Method

The find method searches a collection for a document based on a criteria.

The method accepts the search key, search value, and the collection name. It opens a client connection to the database server, then accesses the database and the collection.

The method uses the driver’s find method, which is a threaded operation. Unlike the insert method, the find method returns a value. To wait for the return value, the method utilizes Scala’s Future/Promise structure.

An empty result sequence is initialized.

A promise container is set up.

Driver’s find operation thread is started, looking for a document in the collection that has a search_by key that matches the specified value. The results are sorted by the created timestamp with the most recent document first.

A subscription is registered with the driver’s find operation. Some time in the future, the operation is completed, the sequence of found documents is appended to result_seq and the promise container is set. Scala’s Future/Promise structure enables the method to wait for the thread to complete.

Future variable is initialized.

Future then waits for the promise.

Instead of returning the most recent result, the method returns Scala’s Option[Object] which allows to account for a case that nothing has been found (None) or a document matching the criteria has been found (Some[Document]).

Find_all Method

The find_all method is similar to the find method, except that it finds all documents in a collection with the search criteria omitted.

The method employs Scala’s Future/Promise structure to return the document sequence once the threaded operation has completed.

Update Method

The update method locates a document based on a search criteria and replaces the original document with a new one.

The method accepts the search key, search value, updated document, and the collection name. It opens a client connection to the database server, accesses the database, and the collection.

The method uses the driver’s replaceOne method which is a threaded operation. An observer is registered with the tread to report back once the replacement operation is completed. Client is explicitly closed when the thread is finished.

Delete Method

The delete method locates a document based on a search criteria and deletes it.

The method accepts the search key, search value, and the collection name. It opens a client connection to the database server, accesses the database, and the collection.

The method uses the driver’s deleteOne method which is a threaded operation. An observer is registered with the tread to report back once the operation is completed. Client is explicitly closed when the thread is finished.

Delete_before Method

The delete_before method locates documents with a search key prior to a specific timestamp and deletes them.

The method accepts the search key, timestamp, and the collection name. It opens a client connection to the database server, accesses the database, and the collection.

The method uses the driver’s deleteMany method which is a threaded operation. An observer is registered with the thread to report back once the operation is completed. Client is explicitly closed when the thread is finished.

Logging Notes

As you can see from the Database utility, all objects and classes in this guide follow a specific logging hierarchy: debug messages are designated for methods and trace messages are designated for inner logic.

Commit and Push Changes

Select VCS - Commit from the top menu.

Review the files and directories and enter the commit message.

Click the dropdown arrow on the Commit button and select Commit and Push.

Click Push to confirm.

Reference

MongoDB Scala Driver

Password Hashing

Standard security practices require that passwords cannot be stored in plain text. Unlike SSL encryption, a password does not need to be decrypted to be compared with some value; instead, a one-way computationally intensive virtual representation, or a hash, is created for that value with a hashing function.

Hashing function produces the same random string for the same phrase, making it vulnerable to dictionary attacks, so an additional random string, called a salt, is generated and fed into the hashing function to produce a truly unique value.

While verifying a user’s password, the stored salted hash is read. The stored hash itself contains the salt with which it was generated. Hashing function takes the same salt to generate a salted hash of the provided text value. The resulting hash is then compared to the original hash.

A great library to use for password hashing operations is Bcrypt. It is currently accepted as the standard in cryptographic algorithms.

Library Dependency

Using IntelliJ, open build.sbt.

Add Bcrypt library dependency. For example:

Please see the latest version at: https://github.com/t3hnar/scala-bcrypt

Click Import Changes when prompted.

Wait for the dependencies to load.

Hash Utility

Ctrl-click utilities and select New - Scala Class.

Enter Hash in the name field and double-click Object.

If prompted, click Add to add the file to version control.

Browse to the project repository to view the file:

LineDrop Play/Scala Web Application - Hash Utility

Copy and paste the contents into the new file.

The create method generates an explicit salt and then creates a hash.

The validate method accepts a string value and a hash as parameters and validates the hash returning a compound object Try[Boolean]. If the hash is safe, Success(result) is returned; otherwise Failure(result) is returned. Failure(result) may be returned if, for example, the salt has incorrect length. With a safe hash, the isBcryptedSafe function then compares the stored hash to the hash generated from the value. If the hashes match, the result is true; otherwise, it is false.

Commit and Push Changes

Select VCS - Commit from the top menu.

Review the files and directories and enter the commit message.

Click the dropdown arrow on the Commit button and select Commit and Push.

Click Push to confirm.

References

Random Generator

Random number generator is a convenient utility for creating password reset tokens, session strings, and unique identifiers.

Random Utility

Ctrl-click utilities and select New - Scala Class.

Enter Random in the name field and double-click Object.

If prompted, click Add to add the file to version control.

Browse to the project repository to view the file:

LineDrop Play/Scala Web Application - Random Utility

Copy and paste the contents into the new file.

Random utility’s alphanumeric method creates an alphanumeric value generated with 128 bits and with the 32 base.

Commit and Push Changes

Select VCS - Commit from the top menu.

Review the files and directories and enter the commit message.

Click the dropdown arrow on the Commit button and select Commit and Push.

Click Push to confirm.

Server Environment

The Server utility provides a concise way for accessing the server URL and Environment configuration settings.

Configuration

Server configuration will change for each of the application environments: development, test, and production.

Using IntelliJ, expand conf directory and open application.conf.

Add the following entry:

Example: LineDrop Play/Scala Web Application - Configuration

Environment Utility

Ctrl-click utilities and select New - Scala Class.

Enter Environment in the name field and double-click Object.

If prompted, click Add to add the file to version control.

Browse to the project repository to view the file:

LineDrop Play/Scala Web Application - Environment Utility

Copy and paste the contents into the new file.

The Environment utility is simply an enumeration object that stores three values: development, test, and production; one for each environment in which the application resides.

Server Utility

Ctrl-click utilities and select New - Scala Class.

Enter Server in the name field and double-click Object.

If prompted, click Add to add the file to version control.

Browse to the project repository to view the file:

LineDrop Play/Scala Web Application - Server Utility

Copy and paste the contents into the new file.

The url method reads server URL setting from application configuration.

The environment method reads the environment setting from application configuration and returns an Environment enumeration value.

Commit and Push Changes

Select VCS - Commit from the top menu.

Review the files and directories and enter the commit message.

Click the dropdown arrow on the Commit button and select Commit and Push.

Click Push to confirm.

Time and Date

Running servers and applications in UTC greatly simplifies time calculations. At LineDrop, all timestamps are created in UTC and converted to the user's local time zone only when displaying views or compiling reports.

TimeStamp Utility

TimeStamp utility provides methods for working with time objects.

Ctrl-click utilities and select New - Scala Class.

Enter TimeStamp in the name field and double-click Object.

If prompted, click Add to add the file to version control.

Browse to the project repository to view the file:

LineDrop Play/Scala Web Application - TimeStamp

Copy and paste the contents into the new file.

The UTC method returns the immediate timestamp in UTC. Timestamp is generated using the Joda library.

The convert method converts a BsonValue datetime, used by MongoDB, to Joda’s DateTime.

Commit and Push Changes

Select VCS - Commit from the top menu.

Review the files and directories and enter the commit message.

Click the dropdown arrow on the Commit button and select Commit and Push.

Click Push to confirm.

Email Integration

The application has to have a way to communicate with the world: welcome messages, notifications, password resets, among others.

Having used in-house email servers and integrated with Gmail, we discovered that for a start-up the best service is provided by SendGrid. It is painless to set up, free up to 100 messages per day, and very reasonable after that.

However, if you have access to an SMTP server you can implement Play-Mailer.

Create a SendGrid Account

Browse to https://signup.sendgrid.com/ and follow the steps to create an account.

Check your email for the welcome message from SendGrid, open the message and click on the link to verify your account.

Library Dependency

Using IntelliJ, open build.sbt.

Add Bcrypt library dependency. For example:

Please see the latest version at: https://app.sendgrid.com/guide/integrate/langs/java

Click Import Changes when prompted.

Wait for the dependencies to load.

Configuration

Server configuration will change for each of the application environments: development, test, and production.

Using IntelliJ, expand conf directory and open application.conf.

Add the following entry:

Update the sender setting to the email address from which the application will send notifications. For example: LineDrop Support .

API Key

In SendGrid, expand Settings on the left side panel and click on API Keys.

Click Create API Key.

Name the key Development and select Restricted Access. Scroll down to Mail Send and click the right-most point of the slider to enable the function.

Scroll down and click Create and View.

Click on the key. It will automatically copy into the buffer.

Paste the key into the key setting in configuration.

The configuration should look similar to:

SendGrid Utility

Ctrl-click utilities and select New - Scala Class.

Enter SendGrid in the name field and double-click Object.

If prompted, click Add to add the file to version control.

Browse to the project repository to view the file:

LineDrop Play/Scala Web Application - SendGrid Utility

Copy and paste the contents into the new file.

SendGrid utility uses the SendGrid API to send an email from the application.

Sender address and the API key are read from the configuration. A private instance of a Logger is instantiated.

The send method accepts the recipient’s email address, html message, and the subject as parameters. The method starts a new thread with the sending_thread method. Once the thread is completed, the success or failure result is captured.

The sending_thread method populates the from and to variables; content type is set to HTML. An instance of Mail object is created, followed by an instance of the request and a SendGrid object. Finally, the request is sent via the SendGrid API.

Commit and Push Changes

Select VCS - Commit from the top menu.

Review the files and directories and enter the commit message.

Click the dropdown arrow on the Commit button and select Commit and Push.

Click Push to confirm.

Production Configuration

In the future, when you have a production server, please update the IP Access Management and Sender Authentication in the SendGrid portal to make sure the emails sent by the application are not blocked.

Log in to your SendGrid account to update the settings.

IP Access Management

Expand Settings in the left panel and click IP Access Management.

Add the IP Address of your production server and your own external IP (listed in the Recent Access attempts) to the whitelist.

Sender Authentication

Expand Settings in the left panel and click Sender Authentication.

Click Get Started in Domain Authentication. Select your DNS host, for example Google Cloud and select Yes to brand the links.

Enter your domain and click Next to view a list of DNS records.

Updating DNS Records

Keep SendGrid open. Open a new browser window and log in to your DNS manager, for example domains.google.com. Add the CNAME DNS records one by one.

  • Name: Sendgrid DNS Name
  • Type: CNAME
  • Data: Sendgrid Canonical Name

Once added, wait a few minutes and then select I’ve added these records and click Verify in the SendGrid window.


Next: Terminal Commands