Docker Data Containers with Nirmata

An exciting thing about Docker is the number of interesting use cases that emerge as developers adopt containers in their application development and operations processes.

One such concept is that of data container. While the usage may not be intuitive at first, data containers are very handy in some scenarios. A great use case is where a common set of configuration files need to be made available to multiple application containers depending on the runtime environment. This requirement actually came up in a recent customer engagement prompting us to revisit this concept.

In this post I will show how data containers can be used as a portable and versioned file system for configuration data, and how Nirmata helps make this easy at scale.

Docker already has an option to map host volumes or folders to a container. This allows host data to be accessed within the container. And since the data is on the host file system, it can be accessed across container restarts and it can be shared amongst different containers. But this approach couples the container to the host. In other words, the container is no longer portable as it needs the data folder to be available on the host prior to starting.

Now, instead of mounting a folder containing the data from the host, what if you can create a data container and use it in your application containers. This can be done as follows:

Create an image for your data container:

docker build -t nirmata/mongo-data-container .

Now, create the data container using the image that contains your data/files:

docker create --name mongo-data-container nirmata/mongo-data-container

Then create and run the application container, in this example mongodb. Use –volumes-from to load the volumes from the data container:

docker run -d --volumes-from mongo-data-container --name mongo-db mongo --smallfiles

With this command, the mongo-db container will access the data files from the mongo-data-container. Once your images are created, this approach is portable and does not require any setup on the host.

Taking advantage of data containers requires a higher level of automation when it comes to deploying your application especially in case of multi-host deployments. Some requirements are:

The data container needs to be created on the each host as the application containers that need to use it.
Data containers needs to be created before the application container is created or else you will get an error: container data-container not found, impossible to mount its volumes
Data container should be deleted after all the application containers using it have been deleted

There seems to be an ongoing debate on whether data containers are really containers at all, as these containers may not be running any application services. The data container approach can be extremely helpful, especially as you use and share Docker based applications across teams. For example, developers can quickly spin up application environments with data containers that have pre-populated application configuration settings. QA teams can create snapshots (Docker images) of their test environments and share them with developers for debugging/troubleshooting issues. A huge benefit of this approach is that your data, stored in a data container, can be versioned and shared just like application container images making them very easy to use.

At Nirmata, as we work with our customers, we have seen the use of data containers as a common and useful pattern. As a result we have made it easy for our users to define data containers in their application blueprints, and Nirmata’s policy based orchestration takes care of the rest across public and private clouds!

On simply specifying the data container in ‘Volumes From’ field, Nirmata will ensure that data containers are always deployed before any application containers that require it are deployed. Nirmata also ensures that other advanced capabilities such as scaling and auto recovery of application containers also work with data containers.

Data containers seem to address the portability problem. We would love to hear more about how you are using data-container or if there any other interesting use cases around containers that you may have come across.