Intro

DevOps (Development and Operations) is a set of automated software practices that combine software development (Dev), testing and IT operations (Ops) to shorten the software development life cycle while delivering features, fixes, and updates frequently in alignment with the business objectives.

DevOps teams use tools to automate and accelerate processes, which helps to increase reliability. A DevOps toolchain helps teams tackle important DevOps fundamentals including continuous integration, continuous delivery, automation, and collaboration.

In a recent survey, 99% of respondents said that DevOps had a positive impact on their organization. The benefits of DevOps include faster and easier releases, team efficiency, increased security, higher quality products, and consequently happier teams and customers.

Reference: Atlassian - DevOps

Automating with Python

In this article, we will look at a portion of DevOps: automation of processes. P.S. This process can also be automated by another tool out of the box (however, this process is for learning purposes and demonstration). To achieve automation, we need to know how it's done manually first, let's look at the underlying approaches of our automation program.

Problem Statement

Jack has a SpringBoot / Java .jar project build and is ready to use on the server. Jack manually runs the .jar file and manages the deployment, monitoring and tailing of his .jar project using available terminal commands. However, this has been tedious for Jack from adding new changes to his project, building .jar, transferring the .jar to the server, stopping the running old file, starting a new one and tailoring it if need be.

Proposed Solution

To help Jack automate some of these processes, assume Jack has built his .jar executable app called app.jar and is using a Windows OS Server.

P.S. There are many options to automate the process we will look at below which include: bash script on Linux (.sh) or Windows Command Prompt scripting language (.bat). However, will use Python as it will run on both environments.

We will create a basic Python command line project, using basic Python with no third-party libraries to

1. Tail service (view its running logs)

2. Check service status (is it running / not)

3. Start service

4. Restart service

5. Stop service (if it's already running)

1. Start Service

Jack got used to running an app.jar via the terminal using the java command (Option 1)

C:\Projects\example> java -jar app.jar

But every time he exits the terminal, his program also stops. So he looked around and started to use the command where he didn't need to see the console output (Option 2) and outputs his project logs to app.log file to tail later.

C:\Projects\example> javaw -jar app.jar >> app.log

2. Tail Service

If you have used Unix systems, you might have come across the tail command - it's used to print the last N number of data of the given input (our log file in this case). But on Windows OS, it's a bit different, On CMD (default, old command prompt) we can use the command

C:\Projects\example> type "app.log" | more

type is a built-in command which displays the contents of a file on Windows OS and more is used to display the contents of a lengthy file one screen at a time.

To achieve an almost similar approach to tail on Linux, Jack looked at using a PowerShell command below, which is used to open the "app.log" file, display its current content, and then continue to monitor the file for any changes.

PS C:\Projects\example> Get-Content -Path "app.log" -Wait

3. Stop Service

This is a bit tricky, once he starts his project, how does he stop it? If he uses Option 1, he can easily close the terminal or via CRTL-C. With Option 2, it's not that straightforward because the application is started in the background, the OS assigns a PID to the running app.jar service and to close it, Jack will have to manually use Task Manager, CMD or from within his application if he has that mechanism in place.

Via CMD

To shut down a Java process started in the background with javaw, Jack used 2 commands.

1. Tasklist - used to display a list of currently running processes on the computer together with their PIDs.

2. taskkill - ends one or more tasks or processes, usually by using a PID. However, using Tasklist can be tedious as he has to look for the services running Java and mistakenly, he might stop other Java services running that depend on Java. To avoid that mistake, Jack resorted to finding only the PIDs running Java with the jps command.

jps - JVM Process Status Tool packaged in JDK is a tool that displays all the Java processes that are running on that machine.

C:\Projects\example> jps -m

10416 sem-project.jar 

13444 Jps -m

11708 app.jar --spring.profiles.active=live

From the sample output above, Jack noticed his service is running with PID 11708 and he stops it using the command

$ taskkill /F /PID 11708

Automating the boring task

Now that we understand the manual tedious approach, let's initiate automation. To achieve this, we will mimic running commands on the terminal using Python's subprocess module.

a built-in module that allows us to create new child processes, a valuable tool for executing external functions or commands in your Python code. Think of it as a way to programmatically interact with your computer using Python instead of manually entering commands in the terminal.

Check running service

In this snippet, we fire up our command, split the result by \n and loop through to check if our app.jar is present. If we found it, we split the line by space to get the PID which is the first item and return True.

Start service

In this snippet, we first check if the service is already running. We only fire our run command if the service is not already running

Tail service

To get application logs, we tailed the .log file to which the app writes its logs.

Kill service

To shut down the service, we will call our check_running_process function to be sure the service is running and get its PID. We then take this PID and pass it to our kill_service function.

From here, we have automated the tasks that Jack performs manually, restarting and stopping the service will reuse the functions we already defined. To conclude, our script will have an old CLI menu to abstract all these "basic" technicalities behind it, with the options mapped to the functions we created.

Remarks

We have seen that we can automate some tedious repetitive tasks we face in our project deployments. Python can achieve just that using its built-in libraries. However, we can achieve the same using other advanced tools or taking the cloud approach or scripting languages like bash on Linux and .bat on Windows. In Python, we can check the OS we are running on and adapt our functions to run the commands for the current host machine, that way we can have our script running on both platforms.