DevOps engineers create solutions for improving the development speed of applications and systems as well as ensuring the stable operation of services. Being involved in both development and operations, they are responsible for a wide range of tasks, from developing and improving applications and systems to building and operating infrastructure environments such as the cloud.

“DevOps” is a word coined from combining “Development” and “Operations”. To put it simply, it is an idea that the development side and the operation side cooperate to ensure speedier development, while also providing stable service operation.

Due to the focus of DevOps on improving development speed, it is easy to confuse this with agile development, which will be explained later.

Amongst all the different IT engineering jobs, to be a DevOps engineer requires a wide range of skills and plenty of experience. With a higher level of responsibility comes higher salary. The average annual income for a DevOps engineer is somewhere between 5 to 15 million yen, while the average for IT engineers in general is 4.52 million yen (※doda company survey).

In the future, the demand for DevOps engineers is expected to rise, as the industry becomes more specialized due to increasing system complexity and more infrastructure environments transitioning to the cloud.

This article provides a brief overview of what a DevOps engineer does and explains how you can become one too.

DevOps engineers differ from infrastructure engineers

A lesser-known position amongst IT engineers, the DevOps engineer is often compared to infrastructure engineers. This is because both are involved in tasks like server management.

The table below provides a brief overview of the differences between DevOps engineers and infrastructure engineers, for you to get a good idea of what DevOps engineers do.

◆Differences between DevOps engineers and infrastructure engineers

What sets a DevOps engineer apart from an infrastructure engineer is the former’s involvement in the development side. Both DevOps engineers and infrastructure engineers work on networks and servers, but DevOps engineers also engage in application and system development.

You can think of DevOps engineers as one level beyond infrastructure engineers. This is why DevOps engineers must have more knowledge and experience than just infrastructure engineers.

The No.1 role of DevOps engineers is automation

Now let’s take a look at what DevOps engineers do. As explained earlier, DevOps is a concept born to “speed up development” and to “enable stable operation” of the system at the same time. What do DevOps engineers do to achieve these goals?

Here are some of the specific roles DevOps play in the IT industry.

“Infrastructure as Code (IaC),” essential for stable system operation

Infrastructure as Code (IaC) means to get infrastructure settings written in code in order to automate the building process of the environment for operating and developing systems and software.

In creating infrastructure, the task of typing all the commands according to a design can be extremely tedious and costly, especially as the scale increases. In addition, people cannot always avoid making little mistakes.

Once infrastructure settings are written in code, all you need is just to execute it in order to automatically create an infrastructure environment. Also, executing the same code means you can reproduce identical environments at scale without the risk of human error. By allowing multiple servers to reproduce the same environment, problems caused by differences in infrastructure settings are less likely to occur.

As a DevOps engineer, you want to achieve the stable operation of a system. Therefore, IaC is an important skill to keep in mind in order to minimize the workload and avoid human errors when creating infrastructure.

However, you need skills for building the infrastructure to be able to practice IaC, which means more investment in acquiring those skills may be necessary. You may also need experience working as an infrastructure engineer.

Build a CI/CD Pipeline to speed up development

Building a CI/CD pipeline is a key skill for DevOps engineers who seek to “speed up” the development of applications and systems.

A CI/CD pipeline is an automated set of steps required to practice CI/CD. CI/CD is a method of automating build, integration, testing, etc. to make software development faster and increase the speed before production.

CI (Continuous Integration) is a method of automating code changes from build to test.
CD (Continuous Delivery) is a method that allows software that has passed testing to be automatically released into a production environment.

◆ CI/CD pipelines

Simply put, by building a CI/CD pipeline, the entire process of

Build – Integrate – Test – Release – Deploy

becomes automatic. Any changes to your code will be automatically applied to every step from build to test to deployment, thus reducing the lead time of your development process.

By getting everyone on a development team to work with this pipeline, software release becomes much faster.

CI/CD tools include on-premises and cloud types, with some examples being Jenkins and Drone for the on-premises and Travis CI and CircleCI for the cloud. DevOps engineers need to use different tools to build CI/CD pipelines depending on their objectives.

“Knowledge of managed services” to reduce management cost and working time

A managed service allows a company to outsource part of the operation management and maintenance of a serve.

Stably operating a server is no simple task. From hardware procurement to equipment/line management to trouble handling, the operation management and maintenance of a server requires expertise offered by dedicated professionals, which can be quite costly.

This is where managed services come in. By capitalizing on managed services, you can reduce the amount of work devoted to issues such as fault response and security measures. This allows for improved efficiency and ultimately leads to stable system operation.

Since not all engineers in a company are knowledgeable about infrastructure, leveraging managed services to reduce management costs and work hours is an important task for DevOps.

As you can see, being a DevOps engineer is not so much about working directly on products and services, but rather focusing on creating solutions to increase productivity for developers and operators.

2021 Roadmap for becoming a DevOps Engineer

So, what does it take to become a DevOps engineer?

This section is based on the article, “The 2021 DevOps RoadMap for Developers” which is another highly recommended read.

Now, let’s take a look at the roadmap for becoming a DevOps engineer. There are 9 steps in total.

◆Roadmap for becoming a DevOps Engineer

1. Learn a programming language

To be involved in system development, you need to know a programming language. If possible, become proficient in at least one of: Java, Python, or Ruby

Each has its own characteristics. Java is generally an all-rounder in any environment, while Python is relatively simple and easy to understand. Ruby originated in Japan and thus more resources are available in Japanese. Its code is simple and easy to read, making it beginner friendly.

2. Understand different OS concepts

This comes under the area of “Operations.” In the past, only system operation managers and support personnel were responsible for the OS and hardware. However, as DevOps engineers engage in both systems and infrastructure, OS-related knowledge is necessary.

You need to be familiar with important systems such as process management, memory storage and file systems.

3. Learn server management

For stable service operation, you must build a server to create a system and environment.

More specifically, you need to monitor the operational status of applications/services and the usage of CPU, memory, and other resources; update the system, fix programs that need to be improved, and respond to failures. All of these require general knowledge of a server.

4. Learn about network and security

The evolving technology nowadays allows for easy access everywhere on the web, which also means security challenges are on the rise. As poor security measures can lead to the leakage of important data such as customer information, it is critical to learn about network and security.

Make sure to have basic but essential ones down, including DNS, FTP, SSL, etc.

5. Understand middleware and learn how to set it up

Middleware is software that performs intermediate processing between an application and an OS in a computer. Examples of middleware are the web server, application server, and database management server.

◆About middleware

Understanding the structure of a web server or application server, and learning how to set it up, will give you a better understanding of the system. Doing so allows you to proceed to the next step where you make the system operation more efficient and automate the building of the infrastructure environment.

6. Learn Infrastructure as Code (IaC)

As explained earlier, Infrastructure as Code (IaC) means to have the infrastructure settings written in code in order to automate the creation of operation/development environments for systems and software.

The benefits of IaC include no need to manually
repeat the same process when creating a different environment, an easier grasp of current infrastructure and server configuration, and no need to manage procedures.

7. Learn CI/CD pipeline building tools

Again, CI/CD is a method of automating build, integration, testing, etc. to speed up software development and to make it ready for release into production immediately.

There are various tools for building CI/CD pipelines from open source to commercial ones. However, one tool cannot automate all of CI/CD, and it is common to create a CI/CD pipeline by combining tools that are appropriate for a specific development stage. This means you need to learn to use multiple tools.

8. Learn to monitor software and infrastructure

In addition to middleware setup and deployment, software and infrastructure monitoring is another important role of DevOps engineers.

Monitoring means constantly checking for anomalies or problems with the functionality of an application, network, server, OS, or middleware.

There are already a variety of tools available for monitoring, so DevOps engineers need to be able to choose the right tool for their companies’ needs.

9. Learn about the public cloud

The public cloud is a service that provides cloud computing environments such as servers, storage, databases, and software through the Internet to an unspecified number of users, including companies and individuals. These include Amazon Web Services (AWS), IBM Cloud, and Google Cloud Platform (GCP).

Since there is no need to own a server in-house, the use of the public cloud is rapidly spreading nowadays, and many companies have already moved their servers to a cloud environment. As a DevOps engineer, you need basic knowledge of the public cloud.

AWS and GCP are well known public clouds that you should get a grasp of.

This is the 2021 roadmap for becoming a DevOps engineer. A DevOps engineer, being a type of IT engineer, of course needs a programming language. On top of that, you should also know how to work with various tools and infrastructures (especially the cloud).

As mentioned at the beginning of the article, to be a DevOps engineer requires a wide range of knowledge and experience, and therefore it is a position one step beyond application engineers and infrastructure engineers. Sounds worth aiming for?

“DevOps” is a model or concept, “agile development” is a development method

As already explained, “DevOps” is a coined word that combines “Development” and “Operations”. The idea of DevOps was born from the contradiction between the development side and the operation side. While the former wants to develop applications and systems quickly, the latter wants to ensure stable operation.

In other words, in order to make a system better, the development side generally takes a policy of “adding new features and updating existing ones.” On the other hand, the operation side often takes a policy of “if things are running stably, keep them as they are,” in order to consistently provide a comfortable experience to the users and to avoid troubles with systems and functions as much as possible.
The concept of “DevOps” was born to bridge these contradicting interests.

Now, “agile development” is a method that, when developing applications and systems, repeats the development process of “planning → design → implementation → test” many times in a short period of time.

Once you have a rough idea of the system you want to create, you iterate through planning, design, implementation, and test, and release the system relatively early. After the system is released, you then repeatedly improve it based on feedback from users and clients. This allows for agility and flexibility in development.

Agile development is the opposite of waterfall development. Waterfall development is a method in which the development process of “design → development → implementation → test” is divided into each stage and developed step by step. In this method, developers decide in advance all the functions and spec details to be installed in the system, then start development and complete one step after the other. Basically, you never go back to the previous stage, and thus systems and software are released only after the entire process is completed. In the past, this has been the customary strategy for many companies.

However, it is common to discover improvements or to receive feedback from users after releasing systems or applications. Even with all the time spent during waterfall development, extra work is always necessary to use this method successfully..

As a result, more and more companies are adopting agile development because it is easy to respond to user needs when “planning → design → implementation → testing” is repeated over and over in a short period of time. Agile development is very compatible with “DevOps,” an idea that enables both speedier development and stable service operation.

By incorporating agile development, DevOps engineers can “speed up development” while creating a mechanism that allows developers and operators to collaborate to achieve “stable service operation.” This is why this newer development method is currently in high demand.

◆The difference between agile development and waterfall development

Source: Agile and DevOps Quality Assurance and Reliability

By the way, the word DevOps first appeared in 2009 at a conference called “O’Reilly Velocity 09” hosted by O’Reilly.

The term appeared in the presentation, “10 Deploys a Day：Dev and Ops Cooperation at Flickr” by John Allspaw and Paul Hammond, who were at the time engineers at an image-hosting service, Flickr.

In this presentation, they proposed that the development side and the operation side can bring about fast development that deploys more than ten times a day by cooperating toward a common purpose.
(*To deploy means to make the program usable.）

DevOps Engineers are increasingly in demand as IT and DX advance

Today, there is a growing demand for DevOps engineers.

This is because, along with the progress of IT and the promotion of digital transformation (DX), there is a growing need for a shorter development period and flexibility in system development.

As previously explained, development today requires companies to continue to improve their services after release using feedback from users. This makes for the rising demand for DevOps who can contribute in stabilizing operations while accelerating not only development but also improvement speed.

According to the article, “Ask Hitachi, the Current Role of DevOps and DevSecOps in Large-scale Development“, the company has adopted the idea of DevOps.

The article reports that DevOps is getting increasing attention due to many Japanese companies and organizations having begun to work on DX. DX is typically applied in new businesses, and therefore the business model is not clear, and the system specifications are not fixed.

Thus, the idea of DevOps has been adopted because it is necessary to improve fixing the problems that come out after development and release, while also ensuring stable
operation. In this way, the concept of DevOps is gaining more recognition regardless of whether it is a large company or a small- and medium-sized venture company.

Moving forward, given that the demand for public clouds such as AWS and GCP will also increase, there will be more spotlight on DevOps engineers who provide expertise on these areas while also being able to engage with the development side.

Summary

Now you have it, an overview of DevOps engineers. The path to becoming a DevOps engineer is steep, given the wide range of knowledge and experience required.

There is no correct way to become a DevOps engineer. However, if you are an inexperienced IT engineer with a goal of becoming a DevOps engineer, it is relatively easy to step up after gaining experience as an infrastructure engineer or a WEB application engineer.

We offer two services for IT engineers: “G Talent“, a career transition service and “GitTap“, a scouting and job information website that allows you to contact companies directly. For foreign IT engineers working in Japan and Japanese IT engineers seeking a global environment, we offer a variety of recruitment opportunities. Please check out the “G Talent” and “GitTap” service pages.