It was the industry itself that created the difficulty of finding competent DevOps engineers. In the old ways, we had system administrators, database administrators, security engineers, shell script developers, and more. It resulted in an organization so big with high cost and large workforce. A job vacancy for every requirement.
Business has since then evolved focusing on leaner and cost efficient organization. This lead to the creation of new kind of IT Professionals. The DevOps Professionals where they are "Versatelist". Versatelist are engineers possessing multi-disciplinary modern technical skills such as: Cloud, Automation, Security, Development, and Operations.". Now; every organization is in search for DevOps resulting to scarcity, real DevOps becomes picky too.
Apart from the contribution of industry that created difficulty of finding DevOps engineers . What keeps hiring manager from finding these rare breed of talents? What is taking them years to build a competent DevOps Team of 5 engineers. What can they do then to fill the DevOps roles?
Top 6 Challenges of DevOps Hiring Managers:
1. Scarcity
One of the many challenges of DevOps hiring manager scarcity. I managed to talk to one Human Resource recruiter. It has been 12 months and she was un able to fill 24 engineers needed by her client. Another hiring manager took her 8 months to replace 1 resigned DevOps engineer.
Solution
Hire a competent senior DevOps engineers 3 - 4 then train a promising prodigy with potential like fresh graduates engineers to become DevOps by building bootcamps. These boot campers are hungry for learning. They becomes your buffer and complementary force.
2. Attrition
The young engineers in DevOps are very intelligent people. A fresh graduate who has 2 years of experience in DevOps will soon receive different offers from different companies. His or her experience as an apprentice for 2 years with senior DevOps makes him a target for other companies. DevOps usually work with a minimum of 5 engineers. Therefore, 1 DevOps engineer with 2 years of experience actually has 10 years because he gained experience working with other engineers.
Solution
DevOps engineers don't usually resign. Pay is secondary for them based on experience working with DevOps engineers. The key so they don't leave is to treat them well, give them respect, show them a vision and a challenge, provide them with work purpose. Protect them from workplace politics. Be a good manager/leader to them. Do this and they will stay long. Treat them just the way you want to be treated.
3. Salary Cost
The average annual salary of experienced DevOps engineers in the USA is $164,000 USD, while the average salary of an Android developer is $111,000 USD based on Glassdoor data. This means that if you are an e-commerce company, your DevOps engineers probably have a salary that is 40% - 50% higher than the rest of the technical team members, e.g., programmers, based on that data. Apart from that, DevOps engineers are scarce, and your competitors are willing to pay a premium salary to attract them to jump ship to their company.
Solution
Competent and real DevOps engineers aren't cheap. You need to conduct business case so that you can justify premium salary of 3 - 4 senior members that are competetive in the market. As mentioned earlier; create a DevOps bootcamp to clone your senior members with less salary leveraging on freshgrads or junior DevOps.
4. Skill Sets
DevOps hiring managers are looking for certain specific skills for their next DevOps engineer. These skills are top-notch and hard to find. Focus on the following talents, less is more when it comes to DevOps. Avoid unnecessary experience. 20 years doing the same means nothing when it comes to DevOps.
Domain Knowledge and Experience
Kubernetes
Docker
Jenkins | ArgoCD | GitLab
Helm | Ansible | Puppet | Chef | Pulumi
Many layers and levels of SecurityÂ
Linux Scripting
Terraform | Cloud Formation
Service Mesh
Cloud Native Infrastructure
Observability
Optimization
Programming e.g. Python
Test Automation
These are just the basics, it will vary depending on technology stack and requirement of the company.
Solution:
Beware of fake DevOps. A candidate full of social media DevOps training photos are red flag. These are engineers who suddenly became DevOps by changing their title without the backing of their skills. Do not rush into hiring them. The solution for this is to carefully hire, for example, three senior DevOps engineers, reviewing their domain experience and knowledge. Make sure they have references to support their skills. Once you have three competent senior DevOps engineers, launch an internal "DevOps University" where your company will qualify potential candidates and have them undergo DevOps immersion. Do not be afraid to overhire because attrition is a problem. You can even spin off the DevOps University to provide DevOps services to other companies and create a revenue-generating business.
5. Warmup Period
Once you managed to fill all your DevOps requirements. If you are lucky enough to build a competent team for 1 year the last challenge of DevOps hiring manager is the warmup period. Warmup period is the time it takes for your DevOps engineering team to build synergy and rapport. Usually it will take them another 12 months to build rapport. There will be friction in technology stacks, they will make deployment mistakes, they will have to build CICD pipeline. The pipeline may break as a result of experimentation resulting delays to your product releases. The new team has to adapt with your current company culture, and since these are foreign bodies, they will initially be rejected by project teams who are not use to DevOps.
Solution:
There is no solution here but rather a question: How much tolerance is your company willing to take in order to have a successful DevOps? How many risks is your company willing to take in order for the DevOps to kick off? How talented is your leadership team to manage friction between traditional and DevOps culture? Lastly, how competent and decisive is your technology executive to make choices based on facts, experience, and guts that will lead to your DevOps end state? During conflict, leaders have to have a clear plan and vision that people will use as a reference to answer conflict. Hypothetically, on a smaller scale, an organization needs Elon Musk-style leadership to deliver DevOps.
6. Uninspiring Hiring Manager During Interviews
True DevOps engineers are very intelligent beings. They can see through people who fake technical skills. Having a hiring manager who will become the future boss of a DevOps engineer trying to attract them but can't even explain what they are building is problematic. A manager that can't explain ingress controller to DevOps engineer to ask the right question during interview is a turnoff. DevOps engineers will put you on the side as they opt for growth over money, and these guys has many choices.
Solution:
Law of the Lid by John Maxwell. Go get a competent DevOps hiring manager who understands the language of DevOps. They should paint inspiring goals to applicants, be able to explain the CI/CD process and what they intend to build, the technology stack of DevOps, the difference between containers and orchestrators, and talk about the challenges in DevOps they are trying to solve. Speak the lingo of your DevOps engineers and they will follow you anywhere you go. Be a con artist and they will leave or not be attracted to be a part of the DevOps Team.
Conclusion: Challenges of DevOps Hiring Managers
Building DevOps Team is the reflection of the hiring manager persona, leadership, and capability. A hiring manager can only build a DevOps team as good as he is, and as competent as he is. Therefore; the biggest challenge of hiring manager in building DevOps is his own self the rest of the challenges mentioned above just follows.