on
Some ideas for future research in DevOps, continuous delivery, microservices, and automated tests
In this post, I list some ideas (in draft mode) for software engineering research derived from my own research. Feel free to take advantage of them if you like.
The future of the operators’ role
Highly automated cloud services point to a possible future in which infrastructure becomes a commodity. Considering other market tendencies and ideas, such as adopting full-stack engineers and NoOps, one could ask about the future of the operators’ role.
Hemon-Hildgen (2020) noticed “operators’ fear of losing their jobs due to automation” and “operators’ resistance to change and fear for their future.” In at least one of our interviews, we also perceived the quandary about operations people who performed menial tasks and were not evolving to work in a more automated environment.
Therefore, research is necessary to understand how this workforce reacts to the market forces pushing for more rapid releases, which demands more automated ways of working. How are these professionals adapting to the situation? What are the companies’ policies about this issue? Would it be possible to give some advice to companies and workers going into this situation? What is the relation between these people’s fate and new roles such as DevOps engineers, full-stack engineers, and SRE?
Moreover, this debate must consider the title of the roles. The professional responsible for carrying IT operations in a software-producer company can be labeled as the infrastructure engineer. This may have different implications of calling these people “operators”. We remark that, in other industries, “operator” is a title attributed to menial workers.
A researcher could also conduct such an investigation by employing our taxonomy. The operator role’s disappearance could be associated with the weakness of the siloed departments and classical DevOps structures and the strengths of the cross-functional teams. Moreover, the evolution of “operators” to proper infrastructure engineers could be related to platform teams’ adoption.
Why developers do not automate tests?
In our interviews, we observed that test automation is still not adequately practiced in many software companies, as also noted by Olsson et al. (2012), and that the lack of tests is a limiting factor for achieving high delivery performance. Although we acknowledge that test automation is more challenging in some domains, as IoT and games, our observation still holds for scenarios outside such realms.
Therefore, although the industry has well-established practices for test automation and the academy has provided extensive research for advancing test automation, it is still necessary to understand why so many software professionals do not build automated tests.
An initial hypothesis is that developers have short deadlines and, therefore, there is no time for building automated tests. For example, one interviewee told us that a colleague was labeled as “incompetent” because he was “taking too much time with tests.” However, it is known that for a long-living system, the lack of automated tests will eventually decrease productivity, as new features are more likely to introduce new bugs.
Another possible explanation is that every software engineer must be educated in test automation. It is not feasible to outsource test automation to specialized people, as it is possible, for example, to let one specialist build an automated pipeline that will be easily used by the rest of the team.
Finally, this research line should provide guidelines and insights for companies and workers about overcoming the barriers to the widespread adoption of test automation. The range of solution types is vast, and it can include new techniques for test automation. But since we understand it is more a sociological than a technical problem, we expect solutions to involve companies’ policies about training or even new ways of training developers for test automation.
Automating the delivery automation for each microservice
Breaking down the system into microservices also requires building multiple delivery pipelines. However, setting-up the delivery pipeline for each microservice may be cumbersome (Leppanen, 2015; Neely, 2013). Therefore, as the number of microservices increases, it becomes crucial to automate the setup of the continuous delivery infrastructure (e.g., the deployment pipeline and static analysis). Therefore, such a situation calls for “automating the automation.”
Obs: GitLab has an initiative towards “automating the automation” called Auto DevOps (https://docs.gitlab.com/ee/topics/autodevops/). But is it useful? Do engineers like to use it? If not, why? Could it be better? How?
Another point in this discussion is about standardization. It seems that a high degree of standardization about how to build microservices within an organization would benefit the approaches for “automating the automation.” However, one of the promises of microservices is technological independence: each microservice should employ the technology best suited to solve its related problem (Lewis and Fowler, 2014). Therefore, the researcher should also be aware of past experiences about enforcing development standards across product teams, especially in large organizations.
The graal of reusability: is it worthy?
Although reusability is a historical goal of software engineering (McIlroy, 1968), Shahin et al. recommend engineers not to focus too much on it. It brings coupling, which is a huge bottleneck to continuous delivery (Shahin, 2016). Therefore, there is a trade-off between reusability (which is supposed to increase productivity) and independence from other components and services (which may cause handoffs and delay deliveries).
An empirical investigation in industrial settings about the motivations, reluctance, and even feelings of engineers toward reusability could guide practitioners in making decisions regarding reusability. Such an inquiry could also provide hints for researchers exploring new ways for increasing reusability.
Some hints on why reusability can be not worthy: when an engineer build a reusable component, she should treat it as a product, which increases costs due to documentation and more general tests. Another problem is defining the ownership of a component reusable by several teams. Moreover, when a team does not have the ownership of a component and needs to change it quickly, it may be tempted to just fork the component without contributing back to the community.
Conway’s Law and microservices
The adoption of microservices is supposed to impact the organizational structure according to Conway’s Law (Conway, 1968). This means each team should have ownership over a clear set of services, and each service should be clearly owned by a team. But so far, there is a lack of empirical evidence that the industry is following this principle. Therefore, it would be worthy of investigating industrial settings to understand how companies are taking into account continuous delivery and microservices to structure their IT technical teams and the implications of such structuring arrangements. More specifically, researchers could observe scenarios with teams following and not following Conway’s Law and so empirically relate observed benefits and drawbacks to such scenarios.
Release windows: are they really necessary?
Another challenge to be considered is the interplay between continuous delivery, microservices, and release windows, which block the continuous flow of deliveries. Although release windows limit continuous delivery, many companies adopt them for several reasons, especially when worried about financial and critical systems or for dealing with large-scale services.
Thus, in a microservices context, release windows bring more questions: does it make sense for each microservice to have its own release window (especially considering that some services can present some level of coupling)? On the other hand, having a single window for all microservices defies the very definition of microservices since they must be independently deployable.
Therefore, researchers could investigate practitioners’ motivations for release windows adoption and evaluate the consistency of such motivations. For this, researchers could compare organizations with similar contexts and understand whether there are factors determining the adoption of release windows.
Extending this post in Jun 2021…
Agility in infrastructure
There are views that the infrastructure world has not embraced agility yet. The infrastructure world still has the waterfall mindset: big projects, big deliveries, window releases, everything scheduled, everyone needs to talk to everyone… Despite the advances brought by the DevOps movement, it is worthy of investigating how the agile mindset is (or is not) permeating the infrastructure landscape within organizations.
Quality assurance and infrastructure
Although many software developers still neglect test automation, practices for test automation are already well established and debated in the context of software development. However, test automation for infrastructure as code is still incipient. What to test? How to test? These are relevant questions in any context, but maybe even more relevant for companies building in-house infrastructure platforms; I mean, in this context, the software under test is software to provision infrastructure.
Moreover, it is possible to evaluate the pertinence of test tools such as ChefSpec. I have used it a little bit, and I hope I’m wrong (it’s a long time ago by now), but I got the following impression: Chef is a declarative language, and in this context, the ChefSpec test seems to test if I declared something… but in this context, maybe the test is not bringing too much value since being able to make the assertion is equivalent to be able to make the corresponding declaration in the production code. For example, if I declare an Apache installation, maybe simply testing if the “Apache install” directive was used has limited value; perhaps testing the side effects (e.g., port and process) would be better. But off-course, I’m far from being sure, and this is why I think it deserves to be investigated.
DevOps structures and roles
My research has focused on organizational structures for DevOps, i.e., how responsibilities are split between development and infrastructure areas and how these groups interact. But the organizational structure is only part of the story… we also have a multitude of roles, such as operators, sysadmins, release managers, SREs, DevOps engineers, etc. There is already research on “DevOps roles.” However, what can maybe be an advancement is understanding the pertinence of each role in the context of different organizational structures.
References
Hemon-hildgen, Aymeric; Rowe, Frantz; Monnier-senicourt, Laetitia; Rowe, Frantz. Orchestrating automation and sharing in DevOps teams : a revelatory case of job satisfaction factors, risk and work conditions. European Journal of Information Systems. 2020
M. Shahin, M. A. Babar, and L. Zhu, “The intersection of continuous deployment and architecting process: Practitioners’ perspectives,” in Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM ’16, pp. 44:1–44:10, ACM, 2016.
H. H. Olsson, H. Alahyari, J. Bosch, Climbing the “stairway to heaven” – a mulitiple-case study exploring barriers in the transition from agile development towards continuous deployment of software, in: 38th Euromicro Conference on Software Engineering and Advanced Applications, 2012, pp. 392–399
M. Leppanen, S. Makinen, M. Pagels, V. Eloranta, J. Itkonen, M. V. Mantyla, and T. Mannisto. 2015. The highways and country roads to continuous deployment. IEEE Softw. 32, 2 (2015), 64–72.
Steve Neely and Steve Stolt. 2013. Continuous delivery? Easy! Just change everything (well, maybe it is not that easy). In Proceedings of the Agile Conference. 121–128.
James Lewis and Martin Fowler. 2014. Microservices. Retrieved from: https://www.martinfowler.com/articles/microservices.html.
M. Douglas McIlroy, J. M. Buxton, Peter Naur, and Brian Randell. 1968. Mass-produced software components. In Proceedings of the NATO Conference on Software Engineering, Software Engineering Concepts and Techniques. 88–98.
Conway, M E. How do committees invent. Datamation, v. 14, n. 4, p. 28-31, 1968.