Executives want a clear measurement of developer productivity, but determining it is proving elusive. Because how do you even define software development productivity? Is it the number of points you completed during a sprint? Lines of code or bugs fixed? Features correlated with tangible business outcomes? Or, is it the satisfaction you feel after a day’s job well done? Or is productivity shaped by the social cues you get from your colleagues?
When Brian Houck, principal productivity engineer, Microsoft, asked this question to the thousands of internal developers at Microsoft, he was surprised to see hundreds of different answers across all the above areas. “Developer productivity is many things and means different things to different people,” said Houck.
I recently attended DPE Forum: NYC, which brought together leaders in the burgeoning field of developer productivity engineering (DPE) to share their experiences implementing DPE in practice. Hosted by Gradle on October 3 at the Nasdaq MarketSite building, the event was an eye-opener into the science behind DPE, data-backed internal studies on developer productivity, and what engineering managers are actively doing to improve efficiencies.
Below, I’ll highlight some key takeaways from talks at the DPE Forum to help you get an edge on this emerging practice. We’ll end with some actionable developer productivity tips that you can apply to your own engineering culture.
Reduce Feedback Cycles to Reduce Cognitive Fatigue
According to Hans Dockter, CEO and founder, Gradle Inc., developers see a huge latent potential in optimizing their toolchain. But, although engineering leadership is becoming more open to new DPE initiatives, they tend not to feel as strongly about its potential. Leaders may not see anything wrong with waiting for a ten-minute build, telling developers to “just check your email while you wait.”
Yet, studies have found that the productivity of multitasking teams is far less than that of teams that focus on singular tasks. Furthermore, cognitive control tasks, in which the brain must build new neural pathways, contribute to added mental fatigue. The pervasive ‘multitasking myth’ leads to a superficial perspective that is a blocker to improving the toolchain processes, said Dockter. “When your muscle is exhausted, it’s a biological effect.”
So, how can we reduce cognitive fatigue? Well, Dockter attributes cognitive fatigue and burnout in part to the lack of quick feedback while testing new builds. Code verification failures, flaky tests and failed CI/CD builds can really yank developers out of a ‘flow state.’ And he’s found that when the feedback cycle is longer than ten minutes, you tend to get more task-switching, accelerating cognitive fatigue and decreasing productivity.
The benefit of investing in DPE, he said, is reducing this feedback cycle, thus reducing context switching that contributes to cognitive dissonance. “You cannot have a great developer experience without a great toolchain experience. And that’s what DPE is about.”
Case Study: Decreasing Cycle Time at Netflix
Engineering leaders are actively looking for ways to keep developers in the “inner loop” with fewer interruptions. Netflix, for example, has been investing in ways to improve developer experience around their toolchain, like instigating automated retries for flaky tests. Aubrey Chipman, senior software engineer, Netflix, shared how Netflix Engineering is actively configuring and optimizing their testing processes to reduce this cycle time.
As part of these endeavors, many Netflix projects now use Testcontainers to optimize testing. They also use Gradle’s Develocity for predictive test selection (PTS), which Chipman estimated has saved roughly 30 thousand potential hour savings in a year.
Netflix is also toying with remote test execution (RTE) as a beta offering, which distributes tests across remote agents to execute them faster. However, Chipman acknowledged that parallel executions come with potential limitations, such as network security issues and debugging for different environments.
Build a Systematic Approach to Developer Velocity
Although all teams are special snowflakes, there are definitely common requirements they all share. This is why the idea of an internal developer platform (IDP) has seen greater interest lately. IDP architectures deliver common ‘paved roads’ to internal developers with guardrails to remove friction within the software development life cycle (SDLC).
Amado Gramajo, VP, infrastructure and DevOps engineering, Nasdaq, defined DPE as enhancing developer productivity and happiness through accelerated technologies. And to him, the SDLC really has focused on improving the Ops side in recent years. Now, there is a big opportunity to focus on improving the build and testing aspects of this toolchain.
To distribute common features, Nasdaq adopts a systematic approach to developer velocity, as he called it. They haven’t created a centralized team per se but have developed a shared service available to all developers to help them improve velocity. This global service cuts across otherwise siloed teams, bringing them shared requirements, like resiliency and compliance features. As areas like Kubernetes, SRE and containerization become more normalized, said Gramajo, we need to build the operational controls so things can be automated.
Case Study: Tracking Toolchain Data at GitLab
Some are taking the idea of a systemizing developer velocity quite literally. For instance, Lee Faus, global field CTO, GitLab, described the concept of operating a ‘software factory.’ How can we optimize the supply chain to figure out how to produce artifacts that customers are happy with every single time, just like Oreo cookies?
Well, Faus identified a core issue in webhooks, which he referred to as “the dumpster fire of our toolchain,” he said. Every tool emits different events, which are produced by many actions within the toolchain. However, if this data isn’t properly triggered or received correctly, you could easily have data corruption, causing issues in the testing process.
Therefore, Faus recommended properly handling event data with solutions that enable bi-directionality and avoid creating data silos. Ideally, collecting data related to the software development process should be traversable to help developers pinpoint where problems emerge from. And this root cause analysis should be tied to actionable outcomes for remediation.
Base DPE Descisions on the Data
Although defining productivity is challenging, we have data to back up certain actions leaders can take to increase developers’ perceived productivity and satisfaction. After surveying thousands of internal developers at Microsoft, Houck was able to make some data-backed recommendations.
Formerly Set Aside ‘Focus Time’
The first finding isn’t surprising. Developers like to code. They don’t like things that distract them. Houck’s research found that the amount of time spent coding was positively correlated to developer satisfaction. Engineers who formerly blocked off uninterrupted time to code (especially in the morning) are far more likely to say they are productive. This perceived productivity remains, regardless of whether focus time is chosen or mandated.
Practice Meeting Hygiene
One way to free developers up for more concentrated work is to reduce their time in low-quality meetings. These could be recurring meetings that have outlived their usefulness or ones where the information is not relevant in the first place. Yet, interestingly, not all meetings are bad. For example, Houck’s research shows developers are 2.5x more satisfied with one-on-one syncs with their direct manager.
Allow Flexible Work Arrangements
Houck’s research found that happiness and productivity linearly decline as developers are asked to go into the office more often. The findings are clear—remote workers are more likely to feel more productive than fully on-site workers. Furthermore, the developers who don’t have agency in where they work feel 11% less productive. While there are certainly benefits to in-person brainstorming and team-building, leaders shouldn’t force engineers back into the office for an arbitrary number of days.
Final Thoughts
Measuring developer productivity is a tricky art. Therefore, Houck recommends choosing a few metrics, such as developer satisfaction, the amount of dedicated focus time developers have and the degree of friction within their workflows.
Similarly, Dockter recommended measuring the obstacles and hidden costs in the developer toolchain—obstacles like slow build and testing cycles—that can be readily removed to logically enhance productivity. “The toolchain gives you indications that certain teams are not as mature as others,” he said. Implementing DPE is steady progress, not firefighting, so you’ll probably want to sneak these changes in slowly, he added.
Fitter, happier, more productive. Developer productivity engineering is an emerging practice with a lot of science and opinions surrounding it. But there’s also the human factor to consider. A solid engineering culture will require meeting people halfway and giving them purpose and incentives to retain their interest.
Finally, anyone’s bound to get burned out sometimes with work. At least by minimizing the unnecessary toil in the toolchain, we can encourage a more satisfied developer experience that keeps up high productivity—however you want to define it.
