Technical Authors in an Agile World

A response to Tom Johnson's posts on technical writers and Agile.

With thanks to Lisa Whincup, Paul Wicking and Samael Wright for proofing and discussion.

Last week Tom Johnson of I’d Rather Be Writing published two blog posts exploring the challenges faced by technical writers working within or alongside Scrum development teams. Part 1 outlined the issues, Part 2 explored possible solutions.

I acknowledge the caveat included at the start of the first post:

I also want to note that one person’s Scrum experience may differ from another. There is a wide variation of Scrum practices from company to company, and from project manager to project manager even within the same company. I’m painting a general picture of my Scrum experiences across companies over at least a dozen years, not just focusing in on one company or team.

In that spirit, I would like to present a different experience, that has led me to different conclusions. I am not about to argue there are no difficulties faced by technical writers working in Scrum, but I believe the suggested solution will only exacerbate the issues. I will go through Part 1 first, and present a different perspective on some of the issues raised. This will lead into a discussion of Part 2 and the suggested solution - documentation teams with their own Scrum.

A note on my background: I have a mighty eight months’ experience as a technical author, and about three and a half years in the IT industry overall. It is only in my current role that I have worked in Scrum properly. At my company, the technical authors are part of the Scrum teams for each project. Reading through Part 1 I developed a sense of my own good luck - it is so vastly different to my own experience. A quick poll of the two other writers here suggested that, at least at this company, technical authors as part of the Scrum is working well. I do not believe we are unusual in our implementation of Scrum (we either tick or address every box listed as Basic mechanics of following a Scrum model). Nonetheless, this is something of a “sticking my head above the parapet” post.

A note on limitations: this post follows Tom’s in focusing on the experiences and practices of dedicated technical writers. In some organisations, this debate will be of limited relevance, as the writers are developers who also document (or writers who contribute to coding). Inevitably, they will be embedded in the Scrum development team.

Part 1 opens with the following summary:

Although it seems like documentation should be treated like other features worked on by a Scrum team, frequently it is not. When technical writers try to integrate into engineering Scrum teams, they usually run into a host of challenges. These challenges stem mainly from floating across multiple projects. Often doc tasks are not assigned points or grouped in with other tasks in a real sprint, nor are technical writers co-located with project teams.

Three points stand out for me: multiple projects are a challenge (agreed), doc tasks are not being treated like development tasks (why not?) and geography can cause difficulties (agreed, but with a caveat - this is not uniquely a technical writer issue).

The pain of separation

Not being co-located with the rest of the Scrum team can be painful. “No co-location means you miss out on undocumented details” (ref)

In this context “co-location” seems to imply sitting at the same group of desks: “it doesn’t make sense to embed yourself permanently in any one area. As such, the writers usually sit by themselves in some other area of the organization.” (ref)

I believe the pain of this distance is overstated. At my company the technical writers do sit in their own group, but on the same floor as the developers, and in the same building as the science department. We use the same corridors, kitchens and watercoolers, and attend the same socials. I am not convinced we lose much by being a few meters away from the rest of the team.

The issue of separation is not uniquely a technical writer problem. Developers and testers may also sit separately. In our case, the product management team are located even further from the developers than we are, and the scientists who consult on (and provide the first end users for) our products are a whole flight of stairs away. More awkwardly, we have remote development sites, meaning that for one of my current projects, the developers are in another city, while the rest of us (testing, management, documentation, science, UI) are at head office.

The reality of large teams and organisations, sometimes across multiple sites, means that some internal documentation is necessary. Situations can arise where “you may often see tasks logged without any details, because the details are discussed during daily scrums or other hallway conversations. Developers aren’t being lazy — this is just Agile.” (ref) Certainly this would be a pain point for a technical writer who is not part of those discussions (though if the writer is part of the Scrum team then the writer should be at the daily scrums), but this is equally painful for any remote team member or function. In practice, it is essential to put proper acceptance criteria on tasks and throw together the occasional architecture document, as well as clearly written use cases. This benefits everyone, not just the technical writer.

Documentation stories in Sprint

Is it a problem with Scrum … or with how you are doing Scrum?

If your documentation tasks are not being treated like development tasks (if they are not being included in Sprint planning, tracked on the same board, pointed and tested) then I would argue you are not integrated into the Scrum team. You are already working alongside Scrum, rather than within it. This is not a problem with Scrum, it is a problem with how your team or company has implemented Scrum.

Tom’s experience is that “most project managers don’t want documentation to be considered as a sprint task, with points that affect the total sprint points possible. They prefer to keep doc tasks separate.” (ref). This is one of those “I owe my folks a beer” moments. It is accepted here that documentation work will be included in the Sprint: documentation stories created, pointed, with sub-tasks that involve input from either development, testing or science.

If you do not have the good luck to fall into a job where the writers are already integrated, you have a couple of options: work to integrate yourself or create your own separate Scrum (as suggested in Part 2). I recognise that pushing for integration is not a trivial task, but the advantages of being integrated into the Scrum developer team are considerable. The one caveat on this would be if you work as a freelancer, constantly moving between companies. If they seem receptive to new ways of doing things, great, but it may not be worth the effort of educating a new set of developers and managers every few months.

Advocate the docs … to a point

The issue of docs not being treated like other tasks appears again in the argument that “Scope doesn’t apply to doc tasks” (ref). What this seems to mean in practice is that scope does not get applied to doc tasks, not that there is something inherently scope-resistant about documentation. Nonetheless, this is a good point - holding up a release for the sake of the docs seems unlikely. However, this is not unique to documentation. Whether you release every couple of weeks, or less frequently, it is likely that marketing and support teams will also be preparing for a release date that they cannot delay. While you cannot slam the brakes on a release 24hrs before, you can make a case for good planning ahead of time. If working on multiple projects, highlight this as a risk and see if it is possible for your projects to alternate. For example, my two main projects start and finish their sprints on alternate weeks - so each week sees only one sprint end, despite being on two projects.

You can help your case by demonstrating the value of documentation: in reducing support requests, or as a marketing tool, as well as an internal resource, not “just” a tool for clients. And you are stronger within Scrum. Even if a release will not be held up for the docs, the Sprint can still be labelled as failed if documentation is not completed. This can be a useful motivator to encourage timely proofing feedback.

Perhaps we also need a dose of reality here. Obviously, given I pay my bills with it, I think documentation is important. In an ideal world, documentation would ship with the release. But a program can work without documentation. It cannot work without being coded, and it is unlikely to work well for long without being tested. Much as we would like documentation to be central to the IT universe, in practice it is secondary. To mangle a psychology concept, Maslow’s hierarchy of needs suggests physiological needs (food, water etc.) are fundamental, with things like safety, and then emotional requirements such as love and belonging, following on. We all feel safety is important (to put documentation on a comparable secondary level in IT processes is not to suggest it can be forgotten) but it is not as fundamental as coding and testing (the food and water of the IT world). Likewise, a product can release without marketing efforts being complete, or support being prepared. This is a very bad idea, but it can be done - and thus some fudging of timelines (a tutorial being finished a few days after the code is released, for example) is something we may just have to live with.

Managing multiple projects

Floating across multiple projects is one of the challenges of technical writing in Scrum. The time wasted in task switching can be substantial, and the meeting load can build up. However, these problems would apply to almost any methodology, unless you are dedicated to one project. And there are ways to mitigate them.

Agility means flexibility

Agile does not need to be a religion. Sensible modifications can be made in consultation with the Scrum team and management. The meeting load can be reduced by agreeing that some Scrum members will not attend every meeting. In our case, we do not attend sprint planning. We email the scrum master beforehand with any stories we want to bring in. This saves several hours a week, without losing out on important information. We rely on the review meeting to understand the developments in the project. This is discussed in detail in the Sprint + 1 section.

Contributing more than words

Even if the meeting load is reduced, there is another potential problem: relevance.

… as a tech writer, you’ll find that 90% of these meetings consist of developer-fueled discussions about topics mostly irrelevant to documentation. Sure, you’ll run across a nugget of relevance now and then — such as a 30-second mention of a feature that is changing, which you probably wouldn’t have realized had you not been at that meeting.

But for the most part, the signal-to-noise ratio is extremely low. The noise is higher for you than for anyone else, and you have three times this noise level because of your project load. (ref)

This feeds directly into one of the Part 1 post’s main conclusions: that integration with Scrum leads to a “Weak, passive portrayal of the tech writing role.” (ref).

When tech writers attend every meeting (even if 90% irrelevant) and contribute very little (because of the irrelevance), they’re essentially portraying themselves as a weak, passive role. What other role would attend every meeting just listening for some relevant piece of information while developers rattle on endlessly about issues irrelevant to docs? It demeans the status and importance of tech writers. If we need doc-specific info, why not set up doc-specific meetings to get the needed info? Why would we sit there listening to developers exchange developer-centric info for such long periods of unproductive time?

This depends on the approach of the technical writer. If we just sit and listen, then this is an issue (though I am not convinced the developers and testers are sitting there silently judging us as weak and passive). But why not contribute? Technical authors can have a broader perspective, and raise issues such as usability or supportability early on. True, there have been times when I have simply taken information and written it up, but there have also been occasions when I have queried something, leading to further testing or a bug being raised. We can also bring perspectives on process, and contribute to retrospectives. We are not just word processors - unless we choose to be.

I am always resistant to the “solve the meetings issue by having another meeting” approach, and the idea of a documentation-specific meeting seems like a poor use of everyone’s time (including the writer’s). You may have questions for several different team members, and questions are likely to come up as you write. Instead of more meetings, ensure the developers are accessible (whether by IM, email or - as a last resort, of course! - walking over and talking to them). This accessibility, combined with good review meetings, and Scrum tasks with clear acceptance criteria, should provide plenty of information.

Nonetheless, we will still end up listening to a lot of developer talk in scrum meetings. However, even if it is not something that needs documentation, it can still be useful to have a broader understanding of the project. It can also be interesting - glimpses of work on OWASP, headaches around AWS deployment, insights into deployment processes, and simply feeling part of the team, all come from … well, being part of the team.

Being part of a team means being unimportant

I would take the above points further, and argue that every team member has moments when they are sitting on the sidelines of discussion, or where their particular part of the project is not in focus. Being part of a large multi-function team means accepting a degree of unimportance. No function, whether documentation or UI or marketing or testing or even development, will always be the centre of attention.

Variety is the spice of life

For all the stresses caused by multiple projects, there are also benefits. As a technical author floating from team to team, we get more variety in our tasks, and an insight into how different teams and aspects of the business work. This is listed as one of the criticisms of technical writers in agile, under Variation across project teams makes for inconsistency. I agree that there can be differences between teams, but this does not have to be a negative. You get to observe experiments in different ways of implementing Scrum. You can share the benefits of this overview, spreading the good practices of one team to another.

There are also times when teams are different because people are different. Retrospectives at my company are a good example. For one team, we use a goReflect board, and find it really helpful being able to note things throughout the sprint. For another, we use post-its and discussion during the retrospective meeting itself. We tried using the goReflect board in the second team, and it did not work out. No particular reason, no major problem with either way of doing the retrospectives, just human nature leading to variety. As long as it does not lead to processes being abandoned - as long as the aims of the process are being achieved - then flexibility with implementation is a strength. It accommodates the human in a field that is generally focused on the machine.

The solutions

This section takes a look at the solution suggested in Part 2 - a separate technical writer Scrum - before presenting an alternative.

Babies and bathwater

Part 2 of the discussion of technical writers in agile lists a few solutions, and ends by advocating for a documentation-focused Scrum. This could be either a Scrum of one if working solo, or as a team with other authors at your company. He argues that this provides the best of both worlds: you avoid irrelevant meetings, drive your own backlog, and unify your focus, while retaining familiarity with the Scrum process, and working in the same way as other teams.

I feel this is a case of throwing the baby out with the bathwater. The initial problem that gave rise to this examination was a sense of loneliness (ref), of the technical writer not being immersed in the team. Separating out into our own Scrum, even if we are lucky enough to have several writers at the company, will only increase this feeling of division. The challenges posed by not being co-located will become even worse if we are no longer attending the daily scrums, reviews and retrospectives. The documentation stories may be nicely organised within the writers’ Scrum, but will be even more detached from the product development than they would be with even a bad integration. And, because this is another team, it will inevitably develop its own quirks, making for yet another different way of doing Scrum.

Sprint + 1

It is easy to criticise someone else’s suggestions, and somewhat harder to come up with a viable solution oneself. What follows is an overview of how we work at my company (a process that I can take no credit for). It has avoided most of the problems mentioned in the two posts that triggered this discussion: meeting load is manageable, technical writers are treated as valued team members, a usually solitary job is made less lonely, and the documentation benefits from the team input.

Sprint + 1 refers to our practice of working a sprint behind development. Thus: in sprint 5, a feature is developed and tested. In sprint 6, we document it. This means we do not have to try and document things that are still in flux. It also reduces meeting load.

We attend daily scrums, reviews and retrospectives. We do not attend sprint planning. By going over the development stories in JIRA, and attending the review, we gather the information we need to do the documentation stories. We add documentation stories to the backlog, each assigned two points. These are linked to completed development stories. Because we do not attend planning, we email the scrum master and product manager with the stories we want to bring in to each sprint.

A documentation story is comprised of three subtasks: documentation (for us to do the writing), proofing (for a developer, tester or scientist to check our work) and editing (for us to implement any changes suggested by the proofer). This allows fine-grained tracking of the progress of our work and ensures the involvement and investment of the whole team. We do not just gather information then go off and write. The production of the documentation is collaborative, with constant expert feedback. As well as providing a sense of team cohesion and collaboration, this process greatly improves the quality of the documentation.

We also maintain a technical author JIRA board, for tracking non-sprint team tasks. This ensures visibility and tracking of this work. In this case it is appropriate to maintain our own board, as these tasks are internal to the technical author team, and not related to any specific product.

The Joys of Scrum

Sprint + 1, outlined above, addresses some of the practical problems technical writers may experience when working in Scrum. By being fully integrated into the team, we are less lonely in our work. Increased visibility leads to our work being more widely recognised and valued. The proofing process means the quality of our output is heightened. I would add one further point: that attitude is just as important as process. This cuts both ways: it is much easier to work with accommodating, approachable developers and managers. But we also need to be flexible in our approach to process, positive about what we can bring to a team, and occasionally prepared to set ourselves and our priorities aside. Sometimes, a review meeting will include an irrelevant (to us) five minute debate over code review tools. However that same meeting may include a group of developers, testers and designers taking time to answer our queries (even though their work on the feature is done, and they would probably rather be getting on with the next task). This is teamwork, and unless you are a truly full-stack freelancer, covering everything from development to deployment to documentation yourself, it is unavoidable. Occasionally, (and especially with beer and cake) it can even be fun.