Evaluation of approaches for documentation in continuous software development

Abstract

With the adoption of values, principles, practices, tools and processes from Agile, Lean, and DevOps, knowledge preservation has become a serious issue because documentation is largely left out. We identify two questions that are relevant for knowledge acquisition and distribution concerning design decisions, rationales, or reasons for code change. The first concerns which knowledge is required upfront to start a project. The second question concerns continuation after initial development and addresses which knowledge is required by those who deploy, use or maintain a software product. We evaluate two relevant approaches for alleviating the issues, which are 'Just enough Upfront' and 'Executable Documentation' with a total of 25 related artifacts. For the evaluation, we conducted a case study supported by a literature review, organizational and project metrics, and a survey. We looked into closed source-code and closed classified source-code. We found two conclusive remarks. First, git commit messages typically contain what has been changed but not why source-code has been changed. Design decisions, rationale, or reasons for code change should be saved as close as possible to the source-code with Git Pull Requests. Second, knowledge about a software product is not only written down in artifacts but is also a social construction between team members.