Asciidoctor for Professional Looking Documentation

· by Christoph Stoettner · Read in about 8 min · (1678 words)

For GPN19 I prepared a second talk on Documentation with any Editor. The talk was based on a previous one from Froscon 13, but the pipeline tooling changed.

This time there was a technical issue during the recording and so there are only the slides available, but you can still watch the video of the Froscon talk: Froscon 13: Documentation with any Editor

All scripts and the complete pipeline definition can be found in the GPN19-documentation Gitlab Repository.

tl;dr

I use Asciidoctor for documentation for years. It’s just easy to write, has more options than Markdown and I create all kinds of output for colleagues, customers or the web, just from one source. Converting the source to HTML, PDF, EPUB or several other formats, can be done on the command line of any Operating system, in Docker containers or delivery pipelines. I decided to use the Asciidoctor Docker container with Gitlab CI/CD. One of the advantages of Asciidoctor is that it can be easily put into a version control system like Git. I used Git for storing it since the early beginning, so the step to use a pipeline in combination was easy.

But why do I use Gitlab instead of other git software and Jenkins? To be honest, I think it’s easier to start with Gitlab, because it’s completely integrated. Gitlab is available on premises (Docker container, Kubernetes, bare server) and public on https://gitlab.com.

Goal

As the title of the talk mentions, the main advantage to use a pipeline with Asciidoctor is, that you can edit your documents with the editor of your choice (I often use even VIM in a termux session on my mobile phone or tablet). Conversion happens after commit and push to the git repository.

Advantage

I can be sure that always the latest version is available, changing something (typos, additional information) doesn’t need any copy & paste to whatever other systems.

Building the pipeline and documents

Directory structure and documents
.
├── docker-asciidoctor     (1)
│   └── Dockerfile
├── documents-personal
│   └── basic-example.adoc
├── documents-work
│   ├── _attributes.asciidoc
│   ├── basic-example.adoc
│   ├── configfiles        (2)
│   │   └── customization
│   │       └── themes
│   │           └── Theme
│   │               ├── applications
│   │               │   └── blogs.css
│   │               └── custom.css
│   ├── example.adoc
│   ├── images
│   ├── include
│   │   ├── network2.adoc
│   │   └── network.adoc
│   ├── main-example.adoc
│   ├── more.asciidoc
│   ├── _variables-linux.asciidoc
│   ├── _variables-project.asciidoc
│   └── _variables-win.asciidoc
├── images                 (3)
│   ├── diag-4e35056b3ac38736b7ce541f3f9bcced.png
│   ├── icons
│   │   ├── caution.png
│   │   ├── example.png
│   │   ├── home.png
│   │   ├── important.png
│   │   ├── LICENSE
│   │   ├── next.png
│   │   ├── note.png
│   │   ├── prev.png
│   │   ├── README.adoc
│   │   ├── tip.png
│   │   ├── up.png
│   │   └── warning.png
│   ├── logo.png
│   ├── png.png
│   └── test.png
├── LICENSE
├── pdftheme               (4)
│   ├── logo.png
│   ├── personal-theme.yml
│   └── work-theme.yml
├── presentations
│   └── slidedeck.adoc
├── public                 (6)
│   ├── html-personal
│   ├── html-work
│   ├── images
│   ├── pdf-personal
│   ├── pdf-work
│   └── presentations
├── README.adoc
├── scripts                (5)
│   ├── create-index.sh
│   ├── html-conversion.sh
│   ├── html-work-conversion.sh
│   ├── pdf-conversion.sh
│   ├── pdf-work-conversion.sh
│   ├── reveal-conversion.sh
│   └── scripts
└── wiki-articles
1 Git submodule pointing to https://github.com/asciidoctor/docker-asciidoctor
2 Some example files to include in Asciidoctor
3 images folder used with all asciidoc files
4 pdftheme for company logo on output
5 some scripts for conversion and generating an overview of all documentation
6 Output folder which is published to Gitlab pages

The idea was to put documents to documents-work and it will generate html5 with side toc and pdf with a company logo on the title page. The pdf theme adds a logo, author, pages and copyright to the headers and footers of each page. The personal documents just used the default themes.

The folder structure is already prepared for confluence wiki and epub, but it’s not implemented until now.

Pipeline definition

Just add .gitlab-ci.yml to your repository and Gitlab will run your defined pipeline.

Build and test the Docker container

I decided to build the container directly in the Gitlab project, run some tests and put it into the included registry. Running Docker in Docker needs some work on the on premises deployments of Gitlab, but it works out of the box on the public one.

I add the original Github repository as a submodule to my project.

git add submodule https://github.com/asciidoctor/docker-asciidoctor
gitlab-ci.yml for building the container and tests
image: docker:git   (1)

services:
- docker:dind       (2)

variables:          (3)
  CONTAINER_TEST_IMAGE: registry.gitlab.com/stoeps/$CI_PROJECT_NAME:$CI_BUILD_REF_NAME
  CONTAINER_RELEASE_IMAGE: registry.gitlab.com/stoeps/$CI_PROJECT_NAME:latest

before_script:      (4)
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
  - git submodule sync --recursive
  - git submodule update --init --recursive

stages:             (5)
  - build
  - test
  - release

build:              (6)
  stage: build
  script:
    - docker build -t $CONTAINER_TEST_IMAGE docker-asciidoctor
    - docker push $CONTAINER_TEST_IMAGE
  only:             (7)
    changes:
      - docker-asciidoctor/Dockerfile
    refs:
      - master
    variables:
      - $BUILDCONTAINER

test 1:             (8)
  stage: test
  script:
    - echo "Run tests here"
    - docker run -t --rm $CONTAINER_TEST_IMAGE asciidoctor -v | grep "Asciidoctor"
  only:
    changes:
      - docker-asciidoctor/Dockerfile
    refs:
      - master
    variables:
      - $BUILDCONTAINER

test 2:
  stage: test
  script:
    - docker run -t --rm $CONTAINER_TEST_IMAGE asciidoctor-revealjs -v
  only:
    changes:
      - docker-asciidoctor/Dockerfile
    refs:
      - master
    variables:
      - $BUILDCONTAINER

release-image:      (9)
  stage: release
  script:
    - echo "Tag Image and push to registry"
    - docker pull $CONTAINER_TEST_IMAGE    (10)
    - docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_IMAGE  (11)
    - docker push $CONTAINER_RELEASE_IMAGE   (12)
  only:
    changes:
      - docker-asciidoctor/Dockerfile
    refs:
      - master
    variables:
      - $BUILDCONTAINER
1 Use docker:git image (work with git repository)
2 dind → Docker in Docker
3 Define Variables (Test image and release image name)
4 Run this scripts before the build starts (login in to registry, pull submodule)
5 3 Stages used in the pipeline (build, test and release)
6 Job build in stage build
7 Run only when Dockerfile is newer, in branch master and when the $BUILDCONTAINER variable is true
8 Job test1 in stage test
9 Job release image in stage release
10 Get test image from registry
11 Tag as release image
12 Push to registry

So the image build runs only when a variable SBUILDCONTAINER is set to true. I did this to save time, because we don’t need to build the container image multiple times. I decided to create a scheduler for the weekend, which sets the variable and is scheduled all week.

gitlab ci schedules
Figure 1. Schedules on Gitlab

And even then, it needs to be a new Dockerfile to run this.

A successful build looks like this:

gitlab ci container
Figure 2. Container build pipeline view

So we see our three stages and the different jobs. Splitting into stages has some advantages. First of all, jobs are running parallel in one stage, when a stage (or job) is not successful, the following stages do not run.

Build the output documents

I decided to run the built of the documents separately, so not the best when you want to save build time, but easier to mount the scripts, images, and documents as volumes into the container. After a successful build, the output is copied to the folder public, a script runs to build an overview html page and then it’s published to Gitlab pages.

I use the script for an overview because I couldn’t find a way to enable directory index for the Gitlab pages.

To do so, I added to additional stages (Conversion and Deploy)

After a successful build, the output is copied to the folder public, a script runs to build an overview html page and then it’s published to Gitlab pages.

I use the script for an overview because I couldn’t find a way to enable directory index for the Gitlab pages.

...

stages:
  - build
  - test
  - release
  - conversion
  - deploy

...

pdf:personal:     (1)
  stage: conversion
  script:
    - echo "Start Asciidoctor conversion"
    - echo $CONTAINER_IMAGE:$CI_COMMIT_SHA
    - docker run -t --rm -v $(pwd)/documents-personal:/documents/ -v $(pwd)/images:/images -v $(pwd)/scripts:/scripts $CONTAINER_RELEASE_IMAGE /scripts/pdf-conversion.sh  (2)
    - cp documents-personal/*.pdf public/pdf-personal  (3)
  artifacts:  (4)
    name: "pdf-personal-$CI_COMMIT_TAG"
    paths:
      - public/pdf-personal
    expire_in: 2 hours
  except:     (5)
    variables:
      - $BUILDCONTAINER

pdf:work:
  stage: conversion
  script:
    - echo "Start Asciidoctor conversion"
    - echo $CONTAINER_IMAGE:$CI_COMMIT_SHA
    - docker run -t --rm -v $(pwd)/documents-work:/documents/ -v $(pwd)/images:/images -v $(pwd)/scripts:/scripts -v $(pwd)/pdftheme:/pdftheme/ $CONTAINER_RELEASE_IMAGE /scripts/pdf-work-conversion.sh
    - cp documents-work/*.pdf public/pdf-work
  artifacts:
    name: "pdf-work-$CI_COMMIT_TAG"
    paths:
      - public/pdf-work
    expire_in: 2 hours
  except:
    variables:
      - $BUILDCONTAINER

html:
  stage: conversion
  script:
    - echo "Start Asciidoctor conversion"
    - echo $CONTAINER_IMAGE:$CI_COMMIT_SHA
    - docker run -t --rm -v $(pwd)/documents-work:/documents/ -v $(pwd)/images:/images -v $(pwd)/scripts:/scripts $CONTAINER_RELEASE_IMAGE /scripts/html-work-conversion.sh
    - cp documents-work/*.html public/html-work
    - docker run -t --rm -v $(pwd)/documents-personal:/documents/ -v $(pwd)/images:/images -v $(pwd)/scripts:/scripts $CONTAINER_RELEASE_IMAGE /scripts/html-conversion.sh
    - cp documents-personal/*.html public/html-personal
  artifacts:
    name: "html-$CI_COMMIT_TAG"
    paths:
      - public/html-work
      - public/html-personal
      - images
    expire_in: 2 hours
  except:
    variables:
      - $BUILDCONTAINER

reveal:
  stage: conversion
  script:
    - echo "Start Asciidoctor conversion"
    - echo $CONTAINER_IMAGE:$CI_COMMIT_SHA
    - docker run -t --rm -v $(pwd)/presentations:/documents/ -v $(pwd)/images:/images -v $(pwd)/scripts:/scripts $CONTAINER_RELEASE_IMAGE /scripts/reveal-conversion.sh
    - cp presentations/*.html public/presentations
  artifacts:
    name: "html-$CI_COMMIT_TAG"
    paths:
      - public/presentations
      - images
    expire_in: 2 hours
  except:
    variables:
      - $BUILDCONTAINER

pages:
  stage: deploy
  dependencies:   (6)
    - html
    - reveal
    - pdf:personal
    - pdf:work
  script:
    - cp -arvf images/* public/images/
    - sh scripts/create-index.sh    (7)
    - chmod +r public -R
  artifacts:
    paths:
      - public
    expire_in: 1 hour
  only:
    refs:
      - master # this job will affect only the 'master' branch
  except:
    variables:
      - $BUILDCONTAINER
1 Job pdf:personal in stage conversion
2 docker command, mounting three volumes into the container and convert to pdf then
3 Copy resulting documents to public
4 Save artifacts, so the outputs of the conversion are saved
5 This time this runs always, except when $BUILDCONTAINER is true
6 The pages job has the other jobs from stage conversion as dependencies, so their artifacts can be used
7 build an overview of all converted documents
gitlab ci conversion
Figure 3. Pipeline view on Gitlab

Gitlab Pages

When we now look at https://stoeps.gitlab.io/gpn19-documentation, we see the built overview page.

gitlab ci pages
Figure 4. Overview on Gitlab pages

Check https://stoeps.gitlab.io/gpn19-documentation/pdf-work/main-example.pdf for an overview of a converted PDF file with title page, logo, plantuml conversion and so on. The source documents can be found in the repository itself with the same name (but extension .adoc).