A GitHub Actions Pipeline to Generate OpenAPI Documentation
Introduction
This is my first post on GitHub Actions (GHA), the pipeline part of GitHub (GH), and GitHub Pages (GHP), the static hosting service. I will give an example of a pipeline I built for producing documentation for an OpenAPI spec. Seasoned devops/platform engineering people will probably find this very basic, but I hope it can have interest to someone who is starting to play with GHA! I chose the GitHub products because, essentially, they are available for free, and yet are very powerful.
Desired Functionality
I would like to:
- Produce HTML documentation from an OpenAPI specification located in some URL, or from a local file, and make it available somewhere
- Store each HTML version of the specification independently, e.g., one file for version 1.0.0, another one for 1.0.1, etc
- Have a template for the documentation, with the ability to add additional contents - header, footer - optionally, as well as insert custom JavaScript and CSS
- Have the generation run on demand
- Be able to change the OpenAPI specification URL easily
Let's get started!
OpenAPI
By now, you probably know what OpenAPI is; previously called Swagger, it is a specification for describing REST web APIs. Majour programming languages and framework allow building web APIs that conform to this specification, like ASP.NET Core does.
GitHub Actions
GitHub Actions is the Continuous Integration/Continuous Deployment (CI/CD) feature of GitHub. It allows building workflows to automate the build, test, and deployment process. Learn about it here.
GitHub Pages
GitHub Pages is a site hosting service that takes static assets such as HTML, CSS, and JavaScript files straight from a repository in GitHub, and optionally runs them through a pipeline defined as a GHA. Read more about GitHub Pages here.
External Libs
To help with this, I will be using some external tools as NPM packages or Linux commands. Here is a list of them.
Curl
Curl should not require introduction: it is a full-featured command line HTTP client. We'll use it to retrieve the OpenAPI specification and store it locally.
Redocly
Redocly is an open-source tool, available as NPM, for generating HTML pages from an OpenAPI specification. It uses an Handlebars file as a template for the page to produce.
markdown-to-html
markdown-to-html is an NPM tool to convert Markdown (MD) files into HTML. We'll use this for the case when we want to add custom MD files to the header and/or footer of the page template.
Redocly Template
We will be using this template, a slightly modified version of the default one (template.hbs):
<!DOCTYPE html>
<html>
<head>
<meta charset="utf8" />
<title>{{title}}</title>
<!-- needed for adaptive design -->
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
body {
padding: 0;
margin: 0;
}
</style>
{{{redocHead}}}
{{#unless disableGoogleFont}}<link href="https://fonts.googleapis.com/css?family=Montserrat:300,400,700|Roboto:300,400,700" rel="stylesheet">{{/unless}}
</head>
<body>
<header>
{header}
</header>
{{{redocHTML}}}
<footer>
{footer}
</footer>
</body>
<script>
{script}
</script>
<style>
{style}
</style>
</html>
The syntax comes from Handlebars, it is a mix of pure HTML and some specific tokens. You may have noticed the {header}, {footer}, {script}, and {style} tokens, these are not Handlebars, we defined them, they will be used by us to inject bespoke contents.
Workflow Definition
Now, this is the actual workflow definition:
name: Generate Redoc documentation on: push: branches: [ master ] workflow_dispatch: branches: [ master ] env: EMAIL_DOMAIN: 'users.noreply.github.com' REPO: '${{ github.event.repository_name }}' USER: '${{ github.actor }}' API_URL: '${{ vars.API_URL }}' jobs: generate-redoc-documentation: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v4 - name: Install Node.js uses: actions/setup-node@v4 - name: Install NPM packages run: | npm install -g @redocly/cli@latest npm install -g markdown-to-html - name: Get OpenAPI spec file if: ${{ github.event_name == 'workflow_dispatch' }} run: curl $API_URL > openapi.json - name: Set env variables run: | echo "NOW=$(date +'%Y%m%dT%H%M%S')" >> $GITHUB_ENV echo "VERSION=$(grep version openapi.json | head -1 | awk '{split($0,a,"\""); print a[4]}')" >> $GITHUB_ENV - name: Modify template run: | markdown HEADER.md > HEADER.html 2>/dev/null markdown FOOTER.md > FOOTER.html 2>/dev/null sh replace.sh - name: Run Redocly run: | [ ! -d "$REPO.github.io" ] && mkdir "$REPO.github.io" redocly build-docs openapi.json -t template_complete.hbs -o $REPO.github.io/index.html cp $REPO.github.io/index.html $REPO.github.io/index-$VERSION.html - name: Commit run: | git config user.email $USER@$EMAIL_DOMAIN git config user.name $USER git add openapi.json git add $REPO.github.io/index.html git add $REPO.github.io/index-$VERSION.html git diff-index --quiet HEAD || git commit -m "Generated at $NOW for $USER and version $VERSION" git push
I won't go into all of the detail, but I will explain how this works.
GitHub Action Variables
Here we make some use of GitHub Action variables, these can be stored at different levels:
- Organisation: for an entire GitHub organisation, all repositories, environments, and workflows, will be able to see them
- Repository: for a specific repository inside an organisation, inherited by all environments and workflows
- Environment: for a build environment, such as Dev or Prod, which will be accessible by all workflows running under that environment
- Workflow: for individual workflow, such as the one we are defining now, and all steps will have access to them
- Step: defined inside a particular step of a workflow
Organisation, repository, and environment-level variables are configured on the GitHub user interface, which means they can be changed easily.
The variables we will use are:
- EMAIL_DOMAIN: a workflow variable that is unchanging, meaning, it contains a constant string; this will be used for the GitHub configuration of the user's email
- REPO: also a workflow variable that is obtained automatically from the execution context and contains the current repository name (github.event.repository_name)
- USER: also a workflow variable that contains the username of the person who triggered the workflow (github.actor)
- NOW: a step variable that will hold the current timestamp from the date command
- VERSION: another step variable that contains the OpenAPI file version and will be used for the filename generation, this is obtained from the OpenAPI spec file; for this, we use a number of commands (grep, head, awk)
There are other ways to store configuration, such as secrets, but I won't cover them here, maybe on a future post.
For the purpose of this post, I created a repository variable called API_URL:
A repository variable |
Workflow Steps
A quick explanation of the workflow steps is in order:
- This workflow will be triggered by any of two events, both on the master branch:
- A commit to the repository where it lives (on push)
- On demand (on workflow_dispatch)
- Runs on Ubuntu Linux, latest release available (runs_on)
- Some workflow variables are defined from different sources, as explained (env)
- The first steps first install some required libs (actions/checkout, actions/setup-node, the @redocly/cli and markdown-to-html NPM packages)
- If the workflow was manually triggered (workflow_dispatch), it fetches the OpenAPI specification using curl and stores it locally on a file called openapi.json
- It then sets two step environment variables, NOW from the current timestamp, and VERSION, from the version extracted from the OpenAPI specification file
- We then proceed to generating HTML files from the MD ones, and replacing the tokens on the template file
- Before we run Redocly we create the target folder, if it does not exist, and then output to the GitHub Pages folder (the repo name plus .github.io) twice, one with name index.html and the other index-VERSION.html, where VERSION is the version extracted from the OpenAPI spec
- Finally, we set some git configuration properties, add the generated files, and check if there is anything to commit, in which case, we do so and push to the local repository (this is needed)
I think these should be easy to understand, as I said, I won't go into too much detail about building a workflow, there are lots of tutorials out there.
Template Customisation
We want to be able to add external files to the "donut holes"/tokens ({header}, {footer}, etc) that we added to the template file, for that, we will use sed, in a custom shell script (replace.sh):
rm -f template_complete_*.hbs sed '/{header}/ { s/{header}//g r HEADER.html }' template.hbs > template_header.hbs sed '/{script}/ { s/{script}//g r SCRIPT.js }' template_header.hbs > template_script.hbs sed '/{style}/ { s/{style}//g r STYLE.css }' template_script.hbs > template_style.hbs sed '/{footer}/ { s/{footer}//g r FOOTER.html }' template_style.hbs > template_complete.hbs
If the files exist and are not empty, then they are inserted where the tokens are found, otherwise, the tokens are just removed. In the end, this produces a file named template_complete.hbs, which is then passed along to redocly.
Conclusion
I think this should be enough to spike your curiosity and to realise that GH, GHA, and GHP make up a powerful CI/CD environment. Hopefully, for those who are less familiar, this may be useful to get started! As always, looking forward for your comments!
Comments
Post a Comment