Software specification documents serve as reference manuals for user interface designers, programmers who write the code and testers who verify that the software works as intended.
In a multi-module application, each component is developed and released individually. Keeping the documents up to date for each component is not easy because it is not only about writing, but also about centralizing all the documents, so they can be easily found by the interested people.
The goal of this article is to present an approach that simplifies this process, relying on Build Automation, to extract and publish the documents.
We will take you through the configuration process of Apache Maven artefacts Jenkins CIserver, and eventually, the creation of a generated project website that reflects the current status of the project, offering access to all available documents from a single place.
Our case study is a framework written in Java programming language, consisting of more Maven modules. Each module contains one or more documents written in Markdown . The documents are written by developers and they are readmes, howtos, technical documentations and others.
Before stepping into the process, we should first imagine how the end result will be like. We want a front html page that presents the latest documents written by developers of the released modules.
It is a common practice to release JavaDoc, release notes and other documents together with the software product. They actually accompany the software product, so the product can be used and understood by the consumers.
By releasing the artifacts with Maven the compiled files end up in a structured Repository. You can release not only the compiled artifacts, but also the JavaDoc, Cross-Reference of source code, or a whole site presenting information about the project dependencies, issue tracking, continuous integration, team and many others.
The project site is the most important element in achieving our goal because our aggregating website will link the websites of each individual module into a single html page.
Imagine the html page containing the followings:
<div>
<h1>module1</h1>
<a href=”modules/module1/index.html”>About</a>
<a href=”modules/module1/Readme.html”>Readme</a>
<a href=”modules/module1/ReleaseNotes.html”>Release Notes</a>
</div>
<div>
<h1>module2</h1>
<a href=”modules/module2/index.html”>About</a>
<a href=”modules/module2/Readme.html”>Readme</a>
<a href=”modules/module2/ReleaseNotes.html”>Release Notes</a>
</div>This requires the presence of the referenced files on the disk, generated during the process.
Our modules are Maven artifacts. The approach relies on Maven Build Lifecycle.
We generate the site by using maven-site-pluginWith this plugin alone, we can only generate the default bits, such as Project Summary, Project Plugins, Dependencies, and others.
With some configuration and help, we can intervene in the plugin›s normal flow and include our Markdown files as html documents referenced from the generated site.
In order to transform the Markdown files into html, we need the doxia-module-markdown as dependency for this plugin. With this in place, the site generation process looks inside src/site/markdown and converts each .md file into html.
This might sound easy, but if our *.md files have images, for example, they are simply ignored by the plugin. The site plugin only copies the contents of src/site/resources.
But we want our Markdown files to be accessible also offline, so the developers can always have a look. By referencing the images from src/site/resources will only work offline, because after the site is generated, the resources folder will not be present, so we will end up with broken links.
What we would really want is to have our Markdown files in src/site/resources, referring images from src/site/resources/images, so simply from images (as relative directory), because after site generation, the contents of images is merged with other images that the site generates in the target/site/images directory.
In conclusion, the module›s src directory has the following structure:
Inside the Readme.md we find a reference to images:
![Alternative text]
(images/image1.jpg "Text descriptiv")After the site is generated, we expect to find our files in the target directory as following:
The 2 html files should be referenced in the generated site html files. In order to make this happen, we need a way to tell Maven the followings:
Before you generate the site, please copy all *.md files from src/site/resources into src/site/markdown. Be careful, the new directory does not exist.
Generate the site for the project, but in the end result I would like to have references to my Markdown based documents. I know I have a Readme.md and a ReleaseNotes.md, so you should link to Readme.html and ReleaseNotes.html.
We start with the second point, by describing in src/site/site.xml the links to our future html files:
<?xml version=”1.0” encoding=”UTF-8”?>
<project xmlns=”http://maven.apache.org/DECORATION/1.4.0” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://maven.apache.org/DECORATION/1.4.0 [http://maven.apache.org/xsd/decoration-1.4.0.xsd](http://maven.apache.org/xsd/decoration-1.4.0.xsd) [http://maven.apache.org/DECORATION/1.4.0](http://maven.apache.org/DECORATION/1.4.0) „>
<!-- skin, banners and other site configs -->
<body>
<!—- menu entry for developer documents -->
<menu name=”Developer documents”>
<item name=”Readme” href=”Readme.html”/>
<item name=”ReleaseNotes” href=”ReleaseNotes.html”/>
</menu>
<!-- menu entry for javadoc, jxr, and others -->
<menu ref=”reports”/>
</body>
</project>The plugin reads the instructions from the above file and generates the site accordingly. We added a new menu entry containing our documents.
First and last points involve file manipulation, and this is the perfect job for an Ant task. In Maven, we can use the maven-antrun-plugin, and we configure it with 2 executions:
first execution creates the src/site/markdown directory and copies all the *.md files in it. This must be done before the site starts to generate, so in pre-site phase;
the second execution removes the src/site/markdown directory, and it must be done after the site is generated, so after the site phase.
The resulting pom.xml is presented below:
<project .. >
<dependencies>
...
</dependencies>
<build>
<!—- prepares Markdown files for maven site-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>pre-markdown</id>
<phase>pre-site</phase>
<configuration>
<tasks>
<delete dir=”${project.basedir}/src/site/markdown” />
<mkdir dir=”${project.basedir}/src/site/markdown” />
<copy todir=”${project.basedir}/src/site/markdown”>
<fileset dir=”${project.basedir}/
src/site/resources” includes=”**/*.md” />
</copy>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
<execution>
<id>post-markdown</id>
<phase>site</phase>
<configuration>
<tasks>
<delete dir=”${project.basedir}/src/site/markdown” />
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
<!—Site generator is related to the reporting section -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.3</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.doxia</groupId>
<artifactId>doxia-module-markdown
</artifactId>
<version>1.5</version>
</dependency>
</dependencies>
</plugin>
</build>
<reporting>
<outputDirectory>${project.build.directory}/site</outputDirectory>
<!—config other reporting related things, such as maven-javadoc-plugin, maven-jxr-plugin -->
</reporting>
</project>By calling mvn site, it will generate the desired site into the target/site directory.
The next phase in achieving our goal is to wrap the generated site into a jar file and upload it into a Maven Repository.
The site plugin knows how to create a jar file from the files inside target/site directory. All we need to do is to call mvn site:jar, with one remark: the pre-site phase gets executed only if we invoke mvn site, without :jar goal. To make sure the Markdown files are taken into consideration even when the target is empty or not present, we should call mvn site site:jar.
The result is a new jar file, target/module1-site.jar. All we need to do now to consider this step complete is to upload this jar file into Maven Repository. It can be done by using the Maven Deploy Plugin.
The purpose of this project is to aggregate all the available resources into a single website. Besides the modules documentation, it can also hold general documents, like first steps for developers or project technical overviews. For those, the maven-site-plugin can be applied in the same manner as for the modules.
In order to download the generated sites we use Maven Dependency Plugin. It helps us retrieve the artifacts and the *-site.jar deployed at previous step. Our goal is to expand all the site archives into target/site/modules, so we can maintain the desired website structure.
To get the *-site archives for the modules, all must be declared as dependencies of the resources project in pom.xml:
<project .. >
<dependencies>
<!-- module 1 -->
<!-- module 2 -->
<!-- ... -->
<dependencies>
<build>
<plugins>
<!-- antrun to generate additional html from markdown -->
<!-- (!) -->
<!-- groovy plugin to perform I/O operations on disk, explained later -->
<!-- (!) -->
<!-- site plugin to generate site from the current project -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin
</artifactId>
<executions>
<execution>
<id>sites-modules</id>
<phase>compile</phase>
<goals>
<goal>unpack-dependencies</goal>
</goals>
<configuration>
<classifier>site</classifier>
<!—this is important, here are enumerated all artifacts that are project libraries, separated by comma (,) -->
<includeArtifactIds>module1, module2, ...</includeArtifactIds>
<failOnMissingClassifierArtifact>false
</failOnMissingClassifierArtifact>
<outputDirectory>
${project.build.directory}/site/modules
</outputDirectory>
<useSubDirectoryPerArtifact>true
</useSubDirectoryPerArtifact>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>This plugin will expand all modules sites contents into separate directories inside the target/site/modules of the resources project.
The last important bit here is to design the index.html in such a way that it links to the sub-sites of the modules. Because our modules have versions, we want our website project to figure out itself the paths of the sub-sites. By making the index page dynamic, we can simply add a script that populates the page with the corresponding content, by declaring an array in a separate.js file, as presented below:
var modules = [
"module1-1.3-SNAPSHOT-site-jar",
"module2-1.5-site-jar",
...
];The JavaScript code can use the modules array and insert the following DOM elements into the index page:
<div>
<h1>module1</h1>
<h2>Version 1.3-SNAPSHOT</h2>
<a href=”modules/module1-1.3-SNAPSHOT-site-jar/index.html”>About</a>
<a href=”modules/module1-1.3-SNAPSHOT-site-jar/Readme.html”>Readme</a>
<a href=”modules/module1-1.3-SNAPSHOT-site-jar/ReleaseNotes.html”>Release Notes</a>
</div>
<div>
<h1>module2</h1>
<h2>Version 1.5</h2>
<a href=”modules/module2-1.5-site-jar/index.html”>About</a>
<a href=”modules/module2-1.5-site-jar/Readme.html”>Readme</a>
<a href=”modules/module2-1.5-site-jar/ReleaseNotes.html”>Release Notes</a>
</div>Our modules.js file is populated during the build of resources project with help from groovy-maven-plugin. The goal is to execute a code that lists the directories inside the target/site/modules and print the names in/site/config/modules.js, so we can obtain our array of module paths. The code is listed below:
...
<plugin>
<!—writes in config/modules.js the names of the corresponding directories -->
<groupId>org.codehaus.mojo</groupId>
<artifactId>groovy-maven-plugin</artifactId>
<version>1.5</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
<source>
<![CDATA[println("==== Creează config/modules.js ====");
File modFile = new File("${project.build.directory}/site/config/modules.js");
BufferedWriter modWriter = new BufferedWriter(new FileWriter(modFile));
modWriter.writeLine("var modules = [");new File( "${project.build.directory}/site/modules").eachDir() { dir -> modWriter.writeLine("'" + dir.getName() + "',");
}
modWriter.writeLine("];");
modWriter.close();
]]>
</source>
</configuration>
</execution>
</executions>
</plugin>
...By invoking mvn site site:jar on resources project, we obtain the archive of the desired website. The archive can be easily expanded into a HTTP Web Server and made available to everyone interested.
By having all the modules configured and the resources project created, all mvn commands can be ran by Jenkins CI easily, and the final website can be deployed on the HTTP Web Server as post build step. Every time a module is released, its sub-website is published and the main website can be regenerated. This way we ensure that the latest documentation is available for developers without any additional effort or intervention and it’s done in the spirit of Continuous Integration.