Building this site

silasjelley.com is developed as a static website, wrangled into an incrementally more deliberate shape with a Python script purpose built by me for the particular needs of this hypertext creation. As time passes the software behind this site grows in scope and complexity. This document specifies the inputs and outputs of that pipeline, and records significant branches in the decision tree from which that specification has emerged.

This is the nitty-gritty of building the site, a higher level overview of its goals and design principles can be found in the design document or, if you want to go deeper still, browse the source code that scaffolds this site.

Summary

The build pipeline begins with a directory of plain-jane text files. Tucked away in the folds of that directory you’ll find1 (among other things), my journal of the last five years (approx 120,000 words), my ledger recording every penny I’ve earned and spent since I turned eighteen (~8 years ago), and my personal collection of notes (around 110,000 words). The build script is only concerned with a subset of these artefacts: those marked for publishing are picked up by the build script, put through a bevy of transclusions and transformations before being injected into the scaffolding of this website as a series of HTML documents.

From there these documents are hoisted into the stratosphere hard drive of another computer, this one somewhere on the internet, from which said documents can be retrieved by any who can wield their own computer sufficiently masochistically to have a read. Today that’s you.

TLDR: I hit buttons, you hit buttons, magic happens and somehow you see the words that I made when I hit the buttons. There’s a bunch of pipes and computers in between, but magic does a lot of the heavy lifting.

Inputs

Metadata

Document metadata is set in a triple-dash delimited special zone containing [YAML] formatted variables.

The metadata keys available for authorship (bold indicates a required element):

  • uid: A universally unique identifier ([UUID] Version 4)
  • title: The title of the document/resource
  • creator: Author of document, currently only supports a single value
  • created: Date of document creation
  • updated: Date of last update to the document, not to be updated for minor corrections/typos
  • published: Date of publishing, used to detect if a document should be output during site builds. In future may evaluate this value rather than simply checking if it’s set to allow for scheduled publishing.
  • location: Location of the author at time of writing or publishing, whichever is most relevant/accurate.
  • slug: The document path on the web-server, deliberately unlinked from the documents path on the local machine and from its title. Should not change.
  • layout: the layout template to use. The templating language used is jinja2.
  • options: Additional options to be interpreted in the build pipeline. Currently:
    • draft adds a notice at the top of a document indicating that it is in a rough draft form.
    • nofeed excludes the document from all feeds. Used for non-content documents such as the site index page.
    • modal inlines modal.js, a script to display enlarged versions of images and show alt-text on click (implemented via templates/base).
    • title-collection removes summary from items on a layout: collection page, creating a list of just document titles instead. (implemented via templates/collection).
    • alphabetical-title-collection sorts the items on a layout: collection alphabetically by title rather than the default of reverse-chronological (implemented via templates/collection).
  • tags: a list of tags for creating taxonomies. Feeds are also generated for all tags.

Some key/value pairs are added and populated during the build process:

  • interlinks: An auto-compiled list of the other documents on the site that this document links to.
  • backlinks: A list of other documents on the site that link to this document.
  • wordcount: The number of words in the content portion of the document (excludes title and other metadata).
  • filename: The local Posix file path the document originated from
  • content: The text of the document itself, having been processed from its input (markdown) into valid [HTML].

All these (the metadata specified in the document itself, and the metadata added during the build) are stored as a list within a dictionary key, each document having its own key (the required uid).

Example of one complete entry in the build pipelines `data` dictionary 
'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXXXXXX':{
    'uid': 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXXXXXX',
    'title': 'A draft design document for this site',
    'author': 'Silas Jelley',
    'created': datetime.date(2021, 8, 31),
    'updated': datetime.date(2022, 4, 14),
    'published': datetime.date(2022, 4, 21),
    'location': 'Nelson, New Zealand',
    'slug': 'design',
    'options': ['draft'],
    'tags': ['meta'],
    'wordcount': 552,
    'interlinks': '/design', '/build.py',
    'backlinks': '/about', '/design',
    'filename': PosixPath('/Users/silas/notes/design.md'),
    'content': ...

Possible future extensions to document metadata:

  • description: or summary:
  • extract: An extract pulled directly from the document itself. Perhaps too closely overlaps with description/summary above.
  • status: Indicate whether a document is in a draft, review, complete status etc
  • previous: + next: Links to the next/previous documents in a series, included in the footer.
  • confidence: Some indication of the level of confidence I have in the substance/thrust of the document. From Gwern.net
  • importance: How important I

  1. Well, you would if you broke into my tool shed and snatched my computer, and hit me with a wrench until I gave you my password and read all my super personal guff.↩︎