Efficient and usable graph display with 1’000 Nodes. A story about tags visualisation.

Context

This my Bookmarks / blog, where I organize things. Historically I was relying only on post title to search content back, then I used the most simplest form of jekyll tags (without using any plugins) so it can still be hosted on github.

Here I am exploring how/if I can make sense of all the tags accumulated all these years, and see the relationship between them. At least that how it started.

d3-force

This d3 module implements an auto-layout model for displaying complex graph. It’s what I choose to use as a starter for the tag graph (didn’t consider other tool).

ChatGPT was able to quicly write down a working solution, which I manage to fit with my jekyll constraint.

Some notes, on how this work.

d3-force(¹) provide the simulation layout that apply the force on nodes(²). But the user is still responsible to display the elements, so we have flexibility to hide or show node as well as in which order or style we display them.

Nodes is an array of object, they could be DOM object (from svg) or anything.
The d3-force simulation will add property for it’s own usage ( like (x,y), (vx,vy), (fx,fy). If choosing svg rendering is coming free, otherwise if using canvas this has to be done explicitly.

The Z-order has to be controlled explicitly since it’s not part of simulation itself. Using collide-force will treat node as circus and avoir overlapping.

Force applied on a given node can be directly push trough simulation by setting (fx,fy) on that node - to be clarified. Custom forces can be applied, to obtain various results.

Edges is also an array, that will be involved in link force, that will push nodes together or appart depending on the value.

Links rely on id properties of node object to identify how nodes interacts in the simulation.

Faster Display

Because of the number of nodes (~1’000) and dense relation between them (which was a surpise), the tool has a hardtime making things stand out.

So additionnal work is needed

• to make the page more responsive
• to make it more usable

This is what we explore below.

The tags page

Is quite big (3.25MB), it contains

• the html text of tags
• the list of node (tags) and their relation (link between tag) in javascript form.

All generated by jekyll liquid templating. As the information is duplicated several time, the size could probably reduced a lot:

• node/tag information is contains in the dom, and could be derived from there.
• links can be stored in js and injected in dom if we want to (see related tag unfold field).

Performance

I started using svg as any example suggest, but ChatGPT suggest to move to canvas for better display performance (this is what is tested in this page).

SVG is still used on the tags page.

• SVG code - https://jsfiddle.net/y_duf/r3dup56f/6/

• infer tags list from DOM h2 entries (to reduce page source size)
  • reduced page size from 3.5M to 2.4M (120s -> 108s generation)
  • require a slugify javascript function that match jekyll behavior, since node id a infered from DOM and other from jekyll preprocessor (in links).
  • infer size from number of post associated to tag
• infer link directly from DOM h2 ul content
  • reduced page size from 2.4M to 1.9M ( -> 96s generation)
• infer related tags direclty from DOM as well
  • reduced page size from 1.9M to 1.3M ( -> 63s generation)
  • there was a bug in jekyll tags generation, it misses some tag from posts (ex graphic was missing in 2d related tags)
• store id as original tag name and use slugify on the fly when needed
• disable javascript processing if flag is unchecked
  • move flag over graph for mobile usability

    Better rendering

Things to explore

• ❌ Layering node & label by their depth in graph relative to current selection
• use community detection (?) - I think we loose purpose except if we can do it dynamically
• ✅ size node relative to tags usage
• ❌ have distance to node be dependant on the relative count of their dependancies
• ✅ Use Canvas
  • ❌ Canvas is blurry when zooming
  • ❌ dragging a node doesn’t work

Canvas direct rendering

Example JavaScript

References