Bloom

Bloom is a tool to create aesthetic visualizations of a neural network’s training process. Here’s an example :

Those fractals are rendered with Fractorium using the Flame algorithm on a set of instructions that define a dynamical system. By mapping the updates to the parameters of a neural network during training to updates to the parameters of the dynamical system, we can visualize the evolution of the network. At the start of training, parameters vary wildly for both the network and the dynamical system. As the network approaches convergence, they settle down and the fractal assumes a final shape.

The videos on this page use training trajectories from Elizabeth Salesky’s machine translation research. Craig Stewart created the colour schemes out by mapping the word embedding spaces of various languages into colour space. I drew flower-like fractals to create the illusion of a bud opening as the network learns, then trembling gently in the wind as parameters keep fluttering lightly late in training. It may or may not have been a -20°C kind of winter in Pittsburgh at the time and we may or may not have all been desperate for spring at that point.