Creative Coding tools unpublished

The age of open-source software, cheap open-source hardware, and an abundance of online documentation has given us (almost) unlimited possibilities to build amazing projects.

There's a wealth of tools available for creating fun, artsy projects that combine media, graphics, and hardware, and they make it rather easy to prototype something cool, quickly. In this post I'll give an overview of "creative coding" tools that will help you get started building your own projects without drowning in low-level library or hardware details. I have used most of the tools myself, and will describe their strengths as well as link to other resources to learn more.

As the field of creative computing, digital art, and physical computing is not well defined (it's only limited by your imagination), this overview will describe a wide variety of tools that might not be interesting to everyone. However, creativity feeds itself by new experiences and possibilities, and I hope the interested reader will find new and inspiring tools to explore to make a programmable world that's more fun to live in!

… each: description (if possible link to example projects) screenshot resources similar to → do some announcements, look for resources that should link here

Visual Coding

Node-RED

..

Max/MSP

..

Scratch (& blockly?)

..

Jupyter notebooks & Mathematica

Not necessarily a tool for creative coding, but I would put Mathematica notebooks and Jupyter notebooks (generally Python) in the same category as interactive visual coding tools. They allow playful experimentation with interactive 'notebooks' that combine code, visual output, and documentation. You can run code sequentially as you read through the document, but also going back and changing, adding, and re-running code as you please, with widget support for control elements or interactive 3D WebGL views.

Notebook-based interfaces are great for writing interactive textbooks for courses and interactive tutorials and documentation, mixing regular text with runnable examples. They're sometimes even used to build and deploy production-ready code (or ML models).

Graphics & Media

Barebone libraries like OpenGL and WebGL are great if you're building a custom application. However, if you just want to prototype graphics-heavy applications with a much simpler API and less boilerplate code, there are options.

Processing

Processing is a great platform to build creative applications, and wraps popular libraries for graphics, audio, video, protocols and hardware communication into a unified and much simpler development environment. Written in Java, it has a coding editor very similar to the Arduino IDE. ..

OpenFrameworks

OpenFrameworks (OFx) is .. .. bit lower-level than Processing. Lots of c++ libraries available that are harder to get working in Processing (e.g., OpenCV, Kinect). Graphics applications might also run faster in OpenFrameworks; I reimplemented multiple Processing projects in OFx and got up to an order of magnitude speedup, which is especially significant on resource-restricted hardware platforms like the Raspberry Pi. ..

Visualisations

viz & interactive viz/storytelling

D3js

..

Blender

..

Unity3D

..

Other tools

plotting libraries: matplotlib, R libs biz graphs: kibana, grafana, Node-RED dashboard module

Hardware and Physical Computing

Creative projects become more interesting when physical elements are added, and the boundaries between digital and physical start to blend. It's now easier than ever to program the physical world, and make even the dumbest object "smart", or at least more interesting to interact with.

Hardware platforms like Arduino and semi-standardised hardware shields and software libraries have given a large boost to the maker movement. Enabling newcomers to start hardware projects more easily isn't the only benefit here: also more advanced hardware tinkerers can iterate on prototypes much faster without reinventing the wheel all the time.

Raspberry Pi

.. most powerful, ARM .. there are other platforms

Arduino

..

NodeMCU / ESP8266 / ESP32-CAM

..

MakeyMakey

..

Other tools

Smartphone: mobile phones contain a bunch of sensors, a touchscreen, WiFi, Bluetooth, 9DOF motion/orientation sensors, and GPS, and have a well-established development environment. If you just need an internet-connected touchscreen with basic sensors, it's often cheaper to buy a (cheap) android phone than to build your own device. Commercial Smart Home devices: there are many internet-connected devices that will get a project up-and-running at the expense of a little more money, and commercial products are generally more reliable and there are often APIs or libraries available. Think internet-connected power switches, simple motion detectors, door sensors (reed switches), cool decorative lights, voice-interface speakers, cameras, and more to be invented and commercialised. 3D cameras: understanding the 3D structure we're in is a powerful input for many creative applications. Commercial devices return a real-time, rich 3D point cloud ready to be consumed, and often have APIs for tracking people and fingers. Popular 3D cameras include the Microsoft Kinect, Intel RealSense series, and the Leap Motion (finger tracking). Other sensors: ..

Resources

Physical computing, Making things Talk

Have fun!