The idea
Create slowly changing artwork whose visual rhythm responds to room activity. V1 uses local acoustic features such as sound level and activity duration. It makes no claim to infer people's emotions accurately. Speech transcription, personal profiling and cloud processing are separate choices requiring a clear purpose and an understandable on-device indication.
The connection
Sound becomes a slow visual composition. The project explores an ambient relationship between a space and its artwork, with local acoustic features as a practical first experiment.
How it could work
Separate sensing, visual composition and display refresh. Compute a smoothed activity signal, map it to a small set of artistic parameters, render an image and update the display on a panel-appropriate schedule. The artwork should remain attractive when disconnected and should not flash on every sound.
Begin with a monitor or image viewer to evaluate the art before buying a large colour panel. E-paper palette, refresh latency and ghosting constrain the visual language. A framed small panel can prove the process, while a large tapestry needs separate handling, mounting and replacement planning. Treat the ESP32 and a Raspberry Pi as alternatives determined by rendering and memory needs.
What would prove it
Prepare labelled sound clips representing quiet, conversation-like activity and workshop noise, with permission to use the recordings. Replay them at controlled volume and compare the resulting feature traces and artwork. Repeat the same seed and features to verify deterministic rendering. Run an eight-hour silent/network-disconnected test.
Proposed gate: three observers can distinguish low-activity from high-activity sequences without being told the mapping; hardware mute prevents capture; no raw audio is written in the local-feature mode; a power interruption preserves the last confirmed art state. Log actual display refresh duration and energy, using its own documented operating limits.
The path to a complete build
- Develop the artistic mapping with synthetic feature traces and a conventional screen.
- Add microphone feature extraction and inspect data leaving the device.
- Integrate a small e-paper module and measure image quality and latency.
- Run the eight-hour test in a real room with the mute control visible.
- Decide whether a larger panel improves the experience enough to justify cost and handling risk.
Open the engineering notebook
Components, interfaces and calculations
Use a microphone, local processor, physical mute, status indicator and a documented display/controller. Store aggregate activity values rather than raw recordings in the first version. Keep a bounded rolling buffer, clear deletion controls and a sensor-disabled mode that still generates artwork.
An artwork record should contain timestamp, input feature window, random seed, palette, renderer version and display checksum. That enables reproducing a pleasing composition or diagnosing why an update failed without preserving a room conversation.
Scope and development questions
Plan 4–7 sessions for the small-panel proof. The artwork should earn its place before a large screen is ordered. Obtain prices for exact panel/controller combinations and a protective frame; cloud inference is optional and must be costed separately. Resolve colour, desired update rhythm, room lighting and whether anyone actually wants conversation semantics rather than abstract sound response.