Sonification of neural data before and after musical mapping

Musical mapping improves pitch control, rhythm, and listener access. It also removes information.

That trade-off is not a flaw in the method. It is the method. The useful question is not whether neural activity should sound “musical.” It is which properties of the recording must survive the translation, which can be transformed, and which changes will mislead the listener.

For electrophysiological recordings, the source may be a continuous voltage trace, a local field potential, an EEG band, or a train of discrete spikes. Each format carries a different structure. Each demands a different mapping strategy. Treating them as interchangeable produces attractive audio with weak analytical value.

The first split: audification is not musical sonification

The word sonification covers several operations. The simplest is direct audification: move a signal into the audible range and play it with minimal transformation. A continuous voltage trace remains continuous. A spike train remains a sequence of events. The listener receives the temporal structure with limited interpretation added by the system.

This approach preserves raw timing better than a musical scale does. It also produces difficult audio.

Neural recordings are not designed for the ear. Spike intervals can be highly irregular. Voltage traces may contain slow drifts, abrupt transients, electrical interference, and amplitude ranges that do not translate cleanly into acoustic parameters. Direct playback can generate clicks, pops, extreme pitch jumps, or long stretches with little audible variation. The result may be technically faithful and perceptually unusable.

Before playback, the signal often requires conditioning:

• Filtering isolates the frequency or event range relevant to the intended sound.
• Normalization prevents one large amplitude excursion from flattening the rest of the recording.
• Resampling or time compression places slow neural events inside an audible time scale.
• Thresholding can convert a continuous trace into discrete events, but this creates a new representation rather than revealing an unchanged one.
• Envelope control prevents abrupt level changes from becoming audio artifacts.
• Artifact rejection removes recording failures that would otherwise dominate the output.

These operations should be logged. A listener cannot infer from the final audio whether a sharp click came from neural activity, a threshold transition, a bad contact, or an unhandled discontinuity.

Raw neural data sonification preserves irregularity first. Musical mapping preserves usability first. Neither is neutral.

A useful production system therefore keeps two outputs. The first is a high-fidelity audit signal. The second is a perceptual version designed for listening, installation, or performance. If the musical rendering becomes the only artifact, later interpretation becomes difficult. The system has hidden its own transformations.

What direct sound can retain

Raw neural data sonification is strongest when timing is the primary variable. A spike train can be represented through brief impulses, clicks, or triggered tones. Inter-spike intervals remain legible as changes in density and rhythm. A continuous field potential can control amplitude or filter movement while retaining slow fluctuations.

The limitations are equally clear. Direct sound does not automatically make patterns interpretable. A dense spike train may become a continuous texture. Two channels with different physiological behavior may collapse into the same perceptual range if their levels, timbres, or spatial positions are not separated. A small but meaningful change can disappear beneath a louder background channel.

The ear also applies its own compression. Repeated events group into rhythms. Close frequencies fuse. Loud transients mask weaker ones. A sonification can therefore lose information even when the digital mapping is formally one-to-one. The auditory system is part of the processing chain.

Parameter mapping: turning neural variables into acoustic controls

Parameter Mapping Sonification, or PMSon, separates the recorded variable from the acoustic variable. Instead of playing voltage as voltage, the system maps a neural parameter to pitch, volume, timbre, duration, or spatial position.

The source parameter may be:

• spike rate;
• inter-spike interval;
• local field potential amplitude;
• power within a frequency band;
• electrode location;
• channel correlation;
• event density over a defined time window;
• the activity of a selected neural population.

The acoustic target may be:

• pitch;
• loudness;
• filter cutoff;
• harmonic density;
• attack and decay;
• stereo or multichannel panning;
• spatial position in an exhibition system.

The mapping is not merely a technical bridge. It defines the audience’s model of the data. If spike rate controls pitch, a rise in pitch means more spikes only because the system has assigned that meaning. If the same rate controls volume, the listener receives a different interpretation. The underlying recording has not changed. The inferred relationship has.

A practical mapping starts by isolating one measurable variable. Do not map every available feature immediately. A system with spike rate controlling pitch, gamma power controlling brightness, channel location controlling panning, and event confidence controlling distortion may sound rich. It may also become impossible to debug.

Use one dominant axis first. Then add secondary parameters only when they have a clear perceptual function.

The key parameter is not always the most obvious one. Pitch is attractive because it is easy to hear and easy to musicalize. It is also a poor container for several simultaneous variables. If ten channels all control pitch, the listener must separate them by timbre, location, register, or time. Without that separation, the output becomes a dense chord with unstable meaning.

Scaling determines what survives

A linear mapping is easy to explain:

• minimum neural value maps to minimum acoustic value;
• maximum neural value maps to maximum acoustic value;
• intermediate values retain their relative position.

But neural data often has a skewed distribution. A few high-amplitude events can consume most of the available range. A logarithmic or percentile-based scale may make quieter changes audible. It also changes the relationship between data and sound.

This is where calibration matters. Define the input range from the recording segment or from a stable reference set. State whether the limits are fixed across sessions. If the range changes automatically, the same neural value may produce different sounds on different days. That may be useful for public engagement, where each performance needs audible movement. It is dangerous for comparison.

A robust workflow records at least four values for every mapping:

1. the raw input variable;

2. the preprocessing applied to it;

3. the scaling function;

4. the acoustic output range.

Without these, the sound cannot be reconstructed or compared.

Before and after musical mapping

The change from raw neural data sonification to musical mapping is not simply a change from noise to order. It is a change from continuous or event-based representation to a constrained symbolic system.

Musical mapping typically applies a scale, rhythm, key, or quantization rule. A continuous neural value is assigned to a discrete pitch. A timing value is snapped to a rhythmic grid. An event may trigger a note whose duration and envelope are selected for musical clarity rather than biological duration.

The result is easier to follow. It is also less precise.

Standard MIDI provides 128 discrete pitch values. Restrict the mapping to a standard major scale and only a subset of those values remains available; a mapping that uses 50 scale degrees, for example, cannot express every intermediate pitch. The neural input may change continuously while the output remains on the same note until it crosses the next threshold.

This produces a step function:

• neural value changes;
• the current threshold is crossed;
• the output jumps to a new pitch;
• all smaller changes remain inaudible as pitch changes.

The threshold may improve consonance. It may also hide a gradual rise, a short reversal, or a fine temporal relationship between two channels.

The same problem appears in rhythm. A spike occurring slightly before or after a grid point may be assigned to the same beat. A burst with uneven internal timing may become a regular pattern. The output communicates that activity occurred in a rhythmic region, not the exact structure of the original burst.

The aesthetic benefit is real

Musical mapping creates a stable listening surface. Pitches sit inside familiar relationships. Repeated patterns can form motifs. Multiple channels can occupy separate registers or timbral families. An audience can remain with the work long enough to compare sections and notice changes.

This matters in public engagement. A direct recording may communicate biological irregularity but fail to sustain attention. A mapped composition can support an exhibition, performance, or educational installation because the audience can orient itself in the sound.

The gain is not scientific precision. It is access.

Musical mapping also allows the designer to control the pace of perception. Neural events can be slowed, expanded, or grouped into phrases. A frequency band can modulate a synthesizer rather than generate an isolated tone. Spatial panning can preserve the relationship between recording location and sound position. These choices make a complex dataset navigable.

But the output should be labelled as a transformation. A consonant melody generated from neural activity is not the brain “singing” in a literal sense. It is a designed auditory representation whose structure contains both the recording and the mapping rules.

The cost is quantization

Quantization creates three predictable losses.

Micro-temporal loss. Small differences in event timing disappear when events are placed on a rhythmic grid or merged into a window.

Pitch-resolution loss. Continuous changes collapse into discrete notes. Two neural states may produce the same pitch even though their values differ.

Relational loss. The listener may hear a pleasing interval between two mapped channels and infer a meaningful neural relationship where the interval was imposed by the chosen scale.

That last point requires strict discipline. Musical consonance is an aesthetic property. It is not evidence of neural synchrony, connectivity, or functional coupling.

A good system can expose this compromise rather than conceal it. One layer can use quantized pitch for the musical surface. Another can retain raw event timing through subtle amplitude modulation, granular texture, or spatial movement. The audience hears a coherent composition while the less obvious layer carries information that would otherwise be discarded.

Brain-wave music needs a channel architecture

When several neural variables are sonified at once, the problem shifts from mapping to system design. The output must give each variable a stable perceptual address.

EEG bands provide a useful example. Delta, theta, alpha, low beta, high beta, and gamma can be mapped to different acoustic dimensions instead of competing for pitch. One band may control low-frequency filter movement. Another may alter timbre. A third may control spatial panning. Volume can represent a selected band’s power, but it should not carry every other feature at the same time.

Earlier multichannel neural sonification systems used as many as 25 frequency-band splits. That number illustrates the engineering limit: more channels do not automatically create more understanding. They create more separation work.

A practical channel architecture assigns each variable a distinct role:

• Pitch carries one primary scalar, such as normalized spike rate.
• Timbre differentiates recording groups or neural regions.
• Panning preserves spatial distribution.
• Amplitude indicates relative energy within a controlled range.
• Rhythm retains event timing or burst structure.
• Texture represents uncertainty, missing data, or aggregation.

Do not use panning as a decorative effect. If electrode location matters, spatial movement should correspond to that location. If location does not matter, panning can become a distraction that gives the listener a false sense of anatomy.

Stereo is also a hard constraint. A channel placed slightly left of center may be indistinguishable from a neighboring channel in a noisy room. A multichannel gallery system can provide more spatial resolution, but it introduces calibration issues: speaker placement, listener position, room reflections, and level matching all affect the result.

The physical installation is part of the measurement chain. A mapping that works on headphones may collapse in a gallery. A low-frequency component that is distinct in a studio may become structural vibration in a public space. Test the output at the intended listening distance, not only in the production environment.

If a listener cannot state which variable changed, the system has added complexity without adding information.

Preserve topography without forcing harmony

Spatial neural data creates a common design conflict. The work needs to maintain the relationship between recording sites, but a direct spatial mapping may produce a harsh or unstable sound field. The tempting fix is to quantize every channel to a common musical scale.

That improves harmonic coherence. It can also flatten topography.

A better compromise is to keep spatial location in panning or speaker position while assigning frequency bands to timbre and using a constrained pitch range for only one global variable. The listener can then detect where activity is concentrated without interpreting every local variation as a separate melodic note.

For zebrafish neurobiology, optical recordings may produce activity distributed across many cells or regions. The visual dataset can be dense even when the biological question is narrow. Sonification should therefore follow the analysis unit. If the question concerns regional activation, aggregate by region before mapping. If it concerns event propagation, preserve the order and latency of events. Do not send every pixel or voxel directly into the sound engine simply because the data is available.

From laboratory signal to public artwork

Neural data sonification for public engagement has two separate objectives:

1. make a pattern perceptible;

2. make the transformation understandable.

The first objective drives musical decisions. The second requires documentation and interface design.

An exhibition can include the source variable, the mapping range, and the transformations applied. It can show raw and mapped outputs side by side. It can allow the audience to switch between direct audification and quantized rendering. This makes the aesthetic change part of the work rather than an invisible processing step.

The comparison is especially valuable when presenting brain wave music. Visitors often assume that a melodic output is a direct recording. A simple toggle can demonstrate what changed:

• raw event timing versus grid-aligned rhythm;
• continuous pitch versus scale-constrained pitch;
• full amplitude range versus normalized amplitude;
• spatial channel placement versus fixed stereo;
• unfiltered data versus band-limited data.

The work then teaches a method, not only an impression.

A production sequence that holds up

A reliable workflow can be built in stages.

1. Define the biological question.

Decide whether the system must preserve event timing, compare activity levels, expose spatial organization, or create an accessible public representation. One output cannot optimize all four.

2. Select the analysis unit.

Use spikes, field potentials, EEG bands, regions, or event windows. Do not mix units without a defined aggregation rule.

3. Build an audit signal.

Generate the least transformed audible version that remains technically usable. Save it separately from the artistic rendering.

4. Calibrate the input range.

Quantify the minimum, maximum, baseline, and outliers. Record whether scaling is linear, logarithmic, normalized, or percentile-based.

5. Assign one dominant acoustic parameter.

Start with pitch, density, amplitude, timbre, or space. Add a second parameter only after the first can be interpreted reliably.

6. Add musical constraints deliberately.

Choose a scale or rhythm only if it serves the audience or the installation. Mark the point at which continuous data becomes discrete.

7. Run a blind interpretability test.

Ask listeners to identify which variable changed. Do not ask whether the sound is beautiful. That is a different evaluation.

8. Compare outputs at matched levels.

A louder mapped version will appear more informative even when it only has more energy. Match loudness before judging the transformation.

9. Document the mapping beside the work.

Include the source data type, preprocessing, time scaling, parameter ranges, and quantization rules.

10. Retain the source and intermediate files.

A public composition may outlive the software used to create it. Preserve enough information to reconstruct the pipeline.

This sequence also supports collaboration between neuroscientists, sound artists, and exhibition designers. Each role can work on a defined layer. The scientist controls signal integrity. The sound designer controls the listening surface. The exhibition team controls orientation, access, and physical delivery.

How to evaluate whether the mapping works

A sonification should be evaluated as an instrument, not only as a composition.

Start with signal fidelity. Can the system preserve the event order? Does normalization suppress meaningful amplitude differences? Does filtering remove a relevant band? Does time compression create overlaps that did not exist in the recording?

Then evaluate perceptual fidelity. Can listeners distinguish a sustained increase from a short burst? Can they detect a change in one channel when other channels are active? Does the room mask low-level events? Does musical quantization make two different neural conditions sound identical?

Finally, evaluate interpretive fidelity. Does the audience understand what the sound encodes? Does a rising pitch reliably indicate the assigned variable, or do listeners read it as emotional intensity, urgency, or narrative climax? Public work does not need to eliminate all metaphor. It does need to prevent accidental claims.

Useful tests are small and concrete:

• present two recordings with one controlled neural parameter changed;
• ask listeners whether they hear a difference;
• repeat with raw and quantized versions;
• compare headphones and installation speakers;
• test with and without explanatory labels;
• inspect whether outliers dominate the output;
• calculate how many distinct input states collapse to the same musical note.

That final measure is simple but revealing. If a large portion of the neural range maps to only a few notes, the work is not showing the full signal. It is showing a categorical summary. That may be the correct choice. It must be named as such.

The hard boundary between analysis and interpretation

Neural data sonification can expose patterns that are difficult to inspect visually. Repetition, density, timing, and spatial movement become available through a different sensory channel. This can support exploratory analysis and public communication.

It cannot remove the need for statistical and biological validation.

A listener may notice a recurring rhythm. That observation can guide further analysis. It does not establish that the rhythm is a stable neural feature. A pleasant harmonic relationship may result from the scale constraint. It does not demonstrate coupling between regions. A dramatic increase in loudness may reflect normalization, gain, or an outlier rather than a physiological change.

The safest practice is to maintain a strict separation between:

• recorded property — what the instrument captured;
• derived property — what preprocessing calculated;
• mapped property — what the sound engine assigned;
• perceived property — what the listener reports;
• biological interpretation — what the evidence supports.

These are different layers. Collapse them and the artwork starts making claims the data cannot carry.

The history of neural sonification already shows this progression. Parameter-mapping methods were formalized in the early 1990s. Neural recordings entered live sonification and exhibition contexts by the early 2000s. Research and practical tutorials have since moved toward Python workflows, digital audio workstations, and hybrid systems that combine data fidelity with musical structure. The tools have become easier to access. The methodological boundary has not changed.

Final troubleshooting parameters

Before releasing a neural sonification, isolate the system in this order:

1. Signal: Can you identify the source variable and its units?

2. Timing: Which temporal relationships survive time scaling, windowing, and rhythmic quantization?

3. Range: What happens to values near the baseline, and what happens to outliers?

4. Resolution: How many distinct neural states produce the same note, level, or timbre?

5. Separation: Can the listener isolate channels by pitch, texture, rhythm, or space?

6. Calibration: Does the same input produce the same output across sessions and playback systems?

7. Disclosure: Can a listener see where musical rules replaced continuous data?

8. Claim: Does the interpretation describe the recording, or only the designed sound?

The strongest work does not hide the distance between signal and composition. It maps that distance precisely.

Raw neural data sonification gives access to irregular timing and unprocessed structure. Musical mapping gives the audience a stable surface: scales, rhythms, timbres, and spatial cues. The first protects fidelity. The second protects attention. A defensible system keeps both, labels the transition, and never lets consonance masquerade as evidence.