我以不同的方式使用音频制作了几个项目： MIDI 文件,语音API ，调用 Windows DLL，以及读取/写入 WAV 文件。
这次我将再次使用 WAV 文件。 JMP 的 BlobToMatrix 和 MatrixToBlob 函数可对 blob（二进制大型对象）进行操作。 LoadTextFile 和 SaveTextFile 知道如何读取和写入 blob。 WAV 文件是一个二进制对象（在文本编辑器中打开 WAV 文件不会带来太多乐趣）。 JSL 矩阵中二进制字节的内存表示形式为 8 个字节长，这意味着 1GB（1.5 小时 16 位 44.1KHz 立体声）WAV 文件需要 8GB 内存。 对于我需要的 60 秒文件来说，这不是问题。同样的想法可以用于任何中等大小的二进制文件，而不仅仅是 WAV，但正如您将看到的，解码二进制文件很复杂。
在之前的一个邮政我从网络日志中收集了数据；我正在使用来自的 IP 到位置文件最大思维将 IP 地址转换为位置和地名。 通过跨时间切片数据（一个月长的日志文件），我可以制作视频。 这是显示一帧的图形生成器。 我会制作 1800（30FPS * 60 秒）。
Color indicates five lat-lon clustersColor indicates five lat-lon clustersColor indicates five lat-lon clustersColor indicates five lat-lon clusters颜色表示五个经纬度簇
JMP 的聚类平台之一使用纬度和经度将点分为五个聚类；每个集群中的活动将驱动音乐的某些方面与视频配合。 音乐生成算法使用状态机是由这个想法。 和弦过渡由其中一个颜色组驱动，打击乐部分由另一个颜色组、吉他和低音吉他驱动。 可能我没有使用全部五种颜色；当我听到一些听起来不错的东西时，我就停下来了。 随机不是一个好声音。
乐器声音（合唱团、吉他、打击乐器）是从我制作的 WAV 文件样本中加载的LMMS 。 经过几次错误的启动后，我决定将样本加载到 JMP 中的最简单方法是将一种乐器的一组完整音符放入一个文件中，均匀间隔，音符之间有一些死区时间。
Really low guitar notes at the start of the WAV fileReally low guitar notes at the start of the WAV fileReally low guitar notes at the start of the WAV fileReally low guitar notes at the start of the WAV fileWAV 文件开头的吉他音符非常低
合唱团相同，打击乐也相似。 保存这些文件，然后将它们加载到 JMP 中，如下所示：
// RIFF loader
RiffUtil = New Namespace(
"RiffUtil"
);
这将是一个大型项目，因此从命名空间开始，以帮助将细节与项目的其他部分隔离。 RIFF 原来是 WAV 文件背后的文件格式的名称。 该名称空间中将有几个实用函数。 接下来，添加一个加载 WAV 的函数。
RiffUtil:loadWav = Function( {filename}, {Default Local},
riff = RiffUtil:load( filename );
{fmtOffset, fmtLength} = riff:chunks["fmt "];
riff:offset = fmtOffset;
// http://soundfile.sapp.org/doc/WaveFormat/
//JMP offset is 1-based, these are 0-based
//20 2 AudioFormat PCM = 1 (i.e. Linear quantization)
// Values other than 1 indicate some
// form of compression.
q = RiffUtil:getShort( riff, 1 );
riff:AudioFormat = q[1];
If( riff:AudioFormat != 1, Throw( "not PCM audio" ) );
//22 2 NumChannels Mono = 1, Stereo = 2, etc.
q = RiffUtil:getShort( riff, 1 );
riff:NumChannels = q[1];
If( riff:NumChannels != 2, Throw( "not 2-channel stereo" ) );
//24 4 SampleRate 8000, 44100, etc.
q = RiffUtil:getLong( riff, 1 );
riff:SampleRate = q[1];
If( riff:SampleRate != 44100, Throw( "not 44.1 kHz" ) );
//28 4 ByteRate == SampleRate * NumChannels * BitsPerSample/8
q = RiffUtil:getLong( riff, 1 );
riff:ByteRate = q[1];
If( riff:ByteRate != 44100 * 2 * 2, Throw( "not expected byte rate" ) );
//32 2 BlockAlign == NumChannels * BitsPerSample/8
// The number of bytes for one sample including
// all channels. I wonder what happens when
// this number isn't an integer?
q = RiffUtil:getShort( riff, 1 );
riff:BlockAlign = q[1];
If( riff:BlockAlign != 2 * 2, Throw( "not expected block align" ) );
//34 2 BitsPerSample 8 bits = 8, 16 bits = 16, etc.
q = RiffUtil:getShort( riff, 1 );
riff:BitsPerSample = q[1];
If( riff:BitsPerSample != 16, Throw( "not expected bits per sample" ) );
Show( riff:AudioFormat, riff:NumChannels, riff:SampleRate, riff:ByteRate, riff:BlockAlign, riff:BitsPerSample );
{dataOffset, dataLength} = riff:chunks["data"];
riff:offset = dataOffset;
nShorts = dataLength / 2;
result = Shape( RiffUtil:getShort( riff, nShorts ), nShorts / 2, 2 );
riff << delete;// the namespace
result; // return integers or... /32768;// return normalized +/- 1.0
);
大多数评论取自参考站点，并描述了如何解析 WAV 文件。 此代码处理一种 WAV 格式，如果格式错误则抛出异常。 （我只需要 16 位 44.1KHz 立体声。）当然，这个函数在执行其他操作之前会调用另一个加载函数。
RiffUtil:load = Function( {filename}, {Default Local},
riff = New Namespace();
riff:data = Blob To Matrix( Load Text File( filename, BLOB ), "uint", 1, "big" );
riff:offset = 1;
If( RiffUtil:getString( riff ) != "RIFF", Throw( filename || " is not a valid wav (RIFF missing)" ) );
q = RiffUtil:getLong( riff, 1 );
riff:length = q[1];
If( RiffUtil:getString( riff ) != "WAVE", Throw( filename || " is not a valid wav (WAVE missing)" ) );
// walk the chunks and record their offsets
riff:chunks = Associative Array();
While( riff:offset < riff:length - 8,
chunkName = RiffUtil:getString( riff );
q = RiffUtil:GetLong( riff, 1 );
chunkLength = q[1];
riff:chunks[chunkName] = Eval List( {riff:offset, chunkLength} );
riff:offset += chunkLength;
);
riff;//return
);
该函数完成繁重工作的第一部分，将 blob 加载到内存中，验证 RIFF 标签，然后验证 WAVE 标签，然后查看 WAVE 数据内部以分割出指定的子块。 然后，早期的函数会对 fmt 和数据块执行更繁重的工作。
RiffUtil:getString = Function( {riff},
{Default Local}, // return a 4-character id
four = riff:data[riff:offset + 0 :: riff:offset + 3];
riff:offset += 4;
Blob To Char( Matrix To Blob( four, "uint", 1, "big" ) );
);
getString 检索 4 个字节作为 JSL 字符串；其他函数获取 1-2 或 4 字节数字。 riff:offset 是一个变量，用于跟踪 blob 中的当前位置以供下一个要处理的字节使用。 请注意上面的“四”是如何分配四个元素的矩阵，然后转换为 blob，然后转换为字符串。 这是在没有循环的情况下完成的并且运行速度很快。 这是数字版本。
// these two transforms assist in the two's complement conversion; if the MSB is on
// ("negative") then also subtract maximum value
RiffUtil:transform2 = ((0 :: 127) * (256 ^ 1)) || ((128 :: 255) * (256 ^ 1) - (256 ^ 2));
RiffUtil:getShort = Function( {riff, n},
{Default Local},
chooser = riff:offset + ((0 :: n - 1) * 2);
little = riff:data[chooser];
biggest = riff:data[chooser + 1];
riff:offset += 2 * n;
little + RiffUtil:transform2[biggest + 1];
);
RiffUtil:transform4 = ((0 :: 127) * (256 ^ 3)) || ((128 :: 255) * (256 ^ 3) - (256 ^ 4));
RiffUtil:getLong = Function( {riff, n},
{Default Local},
chooser = riff:offset + ((0 :: n - 1) * 4);
little = riff:data[chooser];
big = riff:data[chooser + 1];
bigger = riff:data[chooser + 2];
biggest = riff:data[chooser + 3];
riff:offset += 4 * n;
little + big * (256) + bigger * (256 ^ 2) + RiffUtil:transform4[biggest + 1];
);
二进制补码是另一个主题。 稍后我们需要一个 wavWriter；这里是。 它对于字符串和数字具有完全相反的功能。
// wav writer...44.1 kHz 16 bit stereo
RiffUtil:WavWriter = Function( {filename, mat},
{Default Local},
uints = N Rows( mat ) * N Cols( mat );
wav = New Namespace();
wav:mat = J( 44 + 2 * uints, 1, 0 ); // pre-allocate buffer
wav:offset = 1; // 1-based
// http://soundfile.sapp.org/doc/WaveFormat/
//JMP offset is 1-based, these are 0-based
//0 4 ChunkID Contains the letters "RIFF" in ASCII form
// (0x52494646 big-endian form).
RiffUtil:putString( wav, "RIFF" );
//4 4 ChunkSize 36 + SubChunk2Size, or more precisely:
// 4 + (8 + SubChunk1Size) + (8 + SubChunk2Size)
// This is the size of the rest of the chunk
// following this number. This is the size of the
// entire file in bytes minus 8 bytes for the
// two fields not included in this count:
// ChunkID and ChunkSize.
RiffUtil:putLong( wav, J( 1, 1, 36 + 2 * uints ) );
//8 4 Format Contains the letters "WAVE"
// (0x57415645 big-endian form).
RiffUtil:putString( wav, "WAVE" );
//
//The "WAVE" format consists of two subchunks: "fmt " and "data":
//The "fmt " subchunk describes the sound data's format:
//
//12 4 Subchunk1ID Contains the letters "fmt "
// (0x666d7420 big-endian form).
RiffUtil:putString( wav, "fmt " );
//16 4 Subchunk1Size 16 for PCM. This is the size of the
// rest of the Subchunk which follows this number.
RiffUtil:putLong( wav, [16] );
//20 2 AudioFormat PCM = 1 (i.e. Linear quantization)
// Values other than 1 indicate some
// form of compression.
RiffUtil:putShort( wav, [1] );
//22 2 NumChannels Mono = 1, Stereo = 2, etc.
RiffUtil:putShort( wav, [2] );
//24 4 SampleRate 8000, 44100, etc.
RiffUtil:putLong( wav, [44100] );
//28 4 ByteRate == SampleRate * NumChannels * BitsPerSample/8
RiffUtil:putLong( wav, [176400] );
//32 2 BlockAlign == NumChannels * BitsPerSample/8
// The number of bytes for one sample including
// all channels. I wonder what happens when
// this number isn't an integer?
RiffUtil:putShort( wav, [4] );
//34 2 BitsPerSample 8 bits = 8, 16 bits = 16, etc.
RiffUtil:putShort( wav, [16] );
// 2 ExtraParamSize if PCM, then doesn't exist
// X ExtraParams space for extra parameters
//
//The "data" subchunk contains the size of the data and the actual sound:
//
//36 4 Subchunk2ID Contains the letters "data"
// (0x64617461 big-endian form).
RiffUtil:putString( wav, "data" );
//40 4 Subchunk2Size == NumSamples * NumChannels * BitsPerSample/8
// This is the number of bytes in the data.
// You can also think of this as the size
// of the read of the subchunk following this
// number.
RiffUtil:putLong( wav, J( 1, 1, 2 * uints ) );
//44 * Data The actual sound data.
RiffUtil:putShort( wav, mat );
Open( Save Text File( filename, Matrix To Blob( wav:mat, "uint", 1, "big" ) ) );
wav << delete;
);
RiffUtil:putString = Function( {wav, txt},
{Default Local},
mat = Blob To Matrix( Char To Blob( txt ), "uint", 1, "big" );
len = N Rows( mat );
wav:mat[wav:offset + 0 :: wav:offset + len - 1] = mat;
wav:offset += len;
0;//return
);
// should longs in wav files ever go negative? what about > 2GB data blocks?
RiffUtil:putLong = Function( {wav, mat},
{Default Local},
//show(mat);
len = N Rows( mat ) * N Cols( mat );
biggest = Floor( mat / 256 ^ 3 );
mat -= biggest * 256 ^ 3;
bigger = Floor( mat / 256 ^ 2 );
mat -= bigger * 256 ^ 2;
big = Floor( mat / 256 );
mat -= big * 256;
little = mat;
wav:mat[wav:offset + 0 :: wav:offset + 4 * len - 1] = Shape( little || big || bigger || biggest, 1, 4 * len );
wav:offset += 4 * len;
);
RiffUtil:putShort = Function( {wav, mat},
{Default Local}, // mat has two stereo columns of -32k...+32k ints
mat[Loc( mat < 0 )] += 256 ^ 2; // two's complement: -1==65535 ... -32768 == 32768
len = N Rows( mat ) * N Cols( mat ); // len unsigned (or two's complement) 16-bit values (L+R)
mat = Shape( mat, len, 1 ); // now mat is a list of 16 bit samples, one per row
big = Floor( mat / 256 );
mat -= big * 256; // high bytes
little = mat; // low bytes
//show(nrows(mat),ncols(mat),wav:offset,len,nrows(wav:mat),ncols(wav:mat),nrows(little),ncols(little));
wav:mat[wav:offset + 0 :: wav:offset + 2 * len - 1] = Shape( little || big, 1, 2 * len );
wav:offset += 2 * len;
);
您可能已经注意到 riff<<
现在是时候加载笔记了。 我将为笔记加载器 JSL 创建另一个名称空间以保持其隔离。
noteLoader = New Namespace(
"noteLoader"
);
noteLoader:load1 = Function( {path, samples, firstnote},
{Default Local},
Show( path );
result = New Namespace();
result:notes = Associative Array();
temp = RiffUtil:loadWav( path );
For( pos = 1, pos < N Rows( temp ) - samples, pos += samples,
//show(firstnote,pos,pos+samples-1,nrows(temp),ncols(temp));
result:notes[firstnote] = temp[pos :: pos + samples - 1, 0];
firstnote += 1;
);
result; // return
);
“load1”完成了我为这个项目确定的事情；随着项目的进展，“负载”逐渐消失。 load1 使用 wav 加载器获取音频数据矩阵，然后使用样本参数逐步遍历数据并为单个音符创建较短的音频数据矩阵的关联数组。 Firstnote 是（由我）指定给音符升序序列的数字。
还有一个作曲家名称空间；作曲家 JSL 知道如何向乐曲添加注释。
composer = New Namespace(
"composer"
);
composer:create = Function( {seconds},
{Default Local},
composer:mat = J( seconds * 44100, 2, 0 );
composer:choirPad4 = noteLoader:load1( path || "choir1/choirPad4.wav", 44100 * 8, 36);
composer:guitarVarA = noteLoader:load1( path || "Guitar1/varA.wav", 44100 * 2, 12 );
composer:guitarVarB = noteLoader:load1( path || "Guitar1/varB.wav", 44100 * 2, 12 );
composer:guitarVarC = noteLoader:load1( path || "Guitar1/varC.wav", 44100 * 2, 12 );
composer:guitarVarD = noteLoader:load1( path || "Guitar1/varD.wav", 44100 * 2, 12 );
composer:percus = noteLoader:load1( path || "Drum1/percus.wav", 44100 * 2, 1 );
);
composer:compose = Function( {instrument, note, position, volume}, {Default Local},
position = Round( 1 + position * 44100 );
duration = N Rows( instrument:notes[note] );
composer:mat[position :: (position + duration - 1), 0] += (instrument:notes[note]) * volume;
);
composer:new = Function( {}, {Default Local},
composer:mat[0, 0] = 0
);
composer:limit = Function( {}, {},
N Rows( composer:mat )
);
composer:save = Function( {filename}, {Default Local},
x = Max( Abs( composer:mat ) );
composer:mat = (20000 / x) * composer:mat; // normalize to +/- 20,000 out of +/- 32767
RiffUtil:wavWriter( filename, composer:mat );
);
作曲家还知道如何重新开始（新）和完成（保存）。
这是 YouTube 视频中的音频。 如果您获得 LMMS 的副本来制作 WAV 文件，则附加文件将大致复制它（由于随机选择）。 加载 LMMS 项目并导出。
