JMP Workflow Challenge 1: Motif Extraction and Identification from Continuous Power Data

Daniel_Valente

Calling JMP workflow and JSL wizards. Here's a Thursday challenge for you. The attached data set is set of power data off my running power meter. What we want to do is identify and correctly tag the 8 motifs that occur during this ~1000 point signal data set attached to this post, create a new column with function ID and write it in a way that it can be used to automate the tagging of future data that follows this same envelope. 

 

The data collection assumptions for this data set and the future are:

• Prior to the start of a motif the power is 0.
• There is a period before and after the motifs that will be non-zero (warm-up and warm-down). This is not part of the system under test and should not be tagged.
• Each motif has an attack, sustain, decay and release. The shape of the function is identical from motif to motif. It goes from 0 to some peak over a period of samples and then ramps down to the baseline noise floor (walking power). The motif ID is over when the power reads 0 again.
• There will be an unknown number of functions to identify in the future. There are 8 in the data set. The solution needs to be invariant to that.
• The overall length of the functions as well as the peak can vary.

We are working on the left edge of the analytic workflow today (shown in green below). The data comes from a file (.csv), and eventually when we automated this workflow a folder of n .csv files. The Data Access is being used, and what the task is today is to perform the Data Blending and Cleaning tasks on this example file to have the data ready to expand the workflow to other analytic capabilities in the future.

Leave your solutions in the comments.

 

 

 

ExamplePowerCurveData.jmp

brady_brady

It is surprising how far you can get with a single formula column:

 

dt << new column("Motif ID", nominal, formula(Col Cumulative Sum(
	Row() > 6 & Summation( i = 1, 6, Abs( Lag( :power, i ) ) ) == 0 & :power != 0
)));

However, there is still some cleanup to do... this approach creates a "0" group ID, which is the leading data, and does not cut off the last motif where we'd like. The "6" in the summation is because I designate a new group if at least 6 consecutive zeros are encountered prior to a power change.

 

There are of course many other ways to do this that are more sophisticated, involving comparisons to the background floors, attacks/decays, etc... but if a simple approach like this one works, great.

 

The script below incorporates this idea, but cleans up the dataset, as well. Ideally the values in the last few lines would be parameterized.

 

names default to here(1);

dt = current data table();

dtSub = dt << subset(selected rows(0), selected columns(0));

//create ID column
dtSub << new column("Motif ID", nominal, formula(Col Cumulative Sum(
	Row() > 6 & Summation( i = 1, 6, Abs( Lag( :power, i ) ) ) == 0 & :power != 0
)));
dtSub:MotifID << delete formula;

//the next 2 loops truncate the dataset, assuming it ends like the sample dataset ended.
for(i  = nrow(dtSub), abs(dtSub:power[i] - dtSub:power[i-1]) < 20, i--,
	dtSub << Delete Rows(i)
);

for(i  = nrow(dtSub), abs(dtSub:power[i] - dtSub:power[i-1]) > 5, i--,
	dtSub << Delete Rows(i)
);

//delete ID 0 group
dtSub << select where(:Motif ID == 0) << delete rows;

//delete 0 power rows, but not when they lie within a group
i = nrow(dtSub)-1;
while (i > 1,
	if(dtSub:Motif ID[i] < dtSub:MotifID[i+1] & dtSub:power[i] == 0,
		while( dtSub:power[i] == 0,
			dtSub << delete rows(i--);
		)
	);
	i--
);