Hi All,I am looking to generate two new columns based on patterns in other two columns ( "Required column1" and "Required Column2"). Please see the attached image -
Any help/suggestion ?Thanks.
I gave a go with this. I try to avoid using Lag() whenever possible and replace it with Col functions. The most difficult formula seems to be Required Column 1 and I couldn't quickly figure out good formula without using Lag() (there is most likely a solution with Col Cumulative Sum somehow). One idea would be to fill places between Start and End with "Ongoing" or something similar, then you could maybe use Col Cumulative Sum easier, but then the most difficult formula would most likely just move to Start and End.
Names Default To Here(1); dt = Current Data Table(); //Numbering of Occurrence dt << New Column("NumberOfOcc", Numeric, Nominal, Formula( If(:Sensor == "Primary", Col Cumulative Sum(:Sensor == "Primary", :ID, :"Sub-set"n, :Occurrence), . ); )); //Start and End dt << New column("StartEnd", Character, Nominal, Formula( If(:NumberOfOcc == 1, "End" , Col Max(:NumberOfOcc, :ID,:"Sub-set"n, :Occurrence) == :NumberOfOcc, "Start" , //else "" ) )); //Required Column 1, this could most likely be made somehow with Col Cumulative Sum dt << New column("Req1", Numeric, Ordinal, Formula( If(Row() == 1, val = . ); If(:StartEnd == "Start", val = 1 , IsMissing(:StartEnd) | :StartEnd == "End", val += 1; ); If(:Lag(:StartEnd) == "End" | (Row() == Col Max(Row()) & :StartEnd == "Start") | (:StartEnd == "Start" & Lag(:StartEnd, -1) == "End"), //more missing checks can be added here val = ., val; ) )); //Required Column 2 dt << New column("Req2", Numeric, Nominal, Formula( If(!IsMissing(:Req1), Col Cumulative Sum(:StartEnd == "Start", :ID, :"Sub-set"n ) , . ) ));
If you have time, I would suggest testing both ways of solving this, the one @txnelson has provided with more heavy use of Lag and comparisons, and the one I have with Col formulas. Both definitely have their use cases and understanding them will help you in a long run when writing formulas/calculations in JMP. And in some cases when you understand how to combine both, you can solve quite difficult tasks with just formulas.
Also after you get the logic working, I would most likely remove Formulas from the datatable (or replace Formula() with << Set Each value in script), because this will most likely break if order is changed in datatable.
Here is a formula for the column you refer to Required Column 2. When you study the code, you will see that all it is doing is applying the logic rules you have stated that need to be checked when determining the Start and End values. It also contains the setting of the value for the column Required Column 1. To run this formula, simply create the Required Column 1 and Required Column 2. Set Required Column 2 to be a character column. Then open the formula for Required Column 2 and copy the formula below into the column.
RCValue = ""; :Required Column 1[Row()] = .; If( :Sensor != "Primary", other = "Yes" ); If( Row() == 1 | :ID != Lag( :ID ) | :"Sub-set"n != Lag( :"Sub-set"n ), counter = 0; withinStart = "No"; other = "Yes"; ); If( :Sensor == "Primary" & withinStart == "Yes", RCValue = "End"; withinStart = "No"; other = "No";, :Sensor == "Primary" & Lag( :Sensor, -1 ) != "Primary" & withinStart == "No" & other == "Yes" & Row() != N Rows( Current Data Table() ), withinStart = "Yes"; counter = counter + 1; RCValue = "Start"; ); If( withinStart == "Yes" | RCValue == "End", :Required Column 1[Row()] = counter ); RCValue;
Hi @txnelson, It almost worked with few false entries. I believe we must add extra condition on :Occurrence column. That is :Occurrence must change to set :Required Column 2 == "Start" (and may be option - it should be same when :Required Column 2 == "End").
Something like - Dif(:Occurrence) !=0 then only :Required Column 2 == "Start"For my sample data (95 rows) was fine with your formula but my actual data which has more than 300k rows, it is giving many wrong entries like below - (I think it should be fine once additional requirement on :Occurrence will be added.) This is just one off screenshot of the wrong entry.
Hopefully!! I can modify the formula/code and can add requirement of :Occurrence. on top of your formula.Thanks again for your support.
Hi @txnelson and All,I was able to modify your code based perfectly for my requirement. But on the 2nd stage troubleshooting I need something like this --Mark :Sensor = "Primary" (1st) around consecutive change in :Occurrence. In the given screenshot -
required values in between --- Last "Primary" in Occurrence = 2 and First "Primary" in Occurrence = 3 [consecutive change in Occurrence]Finding first Primary is not that difficult, not sure how to find last primary ? Thinking to somehow invert (descending order) the subset data for Occurrence = 2, find the value and then revert it back. Not able to put in a formula!! Any other way to do that ?Any help?Thanks.
You have modified the code I previously provided to get the results that you provide in your most recent response. Please provide the JSL you are using to get the results you are showing in your sample data table.