JMP Script Error with large table only: Maximum Length of Arrays?

sornasst

Hello JMP community,

 

Using the script below, which works great for tables with ~500 columns, I ran into a weird problem when using it on a large table with 20,000 columns: essentially, the script appears to complete normally, producing the output table as desired but the actual values are pretty much random (as verified manually).

 

Any clue on what may be happening? I have a hunch that this might be due to a memory allocation error due to exceeding the maximum length of the arrays I use to collect the data.

 

Note: I cannot share the large table I'm working on because of the confidential nature of the data

Thanks,

Names Default to Here (1);

dt = Current Data Table ();

m = {};
m1 = [];
m2 = [];
m3 = [];
m4 = [];

For (i = 12 , i <= N Col (dt),i++,

		mv = Multivariate(
				Y( :BIOMARKER, column(i) ),
				Estimation Method( "Row-wise" ),
				Scatterplot Matrix(
					Density Ellipses( 1 ),
					Shaded Ellipses( 0 ),
					Ellipse Color( 3 )
				),
				Spearman's ρ( 1 ),
				Kendall's τ( 1 )
			);
			
			mvr = mv << report;
			insert into (m, Column Name (i));
			m1 = m1 || ((mvr [Outline Box (1)] [Number Col Box (1)])<< get) [1];
			m2 = m2 || ((mvr [Outline Box (1)] [Number Col Box (2)])<< get) [1];
			m3 = m3 || ((mvr [Outline Box (1)] [Number Col Box (3)])<< get) [1];
			m4 = m4 || ((mvr [Outline Box (1)] [Number Col Box (4)])<< get) [1];
			mv << close window;
		);
New Table ("Non-Parametric Correlation",
	New column ("Symbol", character, set values (m)),
	New column ("Spearman Rho", numeric, set values (m1)),
	New column ("Spearman p", numeric, set values (m2)),
	New column ("Kendall Tau", numeric, set values (m3)),
	New column ("Kendal p", numeric, set values (m4)),
);

ms

Try this:

Names Default To Here(1);

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");
dt_out = New Table("Non-Parametric Correlation",
    Add Rows(N Col(dt)),
    New Column("Symbol", character),
    New Column("Spearman Rho", numeric),
    New Column("Spearman p", numeric),
    New Column("Kendall Tau", numeric),
    New Column("Kendal p", numeric),
    Invisible
);
r = 1;
For(i = 2, i <= N Col(dt), i++,
    m = [];
    mv = dt << Multivariate(
        Y(:Mass, Column(i)),
        Estimation Method("Row-wise"),
        Scatterplot Matrix(0),
        Spearman's ρ(1),
        Kendall's τ(1)
    );			
    mob = Report(mv)[Outline Box(1)];
    Column(dt_out, 1)[r] = Column(dt, i) << get name;
    For(c = 2, c <= 5, c++,
        Column(dt_out, c)[r] = (mob[Number Col Box(c - 1)] << get)[1];
    );
    mv << close window;
    r++;
);
dt_out << show window(1);