40883_en-US

liliconcom

Probit Analysis & Fit Testing

I am using the Simple Probit Analysis script add-in to determine LD50, LD90, and LD95.

 

Is this script capable of correcting with Abbott's, or how do you set this up?  Currently, I corrected the data myself by applying the Abbott's correction to the raw data to adjust for mortalities.

 

After running the Probit Add-In, how do I determine goodness of fit?  There are no Chi Squared results.  Is there another way I can run this analysis to get my LD values, as well as getting the Chi Square? 

Re: Probit Analysis & Fit Testing

Excluding the high responses is not acceptable from a modeling or statistics point of view. You are suggesting that you exclude valid data to overcome a basic flaw with the study. (I understand that you could not foresee the potency of this agent, but that does not remove the flaw in the data.) You need more data, between dose = 0 and dose = 1.64, not less data, to solve this problem.

Excluding the high dose samples might be acceptable in your situation but I could not justify it. The fact that you (possibly) obtain a 'better' model this way is not validation. You simply cannot determine the LD50 from this sample of dose-response data except to say that LD50 occurs between 0 and 1.64.

Re: Probit Analysis & Fit TestingSo is it acceptable at this point to exclude the high responses to get a "fitted" model, which I can use to estimate a predicted dose or range of doses to test as a follow-up? We were not expecting the high response at the very low doses, which is why I have no data for in between. Right now, I need to be able to determine somewhat of a recommendation for these agents, which can be followed up with a secondary study to confirm.Re: Probit Analysis & Fit Testing

I cannot speak to the discrepancies that you cite, but they call the data into question.

The problem with fitting is that the data exhibits the 'separation' problem. At one level (dose = 0), you have the low response and at all of the other levels (dose > 0), you have the high response. The transition from low to high response is impossible to determine. Try to find LD50 on your own with this data and you will understand the issue. (Does it occur at 0.1? At 0.5? At 1.5?)

You must perform another dose-response test where the dose ranges from 0 to 1.64 in order to fit the probit model and estimate LD50.

Re: Probit Analysis & Fit Testing(Sorry, I'm being pulled 1000 directions today. )

It is a different agent. I tested a total of 6 different agents. I'm being asked to give recommendations at 10 minutes if possible, and 15 minutes as a max. I have a couple of data sets similar to this, where there is very high mortality at doses above 1.64 or 10%.

Why does the model not work in these situations with high mortality at the higher doses?

I think most of it has to do with the data, as I had some strange effects on a couple of the agents. I've had the situation where a lower dose worked more effectively than a higher dose. The model doesn't work in that situation either. A couple at 100 were less effective than lower doses due to evaporation, but the models work fine if I exclude those doses.

In another set, my controls all experience 0 moralities (different species than previous control). Those seem to be causing discrepancies as well.Re: Probit Analysis & Fit Testing

You didn't answer any of my questions.

This example must be a different agent than the example that you tested at 15 minutes. It doesn't make sense that the same agent would be that much more lethal with just five more minutes of exposure.

You must use a different dilution/dose range (higher dilution, lower dose) to observe the transition. The highest dose should be 1.64. You should include the zero in the test and in the data analysis. This new data is the only way to fit the probit model for this agent at this time point.

Re: Probit Analysis & Fit Testing

Here is one of the data sets in particular I'm having issues with. If I Hide/Exclude the 0s, the model will fit.

 

Re: Probit Analysis & Fit Testing

• Is it a data problem?
  • Do you have outliers or data errors?
  • Did you accidentally exclude rows that should be included?
• Is it a model problem?
  • Are you using the correct response columns?
  • Are you using the correct predictor column?
  • Is it lack of fit?
• Is it a model assumption problem?
  • Are the errors distributed as expected?
  • Are the errors constant over the entire range of the predictor?

Re: Probit Analysis & Fit Testing

What do I do when some of the models have a bad fit? 

Re: Probit Analysis & Fit Testing

Thank you!  I was tripped up when you said to not include mortality rate, so I was double checking if I was supposed to be using death or live counts as the response.  Hopefully I won't have anymore major issues!

 

Truly appreciate the help!

Re: Probit Analysis & Fit Testing

Please see my reply with a complete analysis that ends with the question, "So we agree now, no?" It shows you how to set up your data in a JMP data table. It also shows you how to assign the data columns to the analysis roles in the Fit Model launch dialog and change the other settings for a proper Probit analysis. I did not apply any transformation to the dose levels - it is unnecessary.

Re: Probit Analysis & Fit Testing

Do I use # of deaths as my response then?  My "Start" is how many live counts I have before treatment is applied. The 15 min column is deaths.  Is this correct, or do I need to use live counts as my response?

 

Do I need to apply LOG to the Dose with the Transform drop down, or is this analysis fine without it? 

Re: Probit Analysis & Fit Testing

Yes, you include the control group with dose = 0.

You enter both counts for each dose, so you do not use mortality rate and you do not apply Abbott's correction.

You should perform a separate Probit analysis for each time interval. I performed the correct Probit analysis for the dose-response data collected after 15 minutes already (shown above - the reply with the correct analysis ends with my question for you, "So we agree now, no?"). Let me know if you have any questions.

Perform this analysis for the data collected after the other intervals. Let me know if you have trouble repeating this work.

You should not expect the estimates to be the same for each group (collection time). You should perform the analysis correctly and not compare the results to those from other variations that are not correct.

Re: Probit Analysis & Fit TestingAlso, do I need to include the Controls/0s? Which would then be 5 doses, 4 reps each.Re: Probit Analysis & Fit TestingNot a problem, it's a good learning experience for me.

I have 4 rows because I performed 4 replicates at each dose. I had 4 doses, 4 reps each. I did monitor each treatment at 1 min, 5 min, 10 min, and 15 min intervals.Re: Probit Analysis & Fit Testing

I apologize (though it is not my fault) for the replies getting out of order. It does not help the confusion that already exists.

I have one question before I think we can conclude this discussion: why do you have four observations (rows) for each dose? What does each row represent?

Again, forget Life Distribution. Your colleagues are correct, you need a probit analysis. I am working now to make sure that your data table is set up correctly and the analysis is properly defined in Fit Model for a probit analysis. I want to use your previous example of data after you explain why there are four rows for every dose:

Re: Probit Analysis & Fit TestingI figured it out. Probit Add-In needs live data and it calculates the dead counts itself. Generalize Linear takes the dead counts.

So if I run the GLM, do I include my controls/0 doses?Re: Probit Analysis & Fit Testing

We already determined that you do not have life time or time to event data. You have a case of dose-response data. You collected observations at 15 minutes, so time is fixed. You counted the number of deaths and the total number subjects at each dose. This case is handled by GLM as I showed you. Please disregard the example based on life time that used Life Distribution. Survival analysis methods do not apply here. You should use a Probit analysis as you first started.

Re: Probit Analysis & Fit Testing

 

Here is the comparison of the same data in the probit add in vs the gen. linear probit.  If I revert predicted log dose back, it does not match up with the same results in the general linear model. 

Re: Probit Analysis & Fit Testing

I'm not sure that is going to work.  I'm getting resistence from others around me on running the analysis that way.  They want the probit.  

Re: Probit Analysis & Fit Testing

Here's how I set it up and my data:

 

Re: Probit Analysis & Fit Testing

 

So I used my raw, unadjusted data without my controls and came up with this. 

 

Do I interpret this as:  LD90 for 100% Dose is 4.33 minutes?  How do I determine the fit of this since there is no Chi Square?  (I'm being asked to show a goodness of fit for the model I use.)

 

Do I count mortality as a "failure"?

Re: Probit Analysis & Fit Testing

OK, I think I understand now. I was correct to begin with and it is not a survival analysis (life time). We are back to my GLM example but now I will use your data. Here is your data in a JMP data table:

Set up the Fit Model launch dialog as before:

Click Run and then you get these results:

Click the red triangle next to Generalized Linear Model Fit and select Inverse Prediction. Type 0.5 (and any others that you want) for Probability (Died) and click OK.

So now we agree, no?

Re: Probit Analysis & Fit Testing

Your last comment about testing mortality rates over time changes everything, I think. My example is not relevant. My example is a case where a number of samples or subjects (20) is assigned to each factor level (5). Each level is an independent sample (5x20). The total number of samples or subjects is the sum of the samples at each level (100). I believe that you start with a fixed number of subjects (e.g. 60 total), the agent level is introduced at a fixed level, and you monitor the same group (subjects) over time. Yes?

Assuming I am correct, then you want to use repeated measures in survival analysis.

The methods and platforms in JMP were developed for reliability analysis, but it is the same thing. Just be warned that the words and labels are for reliability engineers and not biologists, if you know what I mean.

Set up your data to look like this example:

In this example, I have a total of 60 subjects that I monitor from the start of a 6 month program. I need two columns to represent the response (life) because I am not monitoring the subjects continuously but at monthly intervals. This process leads to interval censoring. I don't observe the exact life but instead I observe that it occurs during an interval.

• The first row represents the start of the test. In this simulation, 2 subjects died at the start.
• The second row represents the number of subjects who died between the start and the end of the first month (3).
• The third row represents the number of subjects who died in the second month (3), and so on.
• The last row represents the number of subjects who did not die by the end of the study (20).

I added another column, Status, to indicate the status of the observations but it is not necessary for the analysis. Perform the analyis by following these steps:

1. Select Analyze > Reliability and Survival > Life Distribution.
2. Select Beginning and Ending and click Y, Time to Event.
3. Select Count and click Freq.

The launch dialog should look like this:

(Note that the Censor analysis role is for right censored data. It is not needed when two response columns are used for interval censoring.)

Click OK.

The initial analysis is based on the non-parametric estimates of the failure probability from the Kaplan-Meier product limit. (You can click the red triangle at the top to change it from a Failure plot to a Survival plot.) You use the check boxes and radio buttons on the left side to scale the plot and fit a distribution model, respectively, to the data. Let's fit a normal distribution. Check the Normal check box and click the Normal radio button:

After selecting a model, use the Statistics outline to obtain the desired results. For example, your LD50 is available here as the quantile corresponding to a failure probability = 0.5, so estimate LD50 with the Quantile Profiler:

The LD50 in this example is 3.688803 months (3.22981 months to 4.1478 months with 95% confidence).

Re: Probit Analysis & Fit Testing

Here is an example of the data:

Dose (% Conc.)	Start (#live)	Mortality @ 15 min. (#dead)
100.00	13	13
100.00	10	10
100.00	11	11
100.00	13	13
25.00	10	9
25.00	23	17
25.00	15	9
25.00	9	6
10.00	14	3
10.00	10	1
10.00	10	4
10.00	16	7
1.64	11	1
1.64	12	3
1.64	13	4
1.64	24	4

 

I am trying to get LD50/LD90/LD95 results for dosage recommendations. I have this information for 5, 10, and 15 minute applications. 

 

This is the data for my control:

Dose (% Conc.)	Start (#live)	Mortality (#dead)
0	10	1
0	11	6
0	10	3
0	15	4

Re: Probit Analysis & Fit Testing

I am confused as much as you are.

You should have two counts for your response for each level of the predictor: one count is the number that died and the other count is the number that might have died. You should have one predictor. It is usually concentration or log concentration. Is that true? Or is it time? Are you trying to estimate the time for LD50 at a fixed concentration?

Please use the data format exhibited in the example that I shared and not rates or proportions. Counts are necessary for the inference. A rate of 0.1 might result from a case of 1/10 or a case of 100/1000. The rate is the same in both cases but the sample size is very different.

Once again, if you enter the data as I showed and set up the GLM as I showed, you can forget about Abbott's correction for rates. You should enter both counts for time=0.

Re: Probit Analysis & Fit Testing

So if I use the GLM analysis, do I include my 0s/Control Data and my initial raw data? And in doing so, is it correct to adjust a "0" to "0.005" for the log transformation?  I am testing for mortality rates in the presence of an agent at various time points. 

Re: Probit Analysis & Fit Testing

Abbott's correction is applied to mortality rates (i.e., proportions) to account for the natural mortality in the absence of the agent be tested. GLM is modeling a binomial distribution with a linear predictor. The GLM model has an intercept for that purpose (baseline mortality). So it is not necessary to apply this correction and doing so will, in fact, change the output. So don't do it if you are using GLM for your analysis.

Re: Probit Analysis & Fit Testing

I attached the wrong Linear models with raw data. Here are the correct:  Gen. Lin. Raw Data 1Gen. Lin. Raw Data 1Gen. Lin. Raw Data 1Gen. Lin. Raw Data 2Gen. Lin. Raw Data 2Gen. Lin. Raw Data 2

Re: Probit Analysis & Fit Testing

Abbott's Gen. Lin.pg 2Abbott's Gen. Lin.pg 2Abbott's Gen. Lin.pg 2Abbott's Gen. Lin. pg 1Abbott's Gen. Lin. pg 1Abbott's Gen. Lin. pg 1Abbott's ProbitAbbott's ProbitAbbott's ProbitRaw Data with 0s Gen. Lin. 1Raw Data with 0s Gen. Lin. 1Raw Data with 0s Gen. Lin. 1Raw Data with 0s Gen. Lin.2Raw Data with 0s Gen. Lin.2Raw Data with 0s Gen. Lin.2Probit Raw Data with 0s 1Probit Raw Data with 0s 1Probit Raw Data with 0s 1Probit Raw Data with 0s 2Probit Raw Data with 0s 2Probit Raw Data with 0s 2

 

Here are the two different results I'm getting with the different data sets.  If I use the 0s and my raw data, both the probit model and the general linear probit model seem to match up.  However, if I correct my data with Abbott's correction and leave out the 0s, then the two models don't match up. 

Re: Probit Analysis & Fit Testing

Okay,

 

It seems if I include my 0s (Control for Mortality) into the model, both the general linear probit model and the probit add-in match up the LD recommendations.  If I use Abbott corrected data, then they have discrepancy between the two.

 

Should I run the model with the 0s and the raw data (no Abbott's adjustment), or is it more correct to adjust the raw data with Abbott's prior to inputing into the probit and disclude the 0s?  

Re: Probit Analysis & Fit Testing

This example is from our training course, JMP Software: Analyzing Discrete Responses.

We are testing seals for failures at increasing pressure levels. Here is the data table with the preferred layout:

Note that we do not enter the failure rate (Failed / Tested) for the response.

Select Analyze > Fit Model. Select Pressure and click Add. Select Failed and click Y. Select Total and click Y. (Note that (1) you must enter a column with the count of targets and another column with the count of opportunities and (2) the must be entered exactly in that order.) Click Standard Least Squares and select Generalized Linear Model. Select Binomial for the Distribution model. Select Probit for the Link function. Your launch dialog should look like:

Click Run.

You can see the non-linear response and the regression diagnostics below:

Re: Probit Analysis & Fit Testing

I have not.

 

I have been attempting to run the model in the Generalized Linear with Probit, however, my LD values do not match up with what the Probit Add-In is giving me, so I'm not sure what is going on.  I've read through the Fitting Linear models, but what I am trying to run analysis on is nonlinear, which I honestly don't have any experience with in JMP.  

Re: Probit Analysis & Fit Testing

Did you contact the author of the add-in or read the description and instructions first?

JMP has several built-in platforms that can estimate such quantities and provide goodness of fit statistics. These leads should get you started.

• Fit Curve can fit common non-linear functions.
• Nonlinear is more general and it can fit custom models.
• Generalized Linear Models can perform a probit analysis.

Did you search probit yet in Help or Help > Books > Fitting Linear Models?