Ideas for follow-up DOE for STMicro data?<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />Looking for some ideas for a plausible follow-up DOE for the STMicroelectronic data set. This would be in the theme of the "Data Mining Enabling More Productive DOE/Holistic DOE."
The data set is attached. Basically if you go through the holistic DOE approach of Explore/Screen/Experiment you find that there are a handful of important parametric tests that correlate to the response: logBin10 failure. The Bootstrap Forest with informative missing lets you quickly sift through the 606 PT params and find the ones that best characterize the presence of the dominate failure mode.
There is a also a split variable in the data set, which is a piece of equipment used for manufacturing these chips. And what you see is that the split is also related to the failure.
So from these data we have a set of PT params that related to the primary failure BIN, which is useful to engineers as the PTs are performed early in the fab process and can give early warning to chips, which will fail the electrical wafer sort. (And due to the data mining we just conducted, they only need to monitor the values of 4-5 of these params to have a good sense of the probable failures).
But it seems like there is more opportunity for learning here, and this data mining exercise should lead nicely into a more controlled experiment where the organization can better understand what is going on. If the split is causing issues in the manufacturing process, what else could be at work here?
It seems like the data mining is on the symptoms perhaps and not the root cause in the manufacturing process? But you wouldn't want to set up a follow-up DOE using the PT params as factors necessarily, or would you? Would that make sense to purposely manufacture chips that would be reading low or high on the PTs to elicit the failure in the EWS? If you purposesly vary these PTs systematically, you'd be able to verify cause and effect vs. just mining the production data.
Based on the Bootstrap Forest, here are the two dominate PT-EWS relationships, with 164 and 451:
Combined with the split, that we'd also like to test, we can set up a Custom Design. This would be the design runs for the two PTs and the split:
I assume there are many other factors that may be impacting the BIN10 failure that might be worth exploring, deposition recipe, raw material vendor, etch speed etc. The goal here would be to create a plausible follow-up design that would make sense in a data-driven DOE context. I appreciate the thoughts and help.
Re: Ideas for follow-up DOE for STMicro data?Thanks, Robert! I look forward to your case as well. That will be a nice complement to the ST case we already have. Scott and I had our first Holistic DOE event in Denver last week (Oct 2.) and I presented this ST case with the follow-up DOE that is described above. It seemed to go over well (in fact someone in the audience before setting up the DOE said: "I bet it is the etch step!").
The seemed to be OK with the fact that Spilt is 4 different machines. I focused on just the Split 3, which had the most interesting and varied response (Splits 1 and 2 had low Bin10 failures, Spilt 4 had all high failures, which I explained might be a defective device that should be investigated).
It's not perfect, but it seemed plausible to the audience. I still want to refine the case a bit more as I get more feedback. We will see how it goes over for the rest of the events we have this month.
Re: Ideas for follow-up DOE for STMicro data?
I have been looking at my semi demo again and there are some problems with it which I am currently trying to fix. The main problem is that you could get to the same answer by using standard JMP functionality rather than Pro. Using standard JMP is not as straightforward as using Pro but even with Pro I needed a script to cycle through different validation columns to achieve a reliable outcome. The key thing that my semi demo illustrates is the importance of using validation if you want to suppress false positives. If I manage to overcome the problems with the semi demo, I’ll make it available.
In the ST white paper I am not sure what splits S1, S2, S3 and S4 are. I don’t think they are different machines. They sound more like an ‘experiment’ in which they have deliberately (or accidentally) varied a process setting or parameter.
I also notice in the ST white paper that they choose parameter 171 as being the most meaningful parameter to use rather than 451. I suspect the reason for this is that they have a better understanding of what parameter 171 is and how that would link to a process setting or device performance characteristic e.g. 171 might be an electrical line width measurement that they can directly relate to a physical dimension that they measure in-line at a particular step. Whereas, 451 might be a leakage current that has a less obvious relationship to that physical dimension.
In terms of a follow up experiment, a dry etch process being the problem would resonate well with most semiconductor audiences. Dry etch processes are often problematic but also lend themselves well to investigation by DoE. They have many factors that can be varied and often the process engineer will not have a good quantitative understanding of how these factors interact with each other and affect the process.
A good set of factors could be: RF power, Bias power, Chamber pressure, Chuck temperature. Helium BSC (backside cooling). I know of two different yield issues that were caused by Chuck temperature and He BSC issues due to calibration differences between etch chambers. I would also suggest running the experiment on both a 'good' and 'bad' chamber.
Yield issues in semiconductors often exhibit a spatial pattern on the wafers as shown in the example below and like the ‘smile’ signature mentioned in the ST white paper. Those patterns can be important clues for the engineers in figuring out what has gone wrong.
Re: Ideas for follow-up DOE for STMicro data?This is a big improvement and I'm sure someone with a deeper background in the semi industry can guide you in the factors and ranges to use in proposed DSD for etch process.
If part of target audience for this demo is existing JMP users then I'd encourage you to think about how you demonstrate the proposed workflow in Pro is simpler and results in a potentially better solution than JMP. For example if you use Fit Y by X to get a bivariate analysis of log(bin10) vs each X and then output the parameter estimates to a combined data table, remove the intercept terms from the combined table then sort by p-value ascending we get the top six parameters being 26, 451, 183, 2, 47, 164. So it seems it wouldn't be too hard to reach the same concluson with JMP with a potentially easier workflow because we don't need to worry about the number of trees in the forest. Maybe we need to modify the ST data to induce some strong interaction effects that would be missed by Fit Y by X approach but would be captured easily by Bootstrap Forest? If part of target audience is existing JMP users, I think we need to be prepared to show they can't recreate solution easily in JMP.
Alternatively, you could use the semi demo Robert created when Pro was first released that showed how Pro detected the cause of a problem which was due to a three-way intection effect that was missed by the above Fit Y by X approach. You could then base the followup experiment around the three adjacent process steps that were interacting to cause the problem. I'll ask Robert to forward you his demo.
Malcolm
Re: Ideas for follow-up DOE for STMicro data?Hi Malcolm -- Thanks for the response. Re: the data table, I will give Ian credit for that. He did a great job putting together the scripts and also a short video for a marketing campaign. I am merely building on that in the holistic DOE context.
To your specific points:
1) Random seed -- By 1,000 trees things have settled down and the results are repeatable -- 26, 451, 164 and 171 are the top 4. These top 4 stay now even when running the Bootstrap Forest again and again. So that is a good change that should be made.
2) 164 is a tricky one because it is going in the wrong direction and seems to have a different behavior vs. 26, 451 and 171. Another nice addition that I will add is the Variable clustering, which shows that 71% of the variation in the data is explained by the 451 param avg.
So I think those two issues are sorted. So the conclusions from the data mining exercise are thus:
1) We used a Bootstrap Forest and PCA to find a key association between the most representative parametric test result (451) and Bin10 Failure.
2) We also found that the split variable indicates that the failure may be due to a piece of equipment (Split 3 and Split 4), which are causing the parametric test to be high in the range causing the bin10 failure.
3) Armed with these facts we can go back to the engineers with the parametric test average as the response (451_Param_average) and design a DOE to investigate the etch step on the split 3 equipment.
As far as simulating the data, I know approximately what range this param should be in and we can sort of figure out what the Bin10 failure rate would be for the 451 level.
So I just need a plausible set of inputs and levels for the follow-up design.
Perhaps a 5-factor DSD:
With simulated data to look something like this when fit:
So the story here could be anything about the machine -- maybe faulty or wearing parts/sensors that are causing intermittent immersion speed or average feed velocities (not present on the other splits, which are operating in spec), This could in turn cause the high param average causing the prosperity for a high bin10 failure.
Not married to this story but it could work if people smarter than me in this area think it could be plausible. Again, appreciate thoughts and comments.
Re: Ideas for follow-up DOE for STMicro data?Hi Dan,
The data table for the historic data analysis is nicely documented with the scripts explaining the process and measurement types and gives a great story line, one that we SE's could replicate and deliver easily within semiconductor and other industries. We get the usual problem with the random seed used for the bootstrap forest giving different solutions, e.g. on the second try I got 164_PARAM_AVERAGE as the top predictor of log(Bin10) and this is not included in the rest of the demo flow. We need to fix this to give consistent results. We could set the random seed in the script for bootstrap forest but I like to avoid that if possible because we can no longer show the dialogue for bootstrap forest. Is it possible to try a larger number of trees in forest to see if that fixes the problem or even a boosted tree?
As you suggest, the parametric tests are symptoms rather than causes of the problem and I wonder if a more realistic next DOE would be to focus on the process step(s) responsible for each of the four parametric test failures? Robert could guide you in this, but I expect a more credible DOE might be on one of the etch steps with the storyline that it is the etch step that is responsible for the feature that is causing the high parametric test value and the high parametric test value is causing the bin10 failure.
Malcolm
