756147_en-US

liliconcom

Factory Yield Ramp-up Approach Through Process Performance Metrics Guided Improvement Activities (2022-US-30MP-1119)

This paper describes an approach for driving yield improvements by analyzing process performance data with JMP. Analysis of performance data -- including process long-term and short-term capability, stability and statistical control -- is particularly useful when monitoring hundreds of process KPIs retrospectively. Modern manufacturing requires many process and metrology steps to ensure healthy product lines and high-quality products. During the production ramp-up phase, identifying the processes of most concern is highly challenging. Using JMP scripting and quality data analysis, Magic Leap’s Eyepiece Manufacturing factory implements an automated process that can pull, analyze, visualize, correlate, predict and verify factory yield improvement based on a variety of performance metrics.

Hello everyone.

My name is Harry Dong.

I'm the director of the Optical Process Engineering Group at Magic Leap.

I'm so excited to be here to present my topic:

Factory Yield Ramp- up Approach

through P rocess Performance Metrics Guided Improvement Activities.

Today I'm going to cover Factory Process Performance Overview,

the Process Screening Tool and the Statistical Process Control

using JMP Script to automate the process capability analysis and the conclusions.

People have been talking about process performance,

S uch as capability stability, but what are they?

Basically, process capability is a measure

of the ability of the process to meet the specifications.

While, the process stability refers to the consistency of KPIs over time.

It's very important to realize there is no inherent relationship

between process stability and process capability.

Thus it is both extremely important aspects of any manufacturing process.

As you can see from the bottom- left picture.

It's showing,

process can be both capable and stable meantime,

this is a perfect world.

Basically your process is super tight

against your lower spec limit,

upper spec limit.

But on the other side.

When you started seeing instability,

when your variation is small,

you'll still be able to meet the process specifications.

But over time,

you will see a lot of variations.

The other scenarios can be your process is super stable.

Over time you don't see a lot of up and downs,

But because it could be like a small

a three Sigma, a lower spec limit, upper spec limit,

you have large variation,

then you'll not be able to meet your specification

against a high yield target, high process capability target.

The worst case can be your process is not capable,

which means you have large variation

and over time you also see a lot of stability issues.

It's everywhere,

Across the whole industry,

monitoring factory process using the process performance plot

becomes very useful.

It's including capability, stability

in the combined metrics.

As you can see,

this is a JMP- generated process performance plot.

On the x- axis you see the stability index

and the y- axis is indicating the capability overall which is a Ppk.

Eventually this is a four quadrant plot.

We want to push everything low stability

and high process capability which is in the green zone.

Often you'll see some process not capable which is under this line

or not stable, which is across the vertical line showing in this graph.

I'm going to talk about

the Process Screening function,

JMP provided.

This is a very powerful tool,

if you are talking about quickly identify unstable process

or some incapable process, meanwhile.

Basically this is a JMP- generated report,

as you can see,

I have 24 processes listed in this report.

Very quickly you can see stability index is showing up.

How do you define the process stability?

It's calculated using the Within Sigma and Overall Sigma,

This is the ratio using the Overall Sigma

divided by Within Sigma,

What do they mean?

Overall Sigma usually treated as a longterm process variation

and the Within Sigma treated as a shortened process variation.

So JMP has certain rules that you can refer to

for this calculation.

But basically you can define color codes.

How do you see your process stable?

You can use 1, 1.3, whatever the number you want,

to color code that.

For me, as you can see,

I color coded process greater than 1.7

as a process not being stable.

Yellow as a process is kind of marginal,

And the green zone meaning the process is super stable.

This is telling us some process can be stable but not capable,

Because you can see the Ppk Cpk are kind of low,

But on the other side, you can see a bunch of process,

they were not stable, but some of them are very capable,

Again, this is aligned with what I went through earlier.

Basically, utilizing the Process Screening tool,

you can quickly identify the unstable processes

as I marked in this graph.

Meanwhile, as you can see in this Process Screening tool,

you can also see the control chart alarms

based on the samples that the raw data you put in the report.

We finally deploy the control chart,

called statistical process control chart,

be able to monitor and improve process capability.

Stability is very important,

This is a JMP- generated control chart,

You can use Windows Scheduler or JMP Live

to automate those charts to be able to pull the data real time.

Meanwhile, you can also use certain web API

to generate the control chart.

JMP is very useful in this case.

It can send out email notification to the group,

to individual engineers.

Meanwhile,

if you can connect us with your internal system ,

I think that's going to give you additional power

to be able to communicate with your process tool,

be able to pause the tool,

put it on hold for the engineers to react to the variations

either process shift or out of control data points.

Process Capability Analysis is a standard,

Basically a few rules that we need to follow,

First of all, you want to make sure

your data set is following the normal distribution.

This can be done using JMP tool,

I won't go through.

But on the other side we realize Cpk calculation changes dramatically

due to the outliers, especially when sample size is small.

The outliers can be driven by special causes,

excursions during the process.

So this can add bias to Process Capability Analysis.

Sometimes if you do see some outliers,

you can drag your Cpk down,

but it's not representative to your standard process variation.

Apply outlier removal method to remove outliers.

We'll help you get rid of those noises

to better understand your true process capability.

Within the JMP,

they offer different methodology

to remove outliers.

I won't go through each of them, but they're all very powerful.

You can read through the instructions which method is the best in your case.

Basically, they can be found under Analyze/S creening /E xplore Outliers.

Basically, I'm going to show the basic method we are using

to exclude the extreme values for our process,

which is a quantile range outlier removal.

Basically this tool we found is very useful,

Because when you try to pull the data through the database,

often you'll find some outliers.

Some data are very extreme,

you know they're outliers, some are not as obvious,

So basically this quantile range outlier method

offer you flexibility.

Basically, as you can see, this is the distribution of our process.

We have upper spec limit, target and you also have the mean.

And follow this Box Plot,

you can find the 10th percentile value,

or the 90th percentile value.

Basically the inter-percentile range

is calculated using 90th percentile minus 10th percentile value

and this is defined as your inter-percentile range.

The lower threshold value is calculated,

using the 10th percentile value

minus three times the calculated inter-percentile range.

So this becomes your low threshold for each individual processes.

On the other side the high threshold

is defined by 90th percentile plus three times inter-percentile range.

This becomes your high threshold.

Basically all the extreme value outside of this range

will be treated as outliers and they can be colored.

They can be marked as missing .

They can be excluded from your data analysis.

As you can see, JMP did provide the flexibility.

How do you define the inter-percentile range?

You can do 0.1, you can do other values as needed.

And also the Q value,

which is this value I highlighted here,

can be changed as well.

It really depending on how much noise

you want to get rid of from your data analysis.

A s you can see,

This is an example that showing some value is being colored

and also being changed to missing value,

so they will be excluded from the data analysis.

This is a quick demonstration to show you how the outlier

is going to impact your Cpk calculation.

As you can see, Cpk value remains equal or better

post outlier removal.

It really depends on your sample size.

Sometimes if your sample size is small, the change can be more dramatic.

But in my case, I believe my sample size is quite large.

This is why they're not showing very big differences.

Yeah, the quantile range o utliers

parameters can be tuned if necessary as I mentioned earlier.

Other things I want to highlight in this page,

it's the process capability box plot.

We figure out this is very useful tool

because you can be monitoring many process parameters.

So be able to put them together

to visualize how tight they are, which direction they're shifting

and how much variation is being counted

to calculate our process capability.

Process stability is very useful,

as you can see.

They use a standardize s pec limit to be able to combine everything together.

It's super useful for data visualization.

I want to quickly show you,

before and after we automate the process performance analysis.

At the beginning team are not using

the JMP scripting to automate this process,

as you can see.

We have to collect the quality data by individual process owners per module,

so data, sometimes, becomes not standard.

They use different formats

and then they often do manual outlier removal

and then they have to grab all the data together

to be able to merge them.

It's also a manual process, very tedious.

After that they have to do the manual process performance

because the variation they see through different people,

different format and then generate report.

They have to eventually repeat everything they did

on like weekly basis, monthly basis or per PEQ build.

But on the other side for the automated process flow,

basically the quality data can be queried all in one step.

All the raw data, all the specification data

can be put from the database

using the SQL JMP scripting,

Of course we apply standard outlier removal methodology across all the data

and then all the sorting, split, spec assignment,

all the different visualization

can be done automatically.

Standard report will be generated.

Then, when we are talking about over time

or per PEQ summary ,

you can simply modify your SQL query filter to update the data

and on top of that, you have basically all the raw data.

You can apply local data filter.

You can add additional functions to make your filter data analysis easy.

Basically, we figure out manual process

is very time consuming,

the feedback is slow and it's not very efficient regarding

the yield improvement.

On the other side,

we figure out the automated scripting process using JMP.

Anybody can perform this complicated process performance analysis in minutes.

Some highlights I want to share for the process capability analysis.

As you can see,

I didn't include the JMP query portion.

But I basically put different process name,

different test label,

all the raw data into the JMP table

and after that we figure out to split this d ata table

because table comes in everything combined

process, part ID, process time, test label.

So we have to eventually split the data table

to be able to perform this process capability analysis,

the box plots or the Cpk Ppk analysis.

So very useful function for JMP is after we split, group them.

So we have some missing value because we sort the data by data time.

Some process leave blank but JMP is smart enough

not counting those missing value

into the capability analysis

which is very useful,

So I do want to mention that.

Please don't forget to sort your data by date time

because that's super important

because sometimes the Cpk Ppk calculation

is really depending on the process sequencing.

If you are not sorting the data by date time,

then the results can be biased,

So we made some mistake earlier, figure out this is useful tip.

The other one is automated outlier removal

using quantile range outlier.

As you can see, b asically, very simple process.

You get all the column names

using this scripting.

Condition can be numeric, continuous

and then you start at launching

the quantile range outlier platform

basically assign them into the report and then go through the report.

Use this for loop function

to be able to exclude all the outliers identified,

This can be repeated as needed .

Then you can also launch the process capability analysis.

If you don't know how to do it, you can basically manually do it

and then grab the code from the log,

This is a new function JMP provided which we figure out is super helpful.

Something I want to highlight here is the spec limit assignment,

This is a super powerful, very useful.

Basically you can assign specification for multiple process variables,

using another data table

that's generated using t he SQL query as well.

Conclusions.

Analyzing process performance data

using JMP is super critical to drive the yield improvement

in modern factories,

especially with many process and methodology stacks

to ensure healthy production lines and deliver high quality products.

Analysis of performance data including long term, short term process capability,

stability and statistical process control

is particularly useful when monitoring

hundreds of process KPIs.

During the production ramp up phase.

Identifying the processes of most concern is highly challenging

and using JMP scripting and quality data analysis platforms ,

our Eyepieces factory implemented an automatic processor

which can pull, analyze, visualize, correlate, predict and verify

factory yield improvement based on a variety of performance matrix.

Magic Leap's Eyepiece Factory,

we demonstrated greater than 90% RTY, which includes hundreds of process KPIs.

Eventually, you have to multiply them together

to get this RTY number, no throughput yield.

We were able to demonstrate greater than 90%e RTY

six months ahead of our next generation product launch,

driven by continuous process improvement activities,

guided by automated process performance analysis,

using JMP scripting, and quality platform tools.

And thank you for your time.

Advanced Statistical ModelingAutomation and ScriptingBasic Data Analysis and ModelingConsumer and Market ResearchData Blending and CleanupData Exploration and VisualizationDesign of ExperimentsMass CustomizationPredictive Modeling and Machine LearningQuality and Process EngineeringReliability Analysis