As mentioned in the previous JMPer cable, the final model that I created in my last post is nowhere near perfect and can only be applied to a very small subsection of the dataset. In this JMPer cable post I aim to highlight some of the work I did prior to the statistical approach to try and incorporate more of the data, in a more of an “engineering first approach”. This should highlight the importance of domain knowledge when constructing Structural Equation Models (SEMs); the importance of knowing the limitations of your model; and the importance of making appropriate design choices which help you design the specific model for your specific needs.
Preliminary Work
Going back all the way to the beginning of my work on the Tennessee Eastman Simulator (TES) the first step (as with any dataset) is to perform exploratory data analysis to determine any potentially interesting characteristics about our data that we may need to be aware of. When I did this one of the analyses that I performed was a factor analysis, mainly because in SEM knowing the number of latent factors required to properly model the system is vital.
As you can see from the scree plot of the full dataset, we need a crazy number of factors to properly model the TES, around 20 or more factors. Which of course is a lot of factors and would make our SEM extremely complicated. But in the fashion of an engineer, I wanted to make my model as robust and comprehensive as possible. So, considering the amount of data, the number of streams and number of process units, I initially attempted to create the SEM using latent factors such as “condition of stream x” or “condition of unit x”. Issues in chemical processes can come from a variety of factors but by basing the latent factors not only on some underlying root cause but also on a physical location in the system it was my hope that it would result in a far more insightful model. However, this was not the case and in fact attempting to create the model in this way lead to the resulting SEM being very confusing to understand while also offering little to no insight into the process conditions at these physical locations.
This was primarily because prior to undertaking this project I didn’t realise the capacity to which the model complexity increases as I try to make it more robust by adding more latent factors and widening the scope of the faults that the model can predict. Just take a look at my attempt at making a SEM based on the streams and units:
I don’t expect you to understand any of the above SEM, I’m including it to demonstrate why I abandoned this more “engineering first” approach in favour of a more simplified model which can still make reasonable predictions. Essentially, each of the manifest variables are the same as my final model but each factor is a stream or a unit. Bear in mind here that this SEM is only step 1, if this idea was taken forward further the complexity would have increased exponentially for relatively little gain. Especially considering a big issue related to autocorrelation that would turn the above model into an even bigger mess if applied.
Autocorrelation issues
As I mentioned above, model complexity rapidly becomes an issue not only in this example but in SEM in general. This is because SEM is a very visual method and so there can only be so many visual elements before the model starts to look very overwhelming. This is why after my initial research I decided to make the design decision to reduce the number of manifest variables that I included with my model as well as the decision not to include lag variables.
So, what are lag variables and why are they so much of an issue here? Well essentially, chemical processes are typically continuous which means that there is a constant flow of resources through the process. This also means that there will be a strong correlation between a data point's current value and its previous value. This makes sense since if I included a temperature increase fault then the temperature will steadily increase to its new equilibrium value rather than jumping straight to the new value. Changes take time to occur in a chemical process and the change observed is relative to the last value observed. How much the previous data point effects the next data point is known as autocorrelation and can be analysed in the JMP time series platform.
Ok that’s a lot of preamble on this idea so let me illustrate the problem simply. Below is the simplest form of autocorrelation possible, it uses one variable and has two lags.
This doesn’t look too bad, this shows us all the correlations between each lag variable, in the context of our SEM we would therefore have a manifest variable for each lag. In this example we only have 3 correlation coefficients to calculate. Let’s try adding one additional variable to the system and see how complicated things are looking:
Here you can see that we are starting to get a crazy number of correlations and that is just for two manifest variables. In this model with only 2 lags and 2 variables we need to calculate 15 correlations! This is a huge increase from the previous model, so you can imagine how many correlations would need to be calculated for a model with just 3 or 4 variables. In the TES there are 52 variables! Clearly attempting to do this manually would be ludicrous, if not impossible.
I did utilize the time series platform in JMP which was really helpful in identifying that there were autocorrelations in my data. Unfortunately, with the difficulty in applying these principles to such a large dataset I had to make the design decision to not include lag variables. Obviously, this will impact the quality of the final model, if you look at my previous post you can see that my final model is by no means perfect and this is one reason why. If you would like to learn more about autocorrelation check out Laura's post on a SEM use case, in which she explains the concept further.
Limitations of my SEM
It’s important to be realistic about our models and evaluate their capacity honestly. During the process of creating my SEM for the TES I had to make various design decisions which ultimately impacted the effectiveness of my final model. Primarily this was through two main choices, the first was the decision to not include lag variables in the model, which I’ve already discussed. The second was the choice to only attempt to identify one fault at a time. This second design choice came from the necessity to try and keep the model simple and easy to understand, but this decision ultimately meant a sacrifice in the applicability of the model. Essentially, I mean that if this SEM was implemented to a real chemical process to try and identify faults then it would fail miserably, since if any other fault occurred other than the expected fault then it would not be detected. Clearly this means that this method could not be used as a replacement for model driven multivariate control charts (MDMCC) in process control which are capable of detecting any deviations. I'd recommend the MDMCC post by Jeremy Ash to understand more the benefits of MDMCC.
Nevertheless, through my time working on this project I have found SEM to be an extremely interesting idea and I can definitely see scope for the improvement of the model I made here. Next month I’ll be suggesting and exploring some ideas that I had during my project which could lead to an improvement in the model. These improvements were left out of this first design since they would have required much more time be dedicated to the project and are potentially quite complicated solutions.
Even outside of direct improvements to this process control SEM, I can also see that there is scope for the idea of SEM to be explored in different areas of chemical engineering. Although the key problem that I had personally when working with SEMs in an engineering context is the definition of a latent variable. In engineering a lot of variables are measured, meaning that we already have the data and since the processes are well understood we already have empirical correlations between them. For this reason, I found it particularly hard to settle on a set of good latent factors, as you may have noticed from my other preliminary work. Therefore, I think identification of latent factors is the key to finding the perfect use of SEM in engineering and so I encourage you to explore this idea and let me know what latent variables have worked well for you!
