Karen Wang after graduating
Root Cause Analysis of Product Rejects in Manufacturing
After 4 years studying Engineering Science and 1 year in Mechanical Engineering, Karen Wang finished her Engineering Masters degree with a thesis on Root Cause Analysis of Product Defects in Manufacturing.
Karen wanted to study the causes of different quality problems and apply algorithms to discover and resolve the root causes of these issues – enhancing human analysis with automated data analysis.
AspectPT’s Bob Dedekind mentored Karen throughout her thesis. “Karen’s work could possibly be incorporated into our software to enable factory staff to identify potential problems before they occur. Karen was a pleasure to deal with – she learnt very quickly and was always ready to explore new challenges.”
Dr. Yuqian Lu (Karen’s university supervisor) connected her with Bob Dedekind at AspectPT. Bob facilitated collaboration with Callaghan Innovation and AspectPT clients Pharmapac, Hella and Nautech. These generous clients supported Karen with their expertise and data and in conjunction with the University of Auckland, enabled Karen to complete her thesis.
Dr Yuqian Lu added. “To the AspectPT team – a huge thanks for hosting and supporting Karen during her Master’s study. As her supervisor and University of Auckland staff member, I am very grateful for the endless support that we have received for supporting student development, education and research.”
What problem was Karen trying to solve?
Karen’s thesis is about identifying the root cause of different quality and production problems in manufacturing. Using an algorithm to discover the reason behind the product rejection and drilling down to potential causes like machinery failure, inferior raw materials or human error. Doing all this with data rather than an experienced production manager.
Karen used a Bayesian Network to graphically represent different models. A Bayesian Network looks like a network connecting nodes. It presents in a diagram how knowledge is acquired from the data and what is the relationship between the data.
Karen says “Using data and analytics to a solve an issue is really like being a detective. I chose this subject for my thesis because I could apply what I had learnt at university to a real world problem.
I started by interviewing local manufacturers about quality issues. I also looked for existing international research and technical solutions. Most of the research I found presented a cause only. It didn’t go as far as offering robust and human interpretable computational models and none offered any variables or probabilities – which is essential for data surety.”
Humans versus algorithms
Most manufacturers rely on their staff to identify root causes for production issues. They use their experience, collaboration with colleagues and data to find causes for issues – essentially human analysis. Manufacturers expressed this is a very difficult and time consuming manual process which is hard to automate and do repetitively. They added that an absence of an experienced production team member can also derail this human analysis.
Karen’s project was to leverage the data to make root cause analysis more robust, automated, intelligent and human interpretable – even for less experienced staff.
This meant emulating the experienced and knowledgeable team member who knows the machines, has experience with variables and can analyse data (likely from an MES) to work out the cause of rejects. Karen was trying to create an algorithm that deciphers problems that are creating inferior products.
Causes and probabilities as percentages
Through her work, Karen learnt that identifying a single root cause was too simple for manufacturers to feel confident about the findings. Instead of identifying just one root cause, she needed to present a list of causes and their probabilities as percentages.
For example, if the data suggested inferior raw material was likely the problem that the data set could be presented as 80% probability – alongside corresponding percentages for other causes. This made the data more understandable for manufacturers and they felt they could trust the data better because they could relate to the workings of the data.
One Bayesian Network became many
“Using one massive Bayesian Network didn’t produce any results – it just ran and ran without any results. This meant we had to run networks for individual products which made the data analysis run time much shorter. We could run several products in parallel and run specifics for products which enabled more outcomes.” said Karen
Data quality and accuracy
The quality and uncertainty of the data was also a surprise. For example, human errors such as reject calculation not accounting for a late machine start, presented variables which meant the data needed cleansing and manipulation before it was inputted into the Bayesian Network.
Findings – did it work?
Karen explains “Using a Bayesian Network did actually produce root causes and probabilities. Additionally, while building the Bayesian Network, we could compare learning algorithm performance to select the algorithms which identified the root cause the best.
Although this project is still in the exploring phase, we now know a Bayesian Network is totally capable of acquiring the knowledge from the data automatically and providing a human interpretable graph for people to use.
We could also use the same method to predict probabilities of a job having a quality issue to a high accuracy.”
Learnings and what is Karen doing now?
“This thesis enabled me to understand real world situations that I had never experienced in the rigid world of academic study.”
Karen adds “Going into a factory for the first time and seeing injection and blow moulding was amazing. I enjoyed talking to manufacturers about how they think and operate and solve problems as they happen.
I’m curious about how things are made and it was great to learn about how manufacturers make things we are using right now. Manufacturing is definitely a huge contributor to the economy and there are many opportunities in New Zealand right now to use more advanced technologies for better productivity.”
Currently, Karen is a Technology Consultant at Ernst and Young working in engineering data and analytics.
“I really enjoy working on the bigger picture and communicating with clients to find solutions to their problems.
I’d like to thank all the manufacturers and especially Bob Dedekind at AspectPT who were so supportive and happy to share knowledge. Your combined help enabled me to complete my thesis and gain my Masters degree.”
