nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Image analysis of an elephant's foot in JMP

JohnPonte

This is a picture of the bottom of an elephant's foot. As you might guess from looking at this picture, this is not a very happy elephant. Elephants kept in captivity often spend their time walking on pavement or other hard surfaces. This is not the substrate they are used to, and it is not what an elephant's foot was designed for. Consequently, this elephant has developed sores on his foot, and in some cases, those sores have even become infected. The caretaker for this elephant is providing treatments, but the elephant continues to walk on his foot, causing further damage. So the caretaker is left wondering if the treatments are helping at all.

I was given a copy of this photo with this question about image analysis: Is there a way to quantifiably measure the condition of the foot so that over time it can be determined if the treatment is having a positive effect?

Looking at the image, I can see that all the areas of the foot basically fall into one of three categories:

There are the areas that are brownish in color, which represent the normal color of the elephant's foot. Those areas are healthy.

There are areas that are some shade of orange. Those are the areas that have developed sores.

Then there are the whitish areas within the sores. Those are the areas where the sores have developed into an infection and are now full of puss.

By categorizing the entire foot into those three buckets, I felt we could get a measure of the condition of the foot. It is important to ignore the background, those areas in the image that do not show the bottom of the foot, so as not to skew our results.

I read the image into JSL (JMP Scripting Language) and got the pixel values from the image as red, green and blue (RGB) values as separate matrices. I could then convert these individual matrices into one matrix, representing all three color components.

img = newImage("elephantFoot.jpg");

{r, g, b} = img << getPixels("rgb");

colr = RGBColor(r, g, b);

But oftentimes, it is better to analyze color in a color scheme other than RGB. So I really want the colors in a color scheme called HLS, which stands for hue, lightness and saturation. The hue is basically the tint of the color, the lightness is the brightness, and the saturation is how much color there is as opposed to how much white. Fortunately, JMP provides a method for converting from RGB to HLS.

{hue, lit, sat} = ColorToHLS(colr);

Now that I have my color in HLS, I can create a JMP data table of my pixel values.

dt = New Table("Pixel Data",

newColumn("Hue", set values(hue)),

newColumn("Lit", set values(lit)),

newColumn("Sat", set values(sat))

);

The reason to do this is so that I can use some of the statistical power of JMP to analyze my data. I’m curious about the breakdown of my color values, so I decided to create a Distribution of each of the color components.

Distribution( Continuous Distribution( Column( :hue ), Outlier Box Plot( 0 ) ) );

Distribution( Continuous Distribution( Column( :lit ), Outlier Box Plot( 0 ) ) );

Distribution( Continuous Distribution( Column( :sat ), Outlier Box Plot( 0 ) ) );

Here is the result.

By looking at my Distributions, I can take a good guess as to what values to look for in my data to identify what pixels are brown, orange or white. I process each pixel, represented by a row of data in my data table, and I am able to categorize each one. I then can report my results.

To generate my report I selected all my missing values (all the pixels that represented the background area) and set them to hide / exclude. I then generated a summary table from my pixel data table. I can then use Graph Builder to generate my graph.

The result is that about half the foot is in good condition. For the half that is not normal, about 30% has developed sores, and the other 20% is infected. We now have a quantifiable measure of the condition of the foot. By doing this over time, we can determine the condition of the foot and know whether or not the treatments are helping.

The image of the elephant’s foot came to me through an organization called WildTrack. WildTrack is dedicated to the preservation of endangered species as well as the overall well-being of animals in the wild.

WildTrack is a JMP customer who will be featured in an upcoming webcast, Analytically Speaking. If you found this blog post interesting, I encourage you to register and attend the webcast. I can assure you that their talks are always fascinating and informative. And as always, please let me know what you think.

Comments

WP_Comment

Scott Moore wrote:

Very cool. A couple of thoughts. The eye is pretty good at analyzing patterns. I would have thought the brown was closer to 35 percent looking at the picture. It would have been interesting if you had made a hypothesis first before your analysis, then compared. I would also like to see some sort of error calculation. Very good process though and a great way to solve a problem.

WP_Comment

John Ponte wrote:

I was a bit surprised when the answer came out to be about 50% normal too. But then I realized the bad areas (orange and white) appear as larger areas where as the good areas (brown) appear as smaller sections, longer and wrapping around the bad areas. I think when you see lots of smaller areas and less larger areas, the perception is that the larger areas will add up to be larger.

An error calculation is a good idea. This was my quick (but usable) solution for the customer. Along with this solution I gave them a number of ideas of how they might improve upon it. I didn't want to take away all their fun.

Thanks for the comments!

WP_Comment

Sky & Zoe wrote:

This is a great example of how the analytical power of JMP could be used to improve decision-making in the treatment of pathology. So much medical imaging is still reliant on expert visual assessment rather than quantification.

This application, once refined, could have great application in standardising assessments of the way wound-healing is progressing (or otherwise) over time. John has a real talent for thinking out-of-the-box, and we believe the work he is doing for JMP imaging will have huge application in a whole range of areas.

WP_Comment

amit bothra wrote:

Hi John,

great post. I am amazed with the capabilites of JMP. As an expert, how would you compare JMP's image processing capabilites with some dedicated softwares like Matlab.

WP_Comment

John Ponte wrote:

Well, first let me dispel the myth that I am an expert! Although, if you see my boss I encourage you to tell him that you think I am. And thank you for saying so.

Secondly, while I am familiar with MATLAB, I do not feel I know enough about it to make a valid comparison between the two products. I have spoken to a number of JMP users who also use MATLAB and they seem happy with it.To those users (teaser alert), make sure you check out some of the new functionality in version 11 of JMP.

Besides the statistical capabilities, one of the features I really like in JMP is JMP's scripting language, JSL. This post shows how to bring an image into JMP and convert it to a data table using JSL. Once you have the image as data you have all the statistical power of JMP available to you. Now you can analyze your image by using Discriminant Analysis or Clustering, for example. I'm currently reading a book on this topic. It is called Measuring Shape, by F. Brent Neal and John C. Russ. So I think the combination of these features makes JMP a powerful tool for image processing. It allows the user to create their own solutions without having to actually implement the functionality.

WP_Comment

Scott wrote:

Great work John! I'm surprised that JMP can be used for such diverse applications.

I'm curious how you removed the background of the image. How did you identify background pixels when the background is non-uniform?

JohnPonte

John Ponte wrote:

In a word - manually! I receive lots of images from people asking how they can identify an area of interest within the image. Unfortunately I can not spend the amount of time I would like to, or need to, to develop a complete solution. So I try to give them a quick solution along with ideas for how they could further develop it. The quick solution was to simply black out the background using another software package (I used Gimp). But in JMP/JSL you could develop different techniques for edge detection that might identify the edge of the elephant's foot for you. In fact, I've demonstrated an example of this in some work I did for WildTrack, where they identify an animal's footprint in an image. You can find a video about this here: http://www.youtube.com/watch?v=VjjcdCDXx9M

WP_Comment

Rodrigo wrote:

Dear John,

I followed your script and got the pixel values for H, L, S. Now, I need to process each pixel and recognize what is green, yellow and brown. How I do this? I am lost in this part: " By looking at my Distributions, I can take a good guess as to what values to look for in my data to identify what pixels are brown, orange or white. I process each pixel, represented by a row of data in my data table, and I am able to categorize each one. I then can report my results."

Any advice will be appreciated. Thanks!

WP_Comment

John Ponte wrote:

Rodrigo,

JMP works in a normalized color space, which means all the values are between 0.0 and 1.0. Looking at my distribution for hue, I see that most of my values are below 0.125. However, I also see some values above 0.9. So that gives me an idea of what values I should be looking for to separate my colors.

What you will need to do, in JSL, is loop through each row (which represents a pixel) looking at the value for hue and light. In your case, you will be looking for green, yellow and brown. Yellow and brown have a very similar value for hue, since they are both made up of equal amounts of red and green. They are distinguished by thier brightness, or in this case the value of light. Yellow will have a higher value for lightness than the brown. Green will have a higher hue value than the yellow and brown. So using the hue and the lightness values, you should be able to distinguish the colors.

Look at your distribution for light and see if you have three somewhat distinct groupings. The highest values are probably your yellow pixels, the lowest values are probably your brown pixels and the ones in between are probably your green pixels. Start with that and see if that gives you the right answer. You can produce a new image the same way I did with the elephant foot to validate your answer. If you dont' get the right answer, you can tweak your value ranges in your comparisons.

I hope this helps!

WP_Comment

John Ponte wrote:

By the way, JMP released a YouTube video of me describing the process of analyzing the elephant's foot. It contains more details about the process. If you found this posting interesting you might enjoy the video as well.

It can be found here: https://www.youtube.com/watch?v=wSkUwgNbv2g

Enjoy!

WP_Comment

Sandeep wrote:

John, is there a tutorial video that you have made for me to learn this technique?

JohnPonte

John Ponte wrote:

Sandeep, the link posted in my previous response is a link to a video describing the technique (https://www.youtube.com/watch?v=wSkUwgNbv2g). You can also find some JSL code on the JMP File Exchange that will help you. Look for ImageToDatatable.jsl under Scripts (https://community.jmp.com/docs/DOC-6100). That will show you how to open an image and generate the data table from the pixel values. Alternatively, you can use the Image Analyzer to generate the data table for you (https://community.jmp.com/docs/DOC-7181).

BTW, if you are coming to Discovery(in Cary, NC in two weeks) please come talk to me and I can help you out. I will be at the Developer workstations as well as wandering around.