Torch Deep Learning Internal Discussion and FeedbackThis is the official internal chat board for the Torch Deep Learning Add-In for JMP Pro. The add-in provides a key component in JMP's roadmap for predictive modeling, machine learning, and artificial intelligence amidst all of the modern-day hype. Please post questions, comments, or problems any time.
Re: Torch Deep Learning Internal Discussion and FeedbackSounds great, @russ_wolfinger .
I will write an email to my customer and to you regarding the manual code modifications.
Thanks again, and I'll be in touch soon.
Re: Torch Deep Learning Internal Discussion and FeedbackHi @DaeYun_Kim Not currently, although in the add-in install file there is a path variable defined that specifies the web location from which all of the downloads take place. A customer could readily change this to be an on-premisis address and then mimic the same folder structure we have online for storing all of the zip files. Happy to work with you and customer to explain this in more detail and help modify the code.
Another option is to install the initial add-in and then have each user manually copy all of the dynamic library files from an internally shared location into the local add-in folder. We often do this internally for testing purposes and it works well.
Re: Torch Deep Learning Internal Discussion and Feedback@russ_wolfinger Can the Torch Deep Learning add-in be installed in a closed environment (without an internet connection or with a firewall)? If not, is there an alternative method to install this add-in in such an environment? Thank you in advance.
Re: Torch Deep Learning Internal Discussion and FeedbackHi All,
We've been getting some great feedback regarding Torch model deployment, which is a great sign that customers are seriously looking to use Torch models in their pipelines. The add-in currently only provides a couple of example Jupyter notebooks, which leave a lot to be desired in terms of actual deployment code.
We've made a substantial improvement recently with implementation of a brand new "Generate Python Code" option off of the model red triangle and are looking to ship this with 18.1 and 19 Early Adopter. The option creates Python code that invokes a new Python class called JMPTorch that will be included with future versions of the add-in. The predict method of this class processes all models saved in a Torch directory and creates an average ensemble prediction, taking care of all of the intermediate steps and embeddings. The attached zip file contains the current jmptorch.py file with this class along with example invoking code and 10 example images.
This code is all Python and can be run in JMP's Python or in external Python after installing the requisite packages. See commented instructions at the top of "Model 1.py" regarding which packages to install, including, of course, torch itself.
Would love to hear your feedback on this and any ideas you have for improvement. We just showed it to Lumentum yesterday and they loved it.
Notes:
- The JMPTorch class is still incomplete for text and more complex models but should work for basic image and tabular classification models.
- There is currently a conflict between the add-in's Torch dynamic libraries and similar ones installed in JMP's Python, so you can only run one or the other in any single JMP session. I've been successfully running things in two separate JMP instances.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @nick_shelton for checking this. The Save CAMs option is not working properly currently but we are planning to remedy that in an upcoming release. Fully agree this is a great way to visualize and interpret how the convolutional neural nets are working.
Re: Torch Deep Learning Internal Discussion and Feedback@russ_wolfinger I've been exploring the "Save CAMs" option. Given JMP's strength in visualizations I think this would be a great conversation starter around using JMP to explain ML models. The problem is I haven't been able to get the option to work for me. Every time I've built a model and used that option I get images that appear completely dark. Is there a step I'm missing?
Re: Torch Deep Learning Internal Discussion and FeedbackHi All,
Want to try a really fast NVIDIA GPU? Seeking volunteers! Please do not let this distract you from your sales duties but this is something you may be able to use for important sales opportunities or in your free time.
You may have heard SAS is very active currently with NVIDIA-specific GPU development via CUDA coding across many of their analytical divisions. They are acquiring a bunch of new NVIDIA gear ranging all the way up to the latest H100s that can cost north of $25K.
They are in need of some benchmarking and I've been in touch indicating how easy and fast it is to get JMP Pro up and running with the Torch add-in and pegging GPUs using all the new examples we have for image and text classification and regression.
They've just set up a new Windows 2022 server for us with four separate L40 GPUs, each with 48 GB Memory! The L40s are three generations more modern than the P100s on TDL.
Please message me directly if you would like to try this new server and I will send you login credentials and instructions. All we ask is that you send back a few run times of models you try (your choice or I can recommend some), which are printed in the JMP log after each Torch model you fit.
I've been trying several different examples from the Torch Storybook for a few initial cases and am seeing ~2-3x speedups over the P100s.
Love this stuff,
Russ
Re: Torch Deep Learning Internal Discussion and FeedbackThank you~!
Re: Torch Deep Learning Internal Discussion and FeedbackThank you Kim!
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @lkimhui. Just so everyone is clear, there's a difference between "disconnect" and "sign out". Disconnecting or simply closing the remote desktop connection application does not terminate the session, JMP and any other apps you've opened are still running on the server. To terminate the session and free up any used resources you need to "sign out". The "sign out" option is available from the windows menu, click on your login name and then select "sign out".
Re: Torch Deep Learning Internal Discussion and Feedback
Seconded this, please log off instances after use, idle users are still consuming the RAM.
Attached is the accumulation of 5 idle users and already consumed ~23% of the 128GB RAM.
Re: Torch Deep Learning Internal Discussion and FeedbackHi @Ben_Ingham , We're still working on streamlining the process, but for now there are a few examples of deploying the saved .pt files in Python notebooks. Click View > Add-Ins, select TorchDeepLearning, click on the Home Folder link, then look in the notebooks subfolder.
Each model is saved in 2-3 separate .pt files to enable output of intermediate embeddings. In Python the exact same architectures need to be specified and then the model weights are loaded from the .pt files. If you do K-fold, there are multiple models saved that can be averaged / ensembled for more stable and accurate predictions.
Re: Torch Deep Learning Internal Discussion and FeedbackHello All,
I'm curious to see suggestions on how a customer would deploy their model outside of JMP following model creation in the Torch add-in - as it sits 'outside' of the formulas you can dispense in the Formula Depot what would be the best process?
Thanks,
Ben
Re: Torch Deep Learning Internal Discussion and FeedbackHi all,
I received some requests to share the slide used to present for the Torch Add-in on-demand webinar.
Here you go!
Re: Torch Deep Learning Internal Discussion and FeedbackI did see it this morning @russ_wolfinger
Re: Torch Deep Learning Internal Discussion and FeedbackHi @Florian_Vogt I just updated the instructions with more details about Remote Desktop at the beginning. Please let me know if it doesn't work for you. I'm not familiar with how things work internationally and there may be additional steps required.
Re: Torch Deep Learning Internal Discussion and FeedbackHi Florian,
Search for a Microsoft remote desktop app from your laptop. If you don't see it you may need to install it from the managed software centre. When you start remote desktop app you will need to follow the instructions from Russ to establish a connection.
Malcolm
Re: Torch Deep Learning Internal Discussion and FeedbackThanks Russ, all is working now.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @Chris_Kirchberg I just updated the instructions with first step being a VPN. For the computer itself IT presumably removed all restrictions with regard to number of concurrent users yesterday afternoon. Have you seen this restriction today? If so I can ask them again.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @malcolm_moore1 I just reset the permissions on C:\Public so Everyone has Read/Write. I had set them before and somehow they changed back. Hopefully they will stick this time.
Re: Torch Deep Learning Internal Discussion and FeedbackHi Russ,
that's great new thanks.
What exactly do you mean by "remote desktop into"? How do I do that?
Thanks
Florian
Re: Torch Deep Learning Internal Discussion and FeedbackTwo things to add to these instructions:
- You must have your computer connected via VPN in order to access server, otherwise you will not be able to connect
- When you are done, please logoff the server, there are a limited number of connections. I see that two people are on when I connected a few seconds ago. One was listed as Away and the other was Idle for 1 hr and 30 min.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks Russ, do you mind checking access permissions to C:\Public as I get denied access when trying to install the addin.
Best,
Malcolm
Re: Torch Deep Learning Internal Discussion and FeedbackExcited to announce Windows Server tdl.na.sas.com has been created for anyone in JMP to remote desktop into and utilize, especially those of you who have slower local compute capabilities. It is nicely equipped with two NVIDIA P100 GPUs, each with 12 GB GPU memory and 128 GB CPU RAM. 2-4 of us should be able to simultaneously run Torch. Instructions:
1. Establish a VPN connection. I use Cisco Secure Client AnyConnect VPN, but not sure what everyone does internationally.
2. Launch Remote Desktop Application on Windows. I believe this is also available on Mac from Microsoft (please correct me if this doesn't work from a Mac or if there is a better alternative). For the Computer enter tdl.na.sas.com then enter your Windows credentials as prompted. This should launch you into a virtual session on the server.
3. After you remote desktop in, find and launch either JMP Pro 19 (most up to date, will be replaced by 18.0.1 when it is ready) or JMP Pro 18 (should work great but has a few issues fixed in 19).
2. Install your own copy of the Torch add-in by dragging C:\Public\TorchDeepLearning.jmpaddin onto JMP Pro. You can alternatively download the add-in from https://community.jmp.com/t5/JMP-Add-Ins/Torch-Deep-Learning-Add-In-for-JMP-Pro/ta-p/733478
3. Click Add-Ins > Torch Deep Learning > Torch Deep Learning and select NVIDIA-GPU.
Allow a few minutes for all files to download and unzip successfully then you are ready to go.
4. This is a shared resource and we may need to take turns at times. To check current usage, first open Task Manager, check CPU and Memory, and then click on the Users tab to see who is logged in. Then open a Command Prompt and enter "nvidia-smi" to see current GPU usage.
If at least one of the GPUs is at 0%, you are good to go. If both GPUs are being used, please come back later. If you are the one running models and using the GPUs, it's okay to keep going for several hours but when finished make sure to save your work and close JMP in order to clear all GPU memory. To use the second GPU in the Torch add-in, under Advanced Options, select Device "cuda:1" the first time you run the add-in.
5. Enjoy running Torch models with NVIDIA GPU speed! Choose from Example Data, the Torch Storybook, or your own data. The C:\Public folder is shared with Everyone and should be convenient for transferring files. Please also feel free to create folders in C:\ to save data and results.
6. Please report any problems, feedback, questions, or suggestions here or direct message me.
We understand Step 2 is wasteful of disk space because each of us ends up with our own set of the large dlls. The C: drive has 1 TB space so we should be okay for a while. If any of you have familiarity with setting up an add-in with shared files on a server like this, please contact me and we can try it here. I'm not sure if the dlls are reentrant--this would be good to ascertain for customers who may opt for a setup like this. Procter & Gamble is already talking about it.
Re: Torch Deep Learning Internal Discussion and FeedbackFix planned for 18.0.1 cutting out the middle of long messages for window display and still sending the full message to the log.
Re: Torch Deep Learning Internal Discussion and FeedbackCross-post from Teams "New in JMP 18 Training" Chat:
Hi All, An updated Torch Storybook is attached with six new examples:
Would like to dedicate “Love Your Smile” to Ian, who’s warm and curious smile has sparked countless fruitful interactions and has been at the center of what makes our JMP culture and software so strong. Not to mention his great add-in for displaying chemical structures from SMILES strings. And get this: similar to the liquid chromatography retention time example from @Chunwei_Ma on peptides, we can accurately predict chemical activities like IGC50 using a transformer language model like BertTiny with SMILES strings as inputs. Really impressive and inspired by Ian :cara_que_sonríe_con_ojos_sonrientes:
Hat tips to @lkimhui for the SEM images, @Byron_JMP for the M&M pictures (same data as Manchester), @jiancao for the Toronto casino survey data, and @Chunwei_Ma for Yeast_RPLC. Really appreciate all the great feedback and interest, and we’re just getting started! Quite a few fixes and improvements are forthcoming in 18.0.1.
@lkimhui and Amanda Loh did a great job with their Torch webinar for Asia/Oceania:
Torch Add-in JMP Pro 18 - 20 Mar 2024.mp4
Finally, for those of you whose hardware is sluggish, we are working with IT to stand up an internal Windows server with NVIDIA GPU passthrough that you can remote desktop into and run JMP Pro with Torch. Details should be coming soon.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @russ_wolfinger , after a good night of sleep JMP allowed me to install again 
Re: Torch Deep Learning Internal Discussion and FeedbackHi @Florian_Vogt There must be some file or process open from previously running the add-in or possibly from a second JMP instance. Make sure all JMP sessions are closed, verify nothing in Task Manager, then try again. If still no good, it is okay to completely delete the com.jmp.torchdeeplearning folder and start over. If your internet speed is okay total install time from scratch should be no more than a minute or two with the downloads.
Re: Torch Deep Learning Internal Discussion and FeedbackHi all,
I wanted to update the Add-In to the Version which was recently released. I downloaded the new file and wanted to install, but then this came up:
After a restart of JMP, the Addin was gone from the menu, but I still can't install it.
Any ideas what I messed up?
Re: Torch Deep Learning Internal Discussion and FeedbackI created a Jira ticket for this issue a while back. It was closed.
Re: Torch Deep Learning Internal Discussion and FeedbackUsing a subset of the NIST data set, the settings shown in the window created an error message so long it scrolled off the page so I couldn't close it, so I had to manually kill JMP. Running on a MacBook Pro M3/Max.
Re: Torch Deep Learning Internal Discussion and Feedback@russ_wolfinger, Thanks it was an Image size issue. When I upped the image size to 112 the model worked without issue. Would it be possible to add a check or set the default image size to be compatible with the selected image model?
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @Peter_Hersh I just posted the following to the SE Training Teams chat and let me repost it here to make sure folks see it. Specifically, interested to hear if you have CUDA installed and if that makes a difference as I think your RTX A3000 should work pretty well for a lot of cases. Note also AlexNet and several other models will need Image Size to be larger than 28. 224 is often very good but will slow down run times. Image sizes of 56 and 112 can also work well in many cases and be faster than 224.
Windows Laptop Users: You might still be in luck for speedups with an NVIDIA card, at least for smaller image and text models. Please try the following steps:
1. Check what NVIDIA card you have by opening Task Manager, click the Performance tab, then look near the bottom. My ThinkPad has a Quadro RTX 3000 with 6 GB RAM, which is not too bad of a card
2. If your NVIDIA card looks okay, open a command prompt and type the command "nvidia-smi". Output should be something like the following, showing your CUDA version, card name, and its current memory and compute usage.
3. If nvidia-smi is not found, install CUDA drivers by following instructions here: CUDA Installation Guide for Microsoft Windows (nvidia.com) I know these look long but it's mostly just checks and this should take only ~10 minutes. Verify CUDA is installed properly by repeating Step 2.
4. Install the Torch add-in from myjmp.com, Early Adopter page. The first time you launch the platform there will be 7 large files downloaded. Please be patient and let these download to 100% and unzip fully. If for some reason the install gets interrupted or only partially completes, delete the entire com.jmp.torchdeeplearning add-in folder (click View > Add-Ins, Torch Deep Learning, Home Folder) and try again.
5. Try the full MNIST example by clicking Add-Ins > Torch Deep Learning > Example Data > MNIST. Click the embedded Torch script green arrow near the upper left corner then Go. While it is running, try nvidia-smi in your command shell several times to check GPU usage. Mine shows around 1 GB memory and 40% compute when running MNIST. Total run time is 50 seconds on my ThinkPad--not too shabby For comparison, the same job actually takes ~2 minutes on my desktop with a larger Quadro RTX 6000 that I used yesterday.
6. For larger image and text models, decrease Batch Size to use less GPU memory. Batch Size 8 or even 4 or 2 may be necessary. You can typically also use fewer Epochs when decreasing Batch Size, because the model takes steps more frequently and converges faster.
7. The NVIDIA C++ API currently does not currently offer a great way to clear GPU memory and it tends to accumulate after running several models. If you get an out-of-memory error message and/or your nvidia-smi output shows nearly full memory, the only recourse is to completely shut down and restart JMP Pro. Save models before doing so from the red triangle and you can pick up where you left off. Also, Ctrl-J is handy to take snapshots of current ouptut and save them as a JMP journal--this is what I did for the journals in the storybook
Re: Torch Deep Learning Internal Discussion and Feedback@russ_wolfinger, Yes go ahead and use the data set it is a subset of a larger data set that I got from Kaggle. I ran a mid-sized data set with ~3000 images and it worked well for the default settings using the LeNet5 image model. Something about those three model types was causing issues for me.
I have a NVIDIA RTX A3000 GPU with 6GB of dedicated memory. I have a 64GB CPU.
Re: Torch Deep Learning Internal Discussion and Feedback@Peter_Hersh Thanks for reporting. The embedded script with 8 different image models runs cleanly all the way through on my machine with a GPU in around 10 minutes and the final GPU memory usage is 4 GB. The error message you received appears to be legit--your machine ran out of memory. May I ask what kind of hardware you are running on?
AlexNet and DenseNet121 require larger values for Image Size like 112 or 224. We are working on better error messaging around this.
This is nice data with chest x-rays. Would it be okay to add it to the storybook? What is the source of the data?
Re: Torch Deep Learning Internal Discussion and FeedbackA couple of the image models that I have tried with this data set give me an error message. The AlexNet, EfficientNetB0, DenseNet121 all gave the below error message.
Re: Torch Deep Learning Internal Discussion and FeedbackHi @russ_wolfinger
A lead has just asked me if it's possible to implement his own model architecture. Is this possible? And how do I go about it?
Thanks for your answer
Re: Torch Deep Learning Internal Discussion and FeedbackHi @Steph_Georges ,
Think it was mentioned in the Add-in Help documentation. Personally, I have yet to try creating my own model, but I do have prospect asking exactly the same question.
Hope this helps
Re: Torch Deep Learning Internal Discussion and FeedbackThanks much @lkimhui for the nice quote and explanation.
Hi @Steph_Georges , as @lkimhui mentions, we have set up the add-in to automatically read all JIT traced .pt files in the indicated folders for image or text modeling. Users can create and pretrain these models in PyTorch, use JIT script or trace to save them, then copy them to these folders and they will be shown in the menu the next time the add-in is launched. We are also very open to suggestions for models to be officially added to the add-in.
Let me also mention that for tabular and image data, the interface itself is fairly flexible for designing custom architectures using Advanced Options like Layer Sizes, Activation, Dropout, and Norm. These are designed to work with MLP (tabular), CustomConv1d (time series / functional), and CustomConv2d (image).
Re: Torch Deep Learning Internal Discussion and FeedbackHi All, Just put together a new intro video: jmp.com/support/downloads/TorchDeepLearning18/videos/Torch_Intro2.mp4
Trying to get better with Camtasia, but wow, have a whole new appreciation for really well-done videos. Would sincerely appreciate any feedback you can provide on content, style, or related; the more critical the better!
Re: Torch Deep Learning Internal Discussion and FeedbackThanks for doing this Russ, in conjunction with your storybook, it will be a great help to SEs.
Regarding your feedback request, it depends on your target audience. For SEs we will likely need more technical details as we dig deeper into the methods.
For users then my guidance would be to start with the story (in your storybook) -- what it is a typical user does today to solve the problem, the issues with staying as is, the opportunity the torch addin offers to improve that, then show just the end result (the aha moment, avoiding how to get to the end result), then describe the user wins and business benefits that result. Leaving the audience motivated to explore further.
Re: Torch Deep Learning Internal Discussion and FeedbackHi Torchers, Putting together a new 2-minute intro video. Seeking feedback on the first cut--attached. Thanks!
Re: Torch Deep Learning Internal Discussion and FeedbackThis is a great and feature rich trailer Russ which will likely appeal to potential users who are already applying the methods to some degree but wish to save time and possibly get to a more informative decision. If on the other hand your target users are those who have the problem or need (e.g. to replace manual image classification with "automated" image classification to avoid tedium, errors and save time) but who are not familiar with any of the methods, then you might instead wish to start with the pain/problem (e.g. it is time consuming, error prone and tedious work to manually classify images) and paint a vision of the solution showing the outcome of the analysis with less emphasis on the how-to part --- more emphasis on the user wins.
Best,
Malcolm
Re: Torch Deep Learning Internal Discussion and FeedbackOn a related note, intro videos for XGBoost and Venn Diagram attached. Spinning these up for the Marketplace--feedback welcome!
Re: Torch Deep Learning Internal Discussion and FeedbackThe newest documentation is extremely helpful particularly in explaining how to score new data. Two features I think would make it even easier. First, an simple way to delete old models. Maybe I'm missing where this is done but I have been going to the folder on disk where the models are stored and deleting them via the OS. Second, a simple way to recover what was used for the validation column(s). I'm assuming when scoring new data, the exact number of validation columns, modeling types, and levels is needed. I find this level of detail is hard to keep track of, particularly when a large number of models is built.
Re: Torch Deep Learning Internal Discussion and FeedbackRuss, I'm running into an error when I try to save a model. I've tried saving it locally in the default AppData folder, on one drive, my desktop, and I keep getting an alert that the folder does not exist. Thoughts?
Re: Torch Deep Learning Internal Discussion and Feedback@Peter_Polito Thanks for reporting. The way the code is currently set up, the Folder field under Advanced Options needs to be specified before running the model. Then when the model finishes training, a subfolder is created under this main folder with default name "Model 1". The "Save Model" red triangle option then just renames this subfolder.
If all else fails, you can also manually rename or move the subfolders in Windows Explorer or Mac Finder. This gets to @DonMcCormack 's suggestion to provide an easier way to manage these folders--nothing yet on this but it is a good idea.
Re: Torch Deep Learning Internal Discussion and FeedbackIt's been a while since I have been on the server. I wanted to re-install the Torch add-in and got the following message ("Unable to complete the install. The destination directory or file in the destination directory is locked." Suggestions?
Re: Torch Deep Learning Internal Discussion and FeedbackI have been getting a couple different errors when I try to score a new data set with a model.
I was trying to recreate something I had done in the past but for some reason, it is no longer working.
Re: Torch Deep Learning Internal Discussion and FeedbackI should have mentioned this was done in JMP Pro 19.0.0. (Everything works fine in JMP Pro 18)
Re: Torch Deep Learning Internal Discussion and FeedbackAny updates on being able to save the CAMs properly? I have a customer asking about this, who is experiencing this "black images" issue.
Re: Torch Deep Learning Internal Discussion and FeedbackI received a question from one of our customers along the following lines:
“I think many users of Torch Deep Learning are already proficient in Python. In that context, what is the value of performing deep learning on JMP, and are there any real-world adoption examples?”
In response, I have organized my thoughts as follows.
First, during the PoC and exploratory phases, JMP offers a clear advantage in speed and ease of experimentation. Image and text-based deep learning models can be tested quickly without writing code, enabling rapid assessment of feasibility.
Second, JMP Pro is not intended to replace Python. In practice, when advanced modeling relies solely on Python-capable users, those individuals can become bottlenecks. Enabling non-Python users to conduct PoCs and validation work in JMP Pro helps distribute responsibilities and move analysis forward more efficiently across the organization.
Based on this, I would appreciate your insights on any additional benefits, as well as real-world adoption examples—particularly typical use cases and examples that have progressed from PoC to production.
Thanks.
Re: Torch Deep Learning Internal Discussion and Feedback@YusukeKoizumi Good thinking and replies.
Third, JMP Pro offers a dynamically interactive environment with a rich collection of numerical and graphical outputs during both training and inference. For example, drag a rectangle over misclassified validation points to select them in the JMP table and perform a visual error analysis, or add a Profit Matrix to your binary response and interactively optimize decision thresholds.
