Torch Deep Learning Internal Discussion and FeedbackHi all. A question came up today on how can the field technical sales members provide feedback on the Torch Deep Learning Add-In (TDLA). I'd like for us to use this thread for internal discussion and bug reports. If you find a bug, please provide a description of what steps led to the issue and also the dataset used (or at least a description of the dataset).
Also, as I mentioned in the training on TDLA today, don't send bugs or crash reports to Technical Support for this extension of JMP.
Re: Torch Deep Learning Internal Discussion and FeedbackA couple of the image models that I have tried with this data set give me an error message. The AlexNet, EfficientNetB0, DenseNet121 all gave the below error message.
Re: Torch Deep Learning Internal Discussion and Feedback@Peter_Hersh Thanks for reporting. The embedded script with 8 different image models runs cleanly all the way through on my machine with a GPU in around 10 minutes and the final GPU memory usage is 4 GB. The error message you received appears to be legit--your machine ran out of memory. May I ask what kind of hardware you are running on?
AlexNet and DenseNet121 require larger values for Image Size like 112 or 224. We are working on better error messaging around this.
This is nice data with chest x-rays. Would it be okay to add it to the storybook? What is the source of the data?
Re: Torch Deep Learning Internal Discussion and Feedback@russ_wolfinger, Yes go ahead and use the data set it is a subset of a larger data set that I got from Kaggle. I ran a mid-sized data set with ~3000 images and it worked well for the default settings using the LeNet5 image model. Something about those three model types was causing issues for me.
I have a NVIDIA RTX A3000 GPU with 6GB of dedicated memory. I have a 64GB CPU.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @Peter_Hersh I just posted the following to the SE Training Teams chat and let me repost it here to make sure folks see it. Specifically, interested to hear if you have CUDA installed and if that makes a difference as I think your RTX A3000 should work pretty well for a lot of cases. Note also AlexNet and several other models will need Image Size to be larger than 28. 224 is often very good but will slow down run times. Image sizes of 56 and 112 can also work well in many cases and be faster than 224.
Windows Laptop Users: You might still be in luck for speedups with an NVIDIA card, at least for smaller image and text models. Please try the following steps:
1. Check what NVIDIA card you have by opening Task Manager, click the Performance tab, then look near the bottom. My ThinkPad has a Quadro RTX 3000 with 6 GB RAM, which is not too bad of a card
2. If your NVIDIA card looks okay, open a command prompt and type the command "nvidia-smi". Output should be something like the following, showing your CUDA version, card name, and its current memory and compute usage.
3. If nvidia-smi is not found, install CUDA drivers by following instructions here: CUDA Installation Guide for Microsoft Windows (nvidia.com) I know these look long but it's mostly just checks and this should take only ~10 minutes. Verify CUDA is installed properly by repeating Step 2.
4. Install the Torch add-in from myjmp.com, Early Adopter page. The first time you launch the platform there will be 7 large files downloaded. Please be patient and let these download to 100% and unzip fully. If for some reason the install gets interrupted or only partially completes, delete the entire com.jmp.torchdeeplearning add-in folder (click View > Add-Ins, Torch Deep Learning, Home Folder) and try again.
5. Try the full MNIST example by clicking Add-Ins > Torch Deep Learning > Example Data > MNIST. Click the embedded Torch script green arrow near the upper left corner then Go. While it is running, try nvidia-smi in your command shell several times to check GPU usage. Mine shows around 1 GB memory and 40% compute when running MNIST. Total run time is 50 seconds on my ThinkPad--not too shabby For comparison, the same job actually takes ~2 minutes on my desktop with a larger Quadro RTX 6000 that I used yesterday.
6. For larger image and text models, decrease Batch Size to use less GPU memory. Batch Size 8 or even 4 or 2 may be necessary. You can typically also use fewer Epochs when decreasing Batch Size, because the model takes steps more frequently and converges faster.
7. The NVIDIA C++ API currently does not currently offer a great way to clear GPU memory and it tends to accumulate after running several models. If you get an out-of-memory error message and/or your nvidia-smi output shows nearly full memory, the only recourse is to completely shut down and restart JMP Pro. Save models before doing so from the red triangle and you can pick up where you left off. Also, Ctrl-J is handy to take snapshots of current ouptut and save them as a JMP journal--this is what I did for the journals in the storybook
Re: Torch Deep Learning Internal Discussion and Feedback@russ_wolfinger, Thanks it was an Image size issue. When I upped the image size to 112 the model worked without issue. Would it be possible to add a check or set the default image size to be compatible with the selected image model?
Re: Torch Deep Learning Internal Discussion and FeedbackUsing a subset of the NIST data set, the settings shown in the window created an error message so long it scrolled off the page so I couldn't close it, so I had to manually kill JMP. Running on a MacBook Pro M3/Max.
Re: Torch Deep Learning Internal Discussion and FeedbackIf you can sent this to the new support email for the add-in, JMPLSAddins@JMP.com, it can be investigated. I have seen the same issue and it is a known problem, however.
Re: Torch Deep Learning Internal Discussion and FeedbackI created a Jira ticket for this issue a while back. It was closed.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @malcolm_moore1 I just reset the permissions on C:\Public so Everyone has Read/Write. I had set them before and somehow they changed back. Hopefully they will stick this time.
Re: Torch Deep Learning Internal Discussion and FeedbackHi all,
I received some requests to share the slide used to present for the Torch Add-in on-demand webinar.
Here you go!
Re: Torch Deep Learning Internal Discussion and FeedbackTwo things to add to these instructions:
- You must have your computer connected via VPN in order to access server, otherwise you will not be able to connect
- When you are done, please logoff the server, there are a limited number of connections. I see that two people are on when I connected a few seconds ago. One was listed as Away and the other was Idle for 1 hr and 30 min.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @Chris_Kirchberg I just updated the instructions with first step being a VPN. For the computer itself IT presumably removed all restrictions with regard to number of concurrent users yesterday afternoon. Have you seen this restriction today? If so I can ask them again.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks Russ, all is working now.
Re: Torch Deep Learning Internal Discussion and FeedbackI did see it this morning @russ_wolfinger
Re: Torch Deep Learning Internal Discussion and FeedbackHello All,
I'm curious to see suggestions on how a customer would deploy their model outside of JMP following model creation in the Torch add-in - as it sits 'outside' of the formulas you can dispense in the Formula Depot what would be the best process?
Thanks,
Ben
Re: Torch Deep Learning Internal Discussion and FeedbackHi @Ben_Ingham , We're still working on streamlining the process, but for now there are a few examples of deploying the saved .pt files in Python notebooks. Click View > Add-Ins, select TorchDeepLearning, click on the Home Folder link, then look in the notebooks subfolder.
Each model is saved in 2-3 separate .pt files to enable output of intermediate embeddings. In Python the exact same architectures need to be specified and then the model weights are loaded from the .pt files. If you do K-fold, there are multiple models saved that can be averaged / ensembled for more stable and accurate predictions.
Re: Torch Deep Learning Internal Discussion and FeedbackHi Russ,
that's great new thanks.
What exactly do you mean by "remote desktop into"? How do I do that?
Thanks
Florian
Re: Torch Deep Learning Internal Discussion and FeedbackHi Florian,
Search for a Microsoft remote desktop app from your laptop. If you don't see it you may need to install it from the managed software centre. When you start remote desktop app you will need to follow the instructions from Russ to establish a connection.
Malcolm
Re: Torch Deep Learning Internal Discussion and FeedbackHi @Florian_Vogt I just updated the instructions with more details about Remote Desktop at the beginning. Please let me know if it doesn't work for you. I'm not familiar with how things work internationally and there may be additional steps required.
Re: Torch Deep Learning Internal Discussion and Feedback
Seconded this, please log off instances after use, idle users are still consuming the RAM.
Attached is the accumulation of 5 idle users and already consumed ~23% of the 128GB RAM.
Re: Torch Deep Learning Internal Discussion and FeedbackThanks @lkimhui. Just so everyone is clear, there's a difference between "disconnect" and "sign out". Disconnecting or simply closing the remote desktop connection application does not terminate the session, JMP and any other apps you've opened are still running on the server. To terminate the session and free up any used resources you need to "sign out". The "sign out" option is available from the windows menu, click on your login name and then select "sign out".
Re: Torch Deep Learning Internal Discussion and FeedbackThank you Kim!
Re: Torch Deep Learning Internal Discussion and FeedbackThank you~!
Re: Torch Deep Learning Internal Discussion and FeedbackHi All,
Want to try a really fast NVIDIA GPU? Seeking volunteers! Please do not let this distract you from your sales duties but this is something you may be able to use for important sales opportunities or in your free time.
You may have heard SAS is very active currently with NVIDIA-specific GPU development via CUDA coding across many of their analytical divisions. They are acquiring a bunch of new NVIDIA gear ranging all the way up to the latest H100s that can cost north of $25K.
They are in need of some benchmarking and I've been in touch indicating how easy and fast it is to get JMP Pro up and running with the Torch add-in and pegging GPUs using all the new examples we have for image and text classification and regression.
They've just set up a new Windows 2022 server for us with four separate L40 GPUs, each with 48 GB Memory! The L40s are three generations more modern than the P100s on TDL.
Please message me directly if you would like to try this new server and I will send you login credentials and instructions. All we ask is that you send back a few run times of models you try (your choice or I can recommend some), which are printed in the JMP log after each Torch model you fit.
I've been trying several different examples from the Torch Storybook for a few initial cases and am seeing ~2-3x speedups over the P100s.
Love this stuff,
Russ
