573697_en-US

liliconcom

Reviewing File Exchange files

Reviewing add-ins/scripts of other people takes a long long time (especially if there are any issues). One big issue is how File Exchange/JMP add-ins work, but it is a different discussion. Some things to think:

• Should we try to define some sort of outline what should be reviewed when going through File Exchange files?
• Should we have some guidelines to users which have to be complied with before the files can be accepted?
• Should be have instructions on how users can have their files published?
• Make the review processes steps public so users know what has to be complied with and what is being checked.
• I would also add some disclaimer to File Exchange section that all of the add-ins are used on the responsibility of the user as review process won't catch everything.

 

There are a lot of things that ruin your JMP experience if you install / use add-ins. Things like Clear Symbols, messing with namespaces, using Quit/Exit and in my opinion most annoying messing with Preferences.

 

Below is a small list that came to my mind about things shouldn't be allowed:

1. No namespace pollution (use here/window/box or anonymous namespaces...)
2. No Clear Symbols/Clear Globals
3. No touching user's preferences unless it is the purpose of add-in
4. No connections to services (requests) outside JMP WITHOUT user clearly knowing it
5. No Quit / Exit
6. No adding scripts directly to the add-in/toolbar as it is a nightmare to review. All scripts should be in .jsl files which are installed during add-in installation
7. If add-in is added to already existing menus besides Add-Ins it has to be made clear that it is a custom add-in.
   1. 
8. If installing toolbar do not mess with the existing toolbars and always add custom.
9. Avoid Start-Up and Exit scripts
10. (No encrypted code which cannot be reviewed, meaning that the key would have to be provided to reviewer to check the code

Re: Reviewing File Exchange files

While a lot of those sound like best practices, I am a worried that the user community here isn't big enough to support those restrictions, meaning that many add-ins or scripts would never be submitted.  I do think we could ask users which of these things they are doing though, and at least make it clear what might happen if someone uses a script. Perhaps something in a new-post template?

 

Re: Reviewing File Exchange files

First, thank you for starting the discussion regarding Add-ins.

 

@jthi I agree with all your statements. Yes, I do think we should provide a set of guidelines that must be met in order to submit an add-in. Yes, we should provide an explanation of the review process. If a user is not willing to abide by the guidelines or not willing to take the time to meet the guidelines, I'm fine with sacrificing the publication of that add-in. I would rather develop a library where a user has a higher level of trust in the add-in as a result.

 

You have provided an excellent list with which to start the guidelines. I can work with our writers to develop a knowledge article for the guidelines and review process along with a disclaimer about using any of the add-ins provided.

 

As you stated, reviewing an add-in is a non-trivial task. What suggestions do you have for reviewing? Is this a task you (the Super Users) would do (possibly in addition to any JMP staff members I can recruit)? Does the review consist of verifying only the guidelines or should additional steps be taken? Interested to hear other thoughts.

 

Add-ins are only one part of the File Exchange. What we decide for this library may also apply to Scripts and Sample Data.

 

For 2023, we hope to begin taking a more in-depth review of each section of the Community to determine ownership, design, functionality, etc. So, please let us know what improvements you think would benefit the File Exchange (or just the add-ins) so that we can further discuss and build into our roadmap.

 

Thanks.

 

Re: Reviewing File Exchange files

I have been working with @Wendy_Murphrey and @_jr to develop a process using two recently submitted add-ins: QuickSelect Toolbar and Graph Builder Toolbar. 

 

Here are a few notes based on our reviews:

 

• Started a template to provide more uniformity among the submissions and hopefully make it easier to find relevant information about the add-in. We did not modify the content of the original post to align with the template.
• While we installed and checked some basic functionality as stated in the associated article, we did not test. We feel testing is beyond the scope of a reviewer.
• Scanned the code to verify it adhered to the guidelines, i.e. does it modify user preferences, does it make calls to the operating system or external service, etc. Time for code review was around 15 minutes.
• Unless absolutely necessary, no more than 2 reviews. The first review may be more rigorous and result in the add-in being pushed back to the author. The second review is to ensure any major violations found in the first review have been corrected (and no new ones introduced). We did not want to make the process of submitting an add-in too much of burden and discourage participation.

We will be documenting the steps we took in reviewing the code as a resource for other reviewers.

 

Our next step is take what we've learned from this iteration and apply to another request.

 

If you have any questions, please let me know. Thanks.