| Reviewer name and names of any other individual's who aided in reviewer | Sidney M. Bell, PhD |
| Do you understand and agree to our policy of having open and named reviews, and having your review included with the published manuscript. (If no, please inform the editor that you cannot review this manuscript.) | Yes |
| Is the language of sufficient quality? | Yes |
| Please add additional comments on language quality to clarify if needed | |
| Is there a clear statement of need explaining what problems the software is designed to solve and who the target audience is? | Yes |
| Additional Comments | |
| Is the source code available, and has an appropriate Open Source Initiative license <a href="https://opensource.org/licenses" target="_blank">(https://opensource.org/licenses)</a> been assigned to the code? | Yes |
| Additional Comments | [SUGGESTION] While not a requirement, I would strongly encourage a more permissive license to promote further reuse and development of CELLxGENE and related packages. The GPL license used can have unintended consequences (for example, at CZI we build almost exclusively with a very permissive MIT license per institutional policy; this sometimes limits our ability to collaborate with other OSS projects when their GPL license would require that we also use this more restrictive license, even though we are building also openly). |
| As Open Source Software are there guidelines on how to contribute, report issues or seek support on the code? | Yes |
| Additional Comments | |
| Is the code executable? | Yes |
| Additional Comments | [CONSIDERATION] This packages does include an override to allow unsigned executables; I do not have the security engineering background required to assess this, but I would advise the editor to seek their input. |
| Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined? | Yes |
| Additional Comments | [REVISION NEEDED] The UI is very confusing as it says to select (or drag/drop) a *file*, but it only accepts a *directory*. Please update this to be more clear; it almost prevented me from using the software entirely. I was only able to verify on Mac. Given that many experimental biologists rely on Windows, it would be ideal if another reviewer was able to verify on Windows. |
| Is the documentation provided clear and user friendly? | Yes |
| Additional Comments | [SUGGESTION] The documentation and videos in the readme are very excellent. This is not required and would be above-and-beyond, but the author may wish to consider a github pages site as this would be more approachable to non-computational users (github can be a bit intimidating) and perhaps also help with SEO and discoverability. |
| Is there enough clear information in the documentation to install, run and test this tool, including information on where to seek help if required? | Yes |
| Additional Comments | |
| Is there a clearly-stated list of dependencies, and is the core functionality of the software documented to a satisfactory level? | Yes |
| Additional Comments | [CONSIDERATION] My only minor concern with this submission is the long-term maintenance incurred by the choice to fork the CELLxGENE-gateway. Has the author considered sending their changes upstream as a PR, instead of maintaining a fork? The UI improvements are nice, but I do wonder about the tradeoff in long-term maintenance as the Novartis team is well staffed and occasionally updates the gateway (e.g., there are changes to the environment variables in the Novartis version of the gateway from 10 months ago which have not been updated in this package's fork of the gateway). Similarly, the version of cellxgene is not pinned (either to a specific version or to `latest`) in either package's requirements; while there are advantages to the current unpinned approach, it is a minor risk worth pointing out. All of this said, however, this approach is reasonable and has its benefits. Neither CELLxGENE Explorer nor Gateway are rapidly being developed, and so the risks described above are probably minimal, assuming that the author has a plan to maintain the package occasionally as the need arises due to upstream changes. |
| Have any claims of performance been sufficiently tested and compared to other commonly-used packages? | Not applicable |
| Additional Comments | |
| Is test data available, either included with the submission or openly available via cited third party sources (e.g. accession numbers, data DOIs)? | Yes |
| Additional Comments | |
| Are there (ideally real world) examples demonstrating use of the software? | Yes |
| Additional Comments | |
| Is automated testing used or are there manual steps described so that the functionality of the software can be verified? | No |
| Additional Comments | |
| Any Additional Overall Comments to the Author | This is a fantastic extension of CELLxGENE's functionality which users have been asking for for years! Thank you for contributing to our community. |
| Recommendation | Minor Revisions |