| Reviewer name and names of any other individual's who aided in reviewer | Juntao Liu |
| Do you understand and agree to our policy of having open and named reviews, and having your review included with the published manuscript. (If no, please inform the editor that you cannot review this manuscript.) | Yes |
| Is the language of sufficient quality? | Yes |
| Please add additional comments on language quality to clarify if needed | |
| Is there a clear statement of need explaining what problems the software is designed to solve and who the target audience is? | Yes |
| Additional Comments | |
| Is the source code available, and has an appropriate Open Source Initiative license <a href="https://opensource.org/licenses" target="_blank">(https://opensource.org/licenses)</a> been assigned to the code? | Yes |
| Additional Comments | |
| As Open Source Software are there guidelines on how to contribute, report issues or seek support on the code? | Yes |
| Additional Comments | |
| Is the code executable? | Yes |
| Additional Comments | |
| Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined? | Yes |
| Additional Comments | |
| Is the documentation provided clear and user friendly? | Yes |
| Additional Comments | |
| Is there enough clear information in the documentation to install, run and test this tool, including information on where to seek help if required? | Yes |
| Additional Comments | |
| Is there a clearly-stated list of dependencies, and is the core functionality of the software documented to a satisfactory level? | Yes |
| Additional Comments | |
| Have any claims of performance been sufficiently tested and compared to other commonly-used packages? | Yes |
| Additional Comments | |
| Is test data available, either included with the submission or openly available via cited third party sources (e.g. accession numbers, data DOIs)? | Yes |
| Additional Comments | |
| Are there (ideally real world) examples demonstrating use of the software? | Yes |
| Additional Comments | |
| Is automated testing used or are there manual steps described so that the functionality of the software can be verified? | Yes |
| Additional Comments | |
| Any Additional Overall Comments to the Author | Julearn is an open-source Python library that allows domain experts without in-depth ML and programming language training to design and evaluate complex ML pipelines without encountering in common pitfalls, such as data leakage and overfitting of hyperparameters. I acknowledged it is a user-friendly software and deserves more attention. Yet, I have some minor concerns. 1. It is advisable to include the documentation website of the software in abstract, as comprehensive documentation serves as a crucial resource for users seeking to utilize the software effectively and efficiently. 2. When I run the first example mentioned in the manuscript, I got some problems below: 1) I run the first step, 1_get_data.py to download the data to the path of data/1_brain_age. But when I executed the second step, 2_predict_brain_age.py, I got the error “FileNotFoundError: [Errno 2] No such file or directory: 'data/ixi.S4_R8.csv'”. I changed the code in line 19 of the 2_predict_brain_age.py, data_dir=Path(__file__).parent.parent/"data", to data_dir=Path(__file__).parent.parent/"data"/"1_brain_age". Then it can run successfully. 2) Before I run the 2_predict_brain_age.py, I got the “ModuleNotFoundError: No module named 'skrvm'” error. I don’t know how to install this python library until check in the 2_predict_brain_age.py file. Providing a README file within the directory of each example that details the steps of executing the example code would be useful, as it would allow users to easily download the data required and comprehend the installation process for required libraries and dependencies. 3) As well as the second example, I need to check the 1_prcess_data.py to known how to download the ADNI data. Still you can write a readme file for each example. 3. I found that Figure 3 appears before Figure 2. Maybe you can change the orders of this two Figures. 4. The codes in Figure 1 and 2 are intended to show the differences between julearn and sklearn. Instead of the Figures, you can just put the code in an in-line text box, maybe it is clearer. Furthermore, code annotations can be included to provide guidance on potential applications of this code to users. 5. In the Inspection and Analysis section, you mentioned that Julearn includes two functionalities: a preprocess_until function and Inspector class. Maybe you can also provide a code in an in-line text box to demonstrate how these two functionalities can be used. |
| Recommendation | Minor Revisions |