Reviewer name and names of any other individual's who aided in reviewer |
Ken Cho |
Do you understand and agree to our policy of having open and named reviews, and having your review included with the published manuscript. (If no, please inform the editor that you cannot review this manuscript.) |
Yes |
Is the language of sufficient quality? |
Yes |
Please add additional comments on language quality to clarify if needed |
|
Is there a clear statement of need explaining what problems the software is designed to solve and who the target audience is? |
Yes |
Additional Comments |
|
Is the source code available, and has an appropriate Open Source Initiative license <a href="https://opensource.org/licenses" target="_blank">(https://opensource.org/licenses)</a> been assigned to the code? |
Yes |
Additional Comments |
|
As Open Source Software are there guidelines on how to contribute, report issues or seek support on the code? |
No |
Additional Comments |
|
Is the code executable? |
Yes |
Additional Comments |
But the tool can only be executed in Linux or inside container with base image is Ubuntu, which is contradicted to `Testing was conducted from multiple operating systems.` mentioned in paper. |
Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined? |
No |
Additional Comments |
The tool's processing time in my case is much longer, dataset 102379 (4.154s), dataset 102374 (6.34s) and dataset EGAS00001003065 (56m20.63s). But the extra time may be due to the location of the bucket region and network speed, so it is not a big concern for the tool's reproducibility. |
Is the documentation provided clear and user friendly? |
Yes |
Additional Comments |
|
Is there enough clear information in the documentation to install, run and test this tool, including information on where to seek help if required? |
Yes |
Additional Comments |
But the tool can only be executed in Linux or inside container with base image is Ubuntu, which is contradicted to `Testing was conducted from multiple operating systems.` mentioned in paper. |
Is there a clearly-stated list of dependencies, and is the core functionality of the software documented to a satisfactory level? |
Yes |
Additional Comments |
|
Have any claims of performance been sufficiently tested and compared to other commonly-used packages? |
No |
Additional Comments |
|
Is test data available, either included with the submission or openly available via cited third party sources (e.g. accession numbers, data DOIs)? |
Yes |
Additional Comments |
|
Are there (ideally real world) examples demonstrating use of the software? |
Yes |
Additional Comments |
But the tool does not work on multipart upload objects, which makes it unavailable to those multipart uploaded datasets. |
Is automated testing used or are there manual steps described so that the functionality of the software can be verified? |
No |
Additional Comments |
|
Any Additional Overall Comments to the Author |
### What did well
1. The script `aws_check_integrity.sh` can be easily executed using docker.
2. The script `aws_check_integrity.sh` has tested on a range of file size, and the processing time is reasonable.
3. The paper has mentioned its limitation on multipart uploaded datasets, and set the boundary of the tool's use case.
### Problems
1. The script `aws_check_integrity.sh` can only be executed in Linux OS, which is contradict to `Testing was conducted from multiple operating systems.` mentioned in paper.
### Suggestions
1. The script `aws_check_integrity.sh` contains 4 errors, 14 warnings after [shellcheck](https://www.shellcheck.net/), which need to be addressed by author.
2. The paper has mentioned its limitation on multipart datasets, but should have also documented in its [repo](https://github.com/SoniaRuiz/aws-s3-integrity-check/) and also in the help message of the script.
3. The script `aws_check_integrity.sh` has typo which need to be addressed by author.
4. To implement tests on the script itself, eg. using the [bats-core](https://github.com/bats-core/bats-core).
5. To remove OS dependency, may be focusing on implementing the tool in Docker and document its details usage in paper and also in [repo](https://github.com/SoniaRuiz/aws-s3-integrity-check/).
|
Recommendation |
Minor Revisions |