投稿记录

Ref.: Ms. No. IJSEKE-D-22-00082
Automatic generation of system-level security test framework from semi-formal natural language
Int'l Journal of Software Engineering and Knowledge Engineering
Dear Dr. Zhi Li,
Reviewers' comments on your work have now been received. You will see that they are advising against publication of your work. Therefore I must reject it.
For your guidance, I append the reviewers' comments below.
Thank you for giving us the opportunity to consider your work.
Yours sincerely
Shi-Kuo Chang
Editor-in-Chief
Int'l Journal of Software Engineering and Knowledge Engineering
Reviewers' comments:

Reviewer #1:

The author's description of the method and the overall structure of the paper are relatively clear. However, there are serious problems in the motivation and experiment of the paper.
1. The authors did not express their motivation or inspiration clearly. For example, although authors claimed "Using requirements templates, i.e. restricted natural language, to represent requirements can solve such problems.", the reasons were not explained. Authors claimed "However, EARS is inadequate for elaborating the environment of complex CPS behaviors and its requirements.", so what is the disadvantage of EARS, please give specific examples. I suggest that the authors clearly express motivation and the advantages of proposed method, rather than a simple statement which will confuse the reader.
2. There are serious problems in the experimental part. There is no description of the evaluation index or even any comparison method in this paper. Therefore, how the effectiveness and time efficiency of the proposed method can be proved?
3. There were many syntax and other errors, such as "The environment in which Cyber-Physical Systems (CPS) are located is often closely connected to the daily activities of users, so it is vital to ensure that the CPS does not run into errors that could have an impact on the environment in which it is operating and cause financial loss, particularly, testing of security-critical systems helps to avoid unexpected behaviours that could lead to catastrophic events[3][4]." In this paper, I suggest that the author re-examine the paper carefully.

Reviewer #2:

1. the writing of the paper needs to be further polished. For example, in the first paragraph, the author wrote, "There are many factors that affect software security, of which the environment is also an important factor[8][7]", which is a bit weird. It would be better if the author used "There are many factors that affect software security, such as the environment." Besides, the author should avoid citing [8] before [7].

2. some points the author gave need to be more solidly supported. For example, the author said, "Using requirements templates, i.e. restricted natural language, to represent requirements can solve such problems", but no reference is provided here. Citing references properly to support your points is essential. Besides, the author proposed EARS+, an extension of EARS, as one of their major contribution. But the original paper of EARS [1] was not even cited.

[1] Mavin, Alistair, et al. "Easy approach to requirements syntax (EARS)." 17th IEEE International Requirements Engineering Conference. 2009

3. The current shortcomings of EARS should be explained more clearly. For example, the author first said, "Furthermore, timing requirements are important to real-time systems, in which data-intensive computations play a key role", then the author said, "thus dealing with large amounts of data can have a great impact on the security and quality of the software". I can hardly understand why would the first sentence be the reason or cause for the latter sentence, as the author used "thus" as a conjunction. Nothing about data affecting the security and quality of the software is discussed in the first sentence, and the author goes straight to the conclusion that it "has a great impact on the security and quality of the software".

The audience can only understand the intuition of the paper if the current shortcomings of EARS are explained clearly.

4. The authors discussed in the related work that the existing extensions of EARS did not deal with the relationships between requirements. But these point was confusing. First of all, the point of novelty was never mentioned in the introduction, and no explanation on the intuition of it was provided. Besides, what does the "relationships between requirements" means? No definition or description about it is given.



Generally, the paper need to be carefully revised before submission. The delivery of the paper need to be improved, as well as English writing.

Reviewer #3:

This paper proposes a framework to automatically generate test cases for cyber-physical systems (CPS) using an extended EARS+ requirement specification. The work is very interesting because CPS systems have been widely used in many areas and the security issue is important for CPS systems. Three data sets are used to evaluate the effectiveness of the proposed framework. The experimental results show that the proposed framework effectively generates test cases.

Extending an existing requirement specification EARS is interesting. However, many points need to be improved further in the paper. Here are the comments about this work.

1. This paper mentions that the proposed generation framework is designed for CPS. However, characteristics of CPS, security issues of CPS, and related work of CPS are not introduced or described. In addition, there is no CPS related dataset used in the experiments. The paper should be revised to emphasize this part.
2. This paper should provide enough descriptions of EARS as a subsection. To explain the EARS+ design, this paper does not provide enough EARS+ examples to show that the security issue can be improved. Especially, more examples are needed to illustrate the transformation.
3. The related work section only describes the past studies. This paper does not discuss the shortcomings of each related approach and describe which is the state-of-the-art approach for the problem addressed in the paper.
4. The related work section does not consider studies of CPS security. If this is the focus of the paper, studies of CPS security should be also discussed in the related work section.
5. This paper only discusses the proposed framework in the experiment section. It does not include other framework(s) for comparison. Is EARS+ better than other approach(es) to solve the security problem in CPS systems?
6. The paper should be carefully revised. Many sentences are ambiguous. For example, "… where INPUT is used for input data, OUTPUT is used for output data and ACROSS is used for use data." on page 5, "… all general requirements ure, complex requirements CR, and data statements DR are obtained." on page 8, and "Table 4.2" on page 10. This paper also needs proofreading by a native English-speaking editor to correct typos and grammatical errors.

__________________________________________________
In compliance with data protection regulations, you may request that we remove your personal registration details at any time. (Use the following URL: https://www.editorialmanager.com/ijseke/login.asp?a=r). Please contact the publication office if you have any questions.