Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool

Abstract

User reviews on software platforms offer direct insights into user attitudes toward software packages, serving as invaluable sources for assessing software quality. However, there is a lack of effective methods or tools to analyze the quality attributes that are of most concern in user reviews, as well as the strengths and weaknesses of software packages in terms of quality. This paper introduces ISOftSentiment, a hybrid tool developed to overcome this challenge and facilitate efficient evaluation of software package quality attributes based on user reviews. ISOftSentiment integrates a sentiment analysis model with the ISO/IEC 25010 product quality model and a fuzzy consensus mechanism. This integration processes user reviews to produce a quality sentiment matrix, which not only summarizes the quality analysis results but also highlights the consensus sentiment for relevant quality attributes. The matrix provides insights into both the strengths and weaknesses of software packages. Based on these insights, software engineers gain a clearer understanding of a package’s quality and trustworthiness, thereby making more informed decisions during the software selection or evaluation process and avoiding the use of unreliable components. Consequently, this tool ensures that the final product maintains high standards of quality.