A satisfaction evaluation scheme based on entropy method for public opinion analysis

Huang Zhiwei; Luo Shiguang; Li Yan; Qiu Lingjie; Tang Huiyu

doi:10.1117/12.2637369

24 May 2022 A satisfaction evaluation scheme based on entropy method for public opinion analysis

Huang Zhiwei, Luo Shiguang, Li Yan, Qiu Lingjie, Tang Huiyu

Author Affiliations +

Proceedings Volume 12260, International Conference on Computer Application and Information Security (ICCAIS 2021); 122601X (2022) https://doi.org/10.1117/12.2637369
Event: International Conference on Computer Application and Information Security (ICCAIS 2021), 2021, Wuhan, China

Abstract

Government policy performance evaluation based on the satisfaction of the masses is a hot issue, involving the design of evaluation system, model construction, parameter simulation and so on. Based on Delphi method, research resolves the government’s policy, the types of the masses, builds satisfaction evaluation index system, Then, based on entropy value method, the satisfaction of a city is evaluated, and finally, the grey correlation measurement is used. Empirical results show that the weights of urban residents, rural residents, business workers, civil servants and migrants are respectively 0.1603, 0.2227, 0.1573, 0.3005, 0.1591, and their evaluations to the government are respectively 3.5781, 3.6414, 3.6969, 3.7734, 3.6276. The results show that the performance evaluation of local government is a comprehensive evaluation of people to the government performance, can give feedback to the appeals of the masses and get public opinion analysis in hot areas in time, is conducive to the improvement of government work. The evaluation system constructed has high discrimination and promotion ability.

1. INTRODUCTION

Government performance evaluation is an indispensable measure for the government’s own construction. It is not only conducive to understanding the performance of policy implementation, but also able to timely obtain public opinions, so as to improve the level of government governance. It is the internal driving force of administrative system reform¹. It is also the internal requirement and guarantee of building the government by law². The key point of the value chain of government performance evaluation lies in the evaluation of the people’s satisfaction³. Satisfaction is a person’s feeling state, is people’s feeling after comparing the expected value with the actual gain. People’s satisfaction with government can be simplified as Public Satisfaction Index (PSI) of government service level. PSI refers to a psychological experience that the public measures and compares the satisfaction degree of its goals, expectations and needs when experiencing public administration, also known as Public Satisfaction Index. Although the government strategy cannot be fully projected on the experience of the masses, it can reflect the effect of government governance on the whole. The core of the evaluation lies in the construction of the indicator system. Xu et al. believe that the indicator system should be composed of two major modules: decision-making performance and executive management performance⁴. However, administrative performance and management performance are limited to the long and inconsistent transmission cycle of policy implementation effects, which are often different from the actual performance.

In this paper, the main contributions are as follows: we build a complete evaluation index system covering different groups based on multiple rounds of Delphi method, and calculate the weight based on entropy method on the basis of excluding expert scoring and AHP. On this basis, we evaluate the satisfaction of different regions and groups in the investigated cities, and finally give the satisfaction structure analysis.

2. RELATED WORK

This paper argues that the construction of evaluation index system should include both decision-making performance and effect refraction performance. In terms of data collection, online public opinion analysis can partially reflect public satisfaction⁵, which is related to the research group. After all, the online public is still not representative in a statistical sense. A large number of domestic literatures and practical experience are available for reference in the study of satisfaction index⁶. In terms of specific evaluation models, rough set theory⁷ and fuzzy comprehensive evaluation⁸ have been widely applied. Some researchers also evaluate satisfaction based on data statistics and comparative analysis, and have achieved good practical results in some fields⁹: On the whole, due to the multi-dimensional compound characteristics of government evaluation, there are both subjectivity and objectivity, both subjectivity and the consideration of audience response, so there are certain requirements for evaluation methods. Some scholars believe that entropy method has unique advantages due to its objectivity of index weight¹⁰. Other scholars believe that the grey correlation method can more accurately analyze the structural satisfaction^11-12, In this paper, on the basis of the above research, entropy value method and grey correlation method are combined to conduct a comprehensive analysis of the satisfaction of a city’s portfolio policy.

The rest of this paper is organized as follows: The third part introduces evaluation model, the fourth part introduces the empirical analysis and our Satisfaction evaluation scheme, the fifth part introduces data analysis, and the last part introduces the conclusion and future work.

3. EVALUATION MODEL

3.1

Entropy method

The key to evaluate system design is to the establishment of evaluation index weight, the widely-used weight establishment method is the expert scoring method and the analytic hierarchy process; However, due to too many indicators involved in this paper, there are nearly 60 third-level indicators, which makes it extremely difficult for experts to score, and the hierarchical analysis is also difficult to carry out. In information theory, information entropy is a measure of the disorder degree of a system. The greater the variation of the index value of an index is, the smaller the entropy of information will be, the more information the index provides, and the greater the weight of the index should be. On the contrary, the less the variation of the index value of an index is, the greater the information entropy will be, the less the information provided by the index is, and the smaller the weight of the index will be. Therefore, the method of index weight in this paper is based on the variation degree of variable value.

3.2

Grey correlation model

A measure of the degree of association between two systems that varies from time to time or from object to object is called the degree of association. In the process of system development, if the trend of the two factors is consistent, that is, the degree of synchronous change is high, that is, the degree of correlation between the two is high; On the contrary, it’s lower. Therefore, the grey correlation analysis method is a method to measure the correlation degree of factors according to the degree of similarity or dissimilarity of the development trend between factors, namely “grey correlation degree”. The solving steps are as follows:

Step 1: Determine reference sequence that reflects the characteristics of the system behaviour and comparison sequence consisting of factors that influence the system behaviour.
Step 2: dimensionless processing is carried out for the reference sequence and comparison sequence
Step 3: calculate the grey correlation coefficient ξ(x_i) between the reference sequence and the comparison sequence;
The so-called correlation coefficient is essentially the degree of difference between geometric shapes of curves. Therefore, the difference between curves can be used as a measure of the degree of correlation. There are several comparison sequences x₁,x₂,…,x_n, for a reference sequence x₀, correlation coefficients ξ(x_i) of the comparison sequence and the reference sequence at each moment (That’s each point in the curve), it can be calculated by the following formula:
In which, ρ is the resolution coefficient, between 0 and 1, usually be taken as 0.5. The minimum difference of the second level is denoted as Δ(min), the maximum difference of the two levels is denoted as Δ(max), Δ_0i(k) is the absolute difference between each point on the curve of each comparison sequence x_i and each point on the curve of the reference sequence x₀.
Step 4: seek correlation degree r_i;
Because the correlation coefficient is the correlation degree value between the comparison sequence and the reference sequence at each moment (that is, each point in the curve), it has more than one number, and the information is too scattered to facilitate the overall comparison. Therefore, it is necessary to concentrate the correlation coefficient of each time (that is, each point in the curve) into a single value, that is, to calculate its average value, as the quantitative expression of the correlation degree between the comparison sequence and the reference sequence. The correlation degree formula r_i is as follows:
r_i : grey correlation degree between comparison sequence x_i and the reference sequence x₀, the closer the value r_i is to 1, the better the correlation is.
Step 5: Rank of Correlative Degree
The degree of correlation between factors is mainly described by the size order of the correlation degree, not only the degree of correlation. The correlation degree of m sub-sequence to the same mother sequence is arranged in order of size, and then the association order is formed, denoted as {x}, which reflects the “good and bad” relationship of each sub-sequence for the parent sequence. If r_0i > r_0j, {x_i} is better than {x_i} for the same parent sequence {x₀}, denoted as {x₀} > {x_j}; r_0i represents the eigenvalue of i th subsequence to its mother sequence.

4. THE EMPIRICAL ANALYSIS: INDEX SYSTEM AND DATA PRE-PROCESSING

Governance according to law, the government efficiency, government credit and government action are standards of government behaviours. Our assessment system forms four first-level indicators including specification, integrity, efficiency and deed, covers 18 secondary indexes, including the rule of law, administrative transparency, accountability, public supervision, free from corruption, credibility of government commitment system, civil servants credit, ethical government environment, government cost, government efficiency, business environment, investment in basic public services, consumer confidence and basic education strength, culture, body and health, public services, public transportation ability, social security, public safety, environmental protection and management, covering 65 third-level indicators.

A city with developed economy is selected as the evaluation object. There are five districts in A city with developed private economy. The economic volume and structure of each district are quite different, with a large number of external population. In view of these characteristics of city A, we adopt the field sampling method in data acquisition. Due to the GDP and population size of each district is roughly balanced, targeted at urban residents within the municipal units and five districts, rural residents, enterprise staff, civil servants, migrant workers, etc. To five groups mentioned above, we distribute nearly 20,000 copies of the questionnaire, among which more than 15,000 copies is recycled, including nearly 15,000 valid questionnaires. In the questionnaire, there are five choices, respectively very satisfied (5 points), relatively satisfied (4points), basically satisfied (3 points), not very satisfied (2 points) and not satisfied (1 point). Through the analysis of the questionnaire, the analysis results of the public’s satisfaction with the government work of all levels and departments in A city were obtained.

5. DATA ANALYSIS

5.1

Satisfaction based on entropy method

Urban residents were selected to analyze the impact of demographic factors such as gender, age, education level, annual income, occupation and local residence time on government satisfaction. In order to better measure the influence rule of each attribute factor on satisfaction, this paper selects the option percentage of each factor as its variable value, so as to quantify each attribute factor. We calculate the proportion of the number of people surveyed in each district among different groups, and take this as the weight to calculate the proportion of satisfaction scores of various indicators of different groups (urban, rural, enterprises, civil servants and migrant workers). The solution results are as follows:

Taking the data in Tables 1-5 and 6 as the original data of entropy legal right, the entropy value e_j and difference coefficient g_j of each evaluation index of different groups (Cities and towns, country, enterprise, civil servants and migrant workers) were calculated by using Matlab statistical software, and then the weights of each evaluation index of different groups were determined. The solution results were as follows:

Table 1.

The proportion of satisfaction score of each index of urban workers.

Satisfaction option	Government action	Economic services	Education services	Health care	Public transport	Social security	Public security	Environment protection
A	0.2117	0.2137	0.2411	0.198	0.1995	0.2181	0.2191	0.1827
B	0.3494	0.3913	0.3781	0.3537	0.3173	0.347	0.3611	0.3054
C	0.3117	0.3318	0.2998	0.3411	0.3384	0.302	0.2884	0.3273
D	0.0516	0.0293	0.0395	0.0597	0.0756	0.0494	0.0451	0.0943
E	0.0254	0.0122	0.0205	0.0371	0.0577	0.0409	0.0444	0.0613
F	0.0502	0.0216	0.0209	0.0107	0.0268	0.0367	0.037	0.0244

Table 2.

The proportion of satisfaction score of each index of rural personnel.

Satisfaction option	Government action	Economic services	Education services	Health care	Public transportation	Social security	Public security
A	0.2349	0.2460	0.2166	0.2434	0.2728	0.2510	0.1995
B	0.2956	0.3734	0.3082	0.3140	0.3302	0.3162	0.3138
C	0.3203	0.2835	0.3105	0.3496	0.3033	0.3260	0.3456
D	0.0594	0.0425	0.0657	0.0446	0.0352	0.0443	0.0627
E	0.0300	0.0165	0.0407	0.0310	0.0184	0.0342	0.0569
F	0.0599	0.0387	0.0402	0.0175	0.0400	0.0283	0.0216

Table 3.

The proportion of satisfaction score of civil servants.

Satisfaction option	Government action	Economic services	Education services	Health care	Public transportation	Social security	Public security	Satisfaction option
A	0.4136	0.3361	0.2193	0.2016	0.1832	0.2468	0.2509	0.1735
B	0.4198	0.4530	0.4248	0.3992	0.3968	0.4386	0.4775	0.3445
C	0.1603	0.1845	0.2468	0.2993	0.2873	0.2484	0.2218	0.3301
D	0.0258	0.0138	0.0621	0.0695	0.0881	0.0424	0.0302	0.0897
E	0.0105	0.0042	0.0358	0.0214	0.0381	0.0138	0.0160	0.0604
F	0.0094	0.0084	0.0113	0.0065	0.0048	0.0072	0.0030	0.0018

Table 4.

The proportion of satisfaction scores of various indexes of migrant workers.

Satisfaction option	Government action	Economic services	Education services	Social security	Environmental protection
A	0.2362	0.2318	0.2157	0.2032	0.2125
B	0.2825	0.2963	0.2535	0.2712	0.3223
C	0.2615	0.3269	0.3005	0.3314	0.3020
D	0.0617	0.0399	0.0688	0.0665	0.0708
E	0.0369	0.0271	0.0637	0.0531	0.0546
F	0.1213	0.0780	0.0978	0.0746	0.0378

Table 5.

The proportion of satisfaction score of enterprise personnel.

Satisfaction option	Government action	Economic services	Public security
A	0.2439	0.2340	0.2345
B	0.4134	0.4224	0.4079
C	0.2760	0.2605	0.2603
D	0.0374	0.0252	0.0515
E	0.0116	0.0227	0.0318
F	0.0178	0.0352	0.0140

Table 6.

Weight of each evaluation index of different groups.

	Satisfaction option	Economic services	Education services	Health care	Public transportation	Social security	Public security	Environment protection
City and town	0.1171	0.1704	0.1505	0.1387	0.1011	0.1141	0.1152	0.0929
Country	0.1170	0.1618	0.1178	0.1640		0.1599	0.1482	0.1313
Enterprise	0.3601	0.3303					0.3096
Civil servants	0.1513	0.1623	0.1046	0.1129	0.1004	0.1308	0.1459	0.0918
Migrant workers	0.1692	0.2636	0.1440			0.1908		0.2325

The weight a_j in Table 6 and proportion of satisfaction score of each index from Tables 1-5 multiplied by the score of each satisfaction option according to entropy method to get the satisfaction score p_ij and put it into the above formula, the overall satisfaction evaluation of different groups v_i was obtained, the score of each satisfaction option (A, B, C, D, E, F) is to be assigned according to the Likert scale: 5 points, 4 points, 3 points, 2 points, 1 point, 0 point. The final solution results are shown in the following Table 7:

Table 7.

Overall satisfaction evaluation table of each group.

City and town	Country	Enterprise	Civil servants	Migrant workers
3.5748	3.5817	3.7421	3.8608	3.3444

As can be seen from the above table, the overall satisfaction scores of city and town, country, enterprise, civil servants and migrant workers are all between 3 points (general) and 4 points (relatively satisfactory), indicating that all groups have a fair evaluation to the government. From the perspective of the industry, the civil servants have the highest overall satisfaction score of 3.8608, followed by enterprises with 3.7421, and the migrant workers have the lowest score.

In order to get the comprehensive evaluation model, we need to give certain weight to different groups, and finally establish the public satisfaction evaluation model of the city. Therefore, based on the different influences and contributions of different groups to the satisfaction of the government, the entropy method is used in this paper again to calculate the weight of different groups, and the results are as follows:

That is to say, citizens’ satisfaction score of the city is 3.65, close to 4 (relatively satisfied), indicating that residents of the city have a good evaluation of the government’s implementation of the new policy. The same method was used to calculate the weight of satisfaction of different groups in the five regions. Then, the weight was multiplied by the satisfaction score of each dimension to get the satisfaction score of different groups in the five regions. The solution results are as follows in Table 8:

Table 8.

The weight of satisfaction scores of each group.

	City and town	Country	Enterprise	Civil servant	Migrant workers
Satisfaction score	3.5748	3.5817	3.7421	3.8608	3.3444

The public satisfaction of the city can be obtained from the above table:

As can be seen from the above Table 9, civil servants in the five districts generally have a higher satisfaction score, while on the contrary, the satisfaction of migrant workers to the new policy implemented by the government is generally low.

Table 9.

Satisfaction scores of different groups in five regions.

	A	B	C	D	E
City and town	3.5594	3.5507	3.4294	3.6987	3.7709
Country	3.5375	3.5484	3.917	3.7124	3.252
Enterprise	3.3392	3.9364	4.0867	3.5853	3.6706
Civil Servants	3.8936	3.8865	3.6773	3.989	3.9218
Migrant workers	3.2963	3.1105	3.3122	3.715	3.4128

Taking the overall evaluation of different groups as the index, the percentage of options as the index value, and the sorted data as the original data of entropy legal weight, calculate the weights of each group under different zones by using Matlab statistical software. The solution results are as follows:

As can be seen from the satisfaction degree and ranking of the five districts in the above Table 10, the satisfaction degree of the five districts is between 3 points (average) and 4 points (relatively satisfactory), indicating that each district is relatively satisfied with the implementation of the new policy by the government. The satisfaction in area D is the highest, while the satisfaction in area A is the lowest.

Table 10.

Satisfaction in five regions.

	A	B	C	D	E
Satisfaction score	3.5781	3.6414	3.6969	3.7734	3.6276
Rank	5	3	2	1	4

5.2

Satisfaction structure analysis based on grey correlation method

In order to better measure the influence law of each attribute factor on satisfaction, this paper selects the percentage of options of each factor as its variable value, so as to quantify each attribute factor. Based on the weight data of each evaluation index of the town group obtained by the entropy method in part 3.2, aiming at weight of each evaluation index of different districts (A, B, C, D, and E districts) for the town group, calculate the public satisfaction degree of the town group in A, B, C, D, and E districts.

The public satisfaction of urban areas obtained in Table 11 is taken as a reference number. According to the calculation steps of grey correlation, MATLAB is used to calculate the correlation degree. Since the correlation degree of each factor calculated is not significant, the correlation degree of each factor’s second-level indicator minuses its mean value to get the correlation degree of each second-level indicator after transformation. After analysis and comparison, relevant factors with correlation degree of public satisfaction greater than 0.05 are selected as the main influencing factors, and the results are as follows.

Table 11.

Correlation degree between satisfaction degree of each district and related factors.

Level indicators	The secondary indicators	Correlation	Correlation after transformation
Age (years)	26-30	0.67	0.058
31-40	0.66	0.050
Education level	College	0.68	0.062
Undergraduate	0.69	0.071
Current occupation	Public institution staff	0.705	0.095
Current annual income (ten thousand yuan)	3-5	0.686	0.083
Local residence time (years)	>10	0.743	0.084

As can be seen from Table 11, the main factors affecting the satisfaction of urban public include age, education level, occupation, income and local residence time. Interviewees are mainly the institution staff, ageing between 26 and 40, educated in junior college and undergraduate, whose annual income is mainly between 30,000 and 50,000 yuan, and whose local residence time is mainly more than 10 years. Generally speaking, people of lower age, higher education background and resume, with middle income have lower demand for government services and are easier to meet, so people at this stage will have relatively higher satisfaction with the government. People who have lived in the local area for a long time will have a relatively high evaluation of the local government due to certain emotional factors.

6. CONCLUSION AND FUTURE WORK

The theoretical basis of the government performance evaluation system based on entropy method is relatively solid, easy to implement and promote. Compared with the traditional evaluation method, the entropy method can objectively reflect the utility value of the entropy value of indicator information, so as to determine the weight. This idea is very similar to the evaluation mechanism of satisfaction degree, so the entropy method is selected. Gray correlation analysis can determine the scores of multiple subjects and each subject in the corresponding indicators, which is very consistent with the research subject of this paper, and the empirical analysis shows that the obtained results have been widely recognized.

The next step is to further improve the evaluation index system and optimize the weight design according to the feedback of this round of evaluation results, and try to introduce the multi perspective data fusion algorithm to evaluate the satisfaction.

ACKNOWLEDGMENTS

Supported by the Research projects of Guangdong Education Science in 2020 in the 13th five year plan (Grant No. 2020JKDY040), Teaching quality and teaching reform project of Guangdong Province in 2020 (Grant No. 414).

REFERENCES

[1]

Zheng, F., Yu, F. and Qín, S., “Overall government performance evaluation: Hypothesis, theoretical demonstration and empirical study on Guangdong Province 2008,” Information Science, 8 (3), 13 –25 (2011). Google Scholar

[2]

Zheng, F. and Lu, Y., “Construction and the performance evaluation system of a nomocracy government,” Chinese Public Administration, 2014 (6), 26 –31 (2014). Google Scholar

[3]

Bao, G. and Zhou, Y., “Study on value carrier model construction of government performance evaluation,” Journal of Public Management, 10 (2), 101 –110 (2013). Google Scholar

[4]

Xu, L., Li, J. and Zhang, Y., “On the government eco-budget performance evaluation—Based on the practical investigation,” Accounting Research, 2012 (12), 74 –81 (2012). Google Scholar

[5]

Zou, K., Zou, S. and Chen, Y., “Government information services public satisfaction evaluation study based on network public opinion,” Information Science, 32 (2), 45 –49 (2016). Google Scholar

[6]

Zhou, Y., Zhang, D. and Xiong, N., “Post-cloud computing paradigms: A survey and comparison,” Tsinghua Science and Technology, 22 (6), 714 –732 (2017). https://doi.org/10.23919/TST.2017.8195353 Google Scholar

[7]

Zhu, H. and Chen, N., “Evaluation model of citizen’s satisfaction degree of government information disclosure based on distance measure assisted rough set,” Journal of Intelligence, 29 (8), 94 –97 (2010). Google Scholar

[8]

Peng, G., “Fuzzy comprehensive model and empirical research on local government public administration performance,” Quantitative & Technical Economics, 2015 (11), 129 –136 (2015). Google Scholar

[9]

Mo, Z., “An empirical research on the quality evaluation of municipal government public information,” Information Science, 36 (8), 114 –119 (2018). Google Scholar

[10]

Zhang, Z., “Practice and reflection on the determination of collective forest land rights in Guangdong—Based on the satisfaction evaluation of farmers,” Journal of Jiangsu Agricultural Science, 46 (4), 272 –276 (2018). Google Scholar

[11]

Meng, X., Lin, C. and Wang, G., “Study on the satisfaction degree of B&B tourism service quality in Weizhou island based on analysis of grey correlation,” Journal of Guilin University of Aerospace Technology, 2019 (1), 89 –94 (2019). Google Scholar

[12]

Zhou, Y. and Zheng, J., “Efficiency evaluation of warship VHF/UHF communication based on Gray relation analysis,” Ship Electronic Engineering, 38 (1), 108 –111 (2018). Google Scholar

Citation Download Citation

Huang Zhiwei, Luo Shiguang, Li Yan, Qiu Lingjie, and Tang Huiyu "A satisfaction evaluation scheme based on entropy method for public opinion analysis", Proc. SPIE 12260, International Conference on Computer Application and Information Security (ICCAIS 2021), 122601X (24 May 2022); https://doi.org/10.1117/12.2637369

Access the abstract

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Analytical research

Transportation security

Medicine

MATLAB

Statistical analysis

Data analysis

Environmental management

1.

INTRODUCTION

2.

RELATED WORK

3.

EVALUATION MODEL

3.1