Information on the article: received: March 2013, reevaluated: March and June 2013, accepted: August 2013
Validity and Reliability in the Assessment of the Vulnerability of
Validez y confiabilidad en la evaluación de la vulnerabilidad de las redes sociales
Orantes-Jiménez Sandra Dinora
Centro de Investigación en Computación Instituto Politécnico Nacional, México
Vázquez-Álvarez GracielaEscuela Superior de Ingeniería Mecánica y Eléctrica
Instituto Politécnico Nacional, México
E-mails: email@example.com, firstname.lastname@example.org
Keywords: • social networks • reliability • vulnerability • validity • self-regulation Abstract
Nowadays, measuring the impact and effectiveness of Social Networks is impor -tant for people who use them in an individual manner, due to social or academic interests, as well as for companies that use them to evaluate or promote their products and businesses. Particularly, it is necessary to monitor and evaluate qualitatively and quantitatively, if possible, this tool that is used for the dissemi -nation of information of any kind. We have tried to create an instrument that aims to validate the reliability of social networks, taking Facebook our case study because it is one of the most widely used currently. This instrument is a tool that measures the “information vulnerability”; information that people decide to upload to the Internet as part of a social network. Regarding validity, a suitable solution focused on four factors was found: legality (self-regulation), social fac -tors, psychological fac-tors, and finally technology. It was concluded that the pro -posed mechanism can be a useful instrument that detects the information vulnerability storage in each social network.
En la actualidad, la medición del impacto y la eficacia de las redes sociales es importante para las todas las personas que los emplean de manera individual, ya sea por intereses sociales o académicos, de igual manera es fundamental para las empresas que los utilizan para evaluar o promocionar sus productos y negocios. En particular, es necesario moni -torear y evaluar cualitativa y cuantitativamente, de ser posible, la herramienta que se utiliza para la difusión de información. Se ha tratado de crear un instrumento que tiene como objetivo validar la fiabilidad de las redes sociales, tomando Facebook como caso de estudio, por considerarse una de las más utilizadas hoy en día. Este instrumento es una herramienta a manera de modelo, que mide la “vulnerabilidad de la información” que las personas deciden subir a Internet como parte de una red social. En cuanto a la validez, se encontró una solución adecuada centrada en cuatro factores: legalidad (autorregulación), aspectos sociales, aspectos psicológicos, y aspectos tecnológicos. Se concluyó que el me -canismo propuesto puede ser un instrumento útil que detecta la vulnerabilidad de la in -formación almacenada en una red social.
Descriptores: • redes sociales • fiabilidad • vulnerabilidad • validez • autorregulación
This study intends to validate the reliability of an ins -trument to measure the vulnerability of the information in social networks. This is due to the fact that during adolescence and youth, individuals require more ap -proval from other people of the same age and commu -nicate with them, and this is when they are most in- terested in belonging to a social network. Thus, in many cases, they show personal information that makes them easy targets for other people, who use the -se means of socializing for their own benefit, almost of any kind.
Owning to the importance that Information Techno -logy has reached in recent years, and in particular so -cial networks, people with other interests have also joined them, causing a rush of information and emotio -nal situations that create social influences.
By 2011, in Mexico, the use of Information Technolo -gy at homes (INEGI, 2011) has shown considerable growth, facilitating access to Internet contents such as social networks. In addition, companies have been in -fluenced and supported by Information Technology and have realized the importance of belonging to the most consulted and used social networks in Mexico.
However, behind this boom of dissemination and use of information in social networks, there is a pro -blem with self-regulation that handles the media, and causes information to be vulnerable and to be misused by people or organizations without scruples.
Reliability can be defined as the ability of a product to perform its function as intended previously. In other words, the reliability can also be defined as the proba -bility that a product will perform its intended function, without incidents during a specified period of time un -der indicated conditions. Running a reliability analysis on a product or on a system should include many types of tests, in order to determine the reliability of the in -tended product or system, to be analyzed.
For our case study, once the analyses are made, it is possible to predict the effects, changes and corrections in storage and handle the information that users provi -de within the social network. Several studies concer -ning social networks are related, linked and examined jointly, in order to determine their reliability under all possible perspectives, identifying potential problems, suggesting corrections, changes or improvements.
Nowadays, the importance of social networks in or -der to obtain emotional stability and social skills in the development of young people, is high. However, due to the misuse of information in these networks, there are socio-emotional problems, and later anxiety and de
-pression. These could determine unpleasant outcomes such as suicide.
Moreover, companies know that social networks are very important tools for their business. However, they also know that there is insecurity and misuse of infor -mation that can affect them negatively. Therefore, in this context, the opportune detection of vulnerable in -formation found in social networks becomes a priority, and the problem can be faced successfully, with proper management strategies and skills.
In Mexico there is no reliable and validated tool, ca -pable of detecting the levels of information vulnerabili -ty in social networks. Our suggestion is:
1) Determine the variance proportion of the original data that the instrument explains from a proposed model.
2) Evaluate the content validity of this instrument. 3) Evaluate the internal consistency of this instrument. Problem
There are a number of features that make social net -works more or less vulnerable to information misuse, thereby affecting their users and in some cases hurting youngsters intellectually, emotionally and socially. In Mexico, there are no reliable and validated instruments to assess the information vulnerability in social net -works. This research proposes a reliable and valid tool to measure the vulnerability of information contained in social networks. Assuming that this vulnerability is associated with the improper use of certain regulations, according to the information contained in these web si -tes with little or low security, this allows a third-party with proper technology to reach this information and use it improperly or incorrectly.
This research aims to validate the reliability of an ins -trument to measure information vulnerability stored in social networks.
Conceptualization and Social Network analysis The term social network refers to a structure and joint interconnected relationships (Mitchell, 1969). However, the term “social network” also refers to Internet plat -forms and with Social Network Analysis, and thanks to existing means it is possible to obtain several state -ments about this. In social networks, the social nature
of the network is modelled as a graph of interconnected nodes.
Consequently, the applicable rules to the measure -ment of directed graphs can be applied to this network of nodes, and the degree of a node can be calculated by adding all input and output edges or counting each edge just once. In addition, as a part of Social Network Analysis, the density of a network can be measured. This network is limited by the maximum number of ed -ges that may exist.
The greater the density on a network, the stronger the awareness among participants (Wellmann, 1981); additionally, it is also possible to calculate the local density in a subnet. First, define an actor and this provi -des a measure that -describes the ratio between the real connections and all possible connections with other ac -tors. Other interesting indicators are centrality measu -res defined by Freemann (1979) showing the popularity on the network.
Furthermore, based on graph theory, it is possible to map a social network as a graph. In the same way, the information, trends and preferences of the members of the network can be recovered.
Methodology Type of study
A descriptive study was conducted with a quantitati -ve methodology, through a questionnaire with the in -tention of a measuring ‘the vulnerability of information in social networks’, validating it and considering it reliable.
The population consisted of a group of male and female students from the first senior level semester from the School of Mechanical and Electrical Engineering (ESI -ME, Escuela Superior de Ingeniería Mecánica y Eléctri -ca) of the National Polytechnic Institute (IPN, Instituto Politécnico Nacional) in Mexico.
A probability sample was chosen considering a confi -dence level of 95% with an estimation error of 5%. The sample included 160 students of both levels. Finally, 90 subjects (56.25%) of the total sample answered the questionnaire.
Of these, 37 (23.13% of the total sample) were fema -les and 63 (39.38% of the total sample) were ma-les and their ages ranged between 18 and 26 years.
The questionnaire “vulnerability of information” was developed taking into account the technological, social, psychological and legal dimensions. This was based on the proposal of coping with information loss, arguing that there are individuals with personal and social cha -racteristics, who are dedicated to investigate the vulne -rability of social networks to extract information, in order to use it for other purposes. This questionnaire originally consisted of 39 items divided into: informa -tion risk (20 items), security strength of social networks (13 items) and social reaction from the misuse of infor -mation (6 items).
Procedure for data collection
Using the program SPSS v.10 (SPSS, 2011), a probabilis -tic study was conducted in which we chose a sample size of 100, just randomly applied to a group of male and female students from the first semester of senior level within the School of Mechanical and Electrical En -gineering (ESIME, Escuela Superior de Ingeniería Me -cánica y Eléctrica) of the National Polytechnic Institute (IPN, Instituto Politécnico Nacional) in Mexico. Their ages ranged between 18 and 26 years, considering that it is the young who are now in the majority of social networks users.
Later, students inside the school were asked for their voluntary research cooperation, ensuring data confidentiality.
A factorial analysis was performed on data obtai -ned from the original questionnaire to identify main components grouped in items. After identifying the components, we proceeded to a content analysis for the factors conceptualization and the theoretical items analysis. This allowed us to evaluate the content vali -dity. Lastly, the reliability of the questionnaire was determined by the response consistency of the items. It was answered with a 5-point Likert scale (Méndez y Peña, 2007), ranging from strongly disagree to stron -gly agree (as shown in Table 1). A stron-gly agree res -ponse corresponded to a value of 1 and 5 to a strongly disagree response.
All items were written positively, so that, a higher score means greater vulnerability.
Table 1. 5-point Likert scale Consideration Value Strongly agree 1 In agreement 2 Indifferent 3 Disagree 4 Strongly disagree 5 Communalities Results
“Commonality” is the name for the variance proportion explained by the common factors in a variable. The commonality (h2) is the sum of the squared factor weights in each row.
Factor analysis (Repositorio Institucional de la Uni -versidad de Alicante, 2013) begins its calculations ba -sed on what it is known as reduced matrix compo-sed by the correlation coefficients between the variables and commonalities in the diagonal.
As communality cannot be found until the factors are known, it is one of the problems of factor analysis. Since the Principal Component Analysis does not assu -me the existence of any common factor, the communa -lity takes 1 as an initial value. Other methods are used in different ways to estimate the initial commonality:
• Estimating the highest correlation commonality in the i-th row of the correlation matrix.
• Considering the commonality as the square of the multiple correlation coefficients between x and other variables. (SPSS gives it by default).
• The average of correlation coefficients of a variable with all others.
• Calculating from the two highest correlation coeffi -cients of that variable, the following operation:
The final commonality of each variable is given by:
Communalities of the variables (items) were estimated by the extraction method named Principal Compo -nents. This analysis considered acceptable those items
that had communalities above 0.3 an overall there were 23 of the 39 that met this condition. Items 5, 7 and 9 were better explained by the obtained factor model as the variance percentages were 60.8%, 60.8% and 60.6%, respectively. Items 10, 3 and 14 were the least explained as the single factor model can only reproduce 29.5%, 32.5%, and 32.5% respectively, of its original variability (see Table 2).
Table 2. Communalities for items asked ESIME’s students to find the vulnerability of Social Networks
Items Initial Extracted
1. Do you think it is important to have an account on a social network? 1.0 0.322 2. Do you agree on the usefulness of social networks? 1.0 0.481 3. Do you think it is important to access frequently to a social network? 1.0 0.325 4. How important is the time you spend on the social network? 1.0 0.395 5. Are the networks you belong to easy to use? 1.0 0.608 6. When you belong to a social network, do you consider it is important to read the legal agreement before signing it up? 1.0 0.516 7. Do you think it is important to have knowledge about the security provided by the social network you belong to? 1.0 0.608 8. When you upload your personal information and photos on the social networks you use, do you think that information in
your profile is in danger? 1.0 0.448
9. Are you accepted by all the people you send a friendship request in the social network you use? 1.0 0.606 10. Do you consider your personal information is at risk in the social network you use? 1.0 0.301 11. Have you ever had any problem in the social network you use? 1.0 0.458 12. Have you ever got a ‘blocked access’ from the social network you use? 1.0 0.351 13. Have you ever been a victim of bullying in social networks? 1.0 0.332 14. Have you ever experienced spoofing problems and loss of access to your account on the social network that you use? 1.0 0.325 15. Did you respond appropriately and satisfactorily to the social network you belong to, when you discovered that you were
a victim of the misuse of your information? 1.0 0.413 16. Would you recommend the social network you use? 1.0 0.370 17. Do you usually send all kinds of personal information in your account (personal photos, family, friends, address, etc)? 1.0 0.496 18. If you make available your information to only some people do you think there is enough security? 1.0 0.475 19. Do you think the information in your profile and in your social network account could be in some kind of danger? 1.0 0.445 20. Do you think the social network, you belong to, has satisfied your expectations of quality of the service they offer? 1.0 0.362
2 2 2 2 1j 2j
P2 xy xz yz r r h r =
Factor 1 (Legality (self-regulation)), Factor 2 (Social As -pects), Factor 3 (Psychological Aspects) and Factor 4 (Technology) these factors explain 44.5% of the total va -riance from the original data. It is observed that the first component explains 21.8% of the variance; whereas the rest of the components explain 22.7% of the variance (see Table 3).
Subsequently, the matrix of factorial structure was created, where the items are grouped in each of four factors. Variables were grouped into components accor -ding to the following criterion: factor loads greater than 0.3. In the case of the variables that were filled in two or
more factors, these were grouped in the most factorial load factor and those corresponded to the conceptual component (see Table 4).
Table 4. Factor structure matrix obtained from the survey ESIME’s students
Items Legality Social Psychological Technology 1. Do you think it is important to have an account on a social network? 0.524 0.684
2. Do you agree with the usefulness of social networks? 0.742 3. Do you think it is important to access frequently to a social network? 0.758 0.701
4. How important is the time you spend on the social network? 0.643 0.650
5. Are the networks you belong to easy to use? 0.726 6. When you belong to a social network, do you consider it is important to
read the legal agreement before signing up? 0.514 7. Do you think it is important to have knowledge about the security
provided by the social network, you belong to? 0.634 8. When you send your personal information and photos on the social
networks that you use, do you think that information in your profile is
in danger? 0.517
9. Are you accepted by all the people you send a friendship request in the
social network you use? 0.551 0.496
10. Do you consider your personal information is at risk in the social
network you use? 0.729
11. Have you ever had any problem in the social network you use? 0.404 12. Have you ever got a ‘blocked access’ from the social network you use? 0.714 0.650
13. Have you been a victim of bullying in social networks? 0.758 0.701 14. Have you ever experienced spoofing problems and loss of access to
your account on the social network you use? 0.554 15. Did you respond appropriately and satisfactorily to the social network
you belong to when you discovered that you were a victim of the
misuse of your information? 0.353
16. Would you recommend the social network you use? 0.418 17. Do you usually send all kinds of personal information in your account
(personal photos, family, friends, address, etc.).? 0.742 0.726 18. If you make available your information to only some people. Do you
think there is enough security? 0.359 19. Do you think the information in your profile and in your social network
account could be in some kind of danger? 0.359 0.326 20. Do you think the social network, you belong to, has satisfied your
expectations of quality of the service they offer? 0.451
Table 3. Percentage of variance explained by model’s factors to measure the vulnerability of social networks
Factors Total Percent variance Percent varianceCumulative Legality 5.024 21.845 21.845 Social 2.135 9.283 31.128 Psychological 1.621 7.046 38.175 Technology 1.444 6.280 44.455
a. (Factor 1) Legality (self-regulation), (Factor 2) social aspects, (Factor 3) psychological aspects and (Factor4) technology.
The factors were conceptualized as follows: Legali -ty (self-regulation) (Factor 1), social aspects (Factor 2), psychological aspects (Factor 3) and technology (Factor 4). After analyzing the validity of the items, in the proposed questionnaire a clear definition of the factors and indicators classified within each was ob -tained (see Table 5).
Due to its high sensitivity, Cronbach’s alpha (Cronbach, 1951) was used for detecting measurement error sources that are attributable to the sampling of situa -tional factors.
Table 6 presents the reliability indexes correspon -ding to each of the four major factors. These four relia -bility indexes can be considered acceptable according to Hernandez, Fernandez and Baptista (Hernández et al., 2006), since the highest theoretical value of alpha is 1, 0.80 is generally considered an acceptable value, therefore the high the value of alpha, increases reliabi -lity (Bogliaccini et al., 2005).
Table 6. Reliability indexes for measuring factor “Vulnerability of information in social networks”
Factors Media Standard deviation Cronbach’s alpha Legality 1.98 0.83 0.76
Social 1.57 0.80 0.69
Psychological 2.05 0.88 0.59 Technology 1.96 0.86 0.63 Table 5. Specifications from the questionnaire to measure the “Vulnerability of the Information on Social Networks”
Factor Definition Indicators
(self-regulation) There are laws that regulate the use of information in Information Technology, each Web site has the ability to self-regulate (creating its own policies) based on the existing laws of security to be offered to its users.
- When you want to belong to a social network, do you consider it is important to read the legal agreement before signing up? - Do you think it is important to have knowledge about the security
provided by the social network, you belong to?
- When you send your personal information and photos on the social networks that you use, do you think that information in your profile is in danger?
- Do you consider your personal information is at risk on the social network you use?
- Have you ever experienced spoofing problems and loss of access to your account on the social network you use?
- Did you respond appropriately and satisfactorily to the social network you belong to when you discovered that you were a victim of the misuse of your information?
- If you make available your information to only some people, do you think there is enough security?
- Do you think the social network, you belong to, has satisfied your expectations of quality of the service they offer?
Social Human qualities - Do you think it is important to have an account on a social network? - Do you think it is important to access frequently to a social network? - How important is the time you spend on the social network?
- Are you accepted by all the people you send a friendship request to on the social network you use?
- Have you ever got a ‘blocked access’ from the social network you use? - Do you consider being a victim of bullying through the social network? - Do you usually send all kinds of personal information in your account
(personal photos, family, friends, address, etc)?
- Do you think the information in your profile and in your social network account could be in some kind of danger?
Psychological Study mental processes in humans beings, animals and everything that is related to human behavior.
- Do you agree with the usefulness of social networks? - Are the networks you belong to easy to use?
- Have you ever had any problem in the social network you use? - Would you recommend the social network you use?
Technology Set of theories and techniques for the practical use of scientific knowledge.
Regarding the content validity of the evaluated ques -tionnaire, a suitable factor solution with four factors that explain up 44.5% of the variance of the measured construct was found; in this case, the vulnerability of information on social networks.
The percentage of the obtained variance, although not satisfactory, represents an important step in valida -ting the questionnaire, considering the novelty of the context in which it was applied.
According to the obtained responses, the possible merger of two factors, social and psychological aspects, was observed, because psychological aspects increase the social aspects.
On the other hand, it is possible to adapt and valida -te measurement questionnaires on information vulne -rability as needed, in the case of social networks.
The reliability indexes of each information vulnera -bility questionnaire scales on social networks can be considered as satisfactory.
Nowadays, the importance of social networks is unde -niable and they will continue to be valid in an immedia -te future. The obtained results for this sample show that the questionnaire could be very useful for early detec -tion of any possible risk in any social network. For example: identity theft or information theft.
Two important factors, which encourage people to join social networks, are the social and psychological aspects and these tend to be confused. So, it can be con -sidered that most people who join take them as only one aspect.
Bogliaccini J., Cardoso M., Rodríguez F. Construcción de índices, Confiabilidad, Alpha de Cronbach, Autexto No. 5, Serie esta -dística, Universidad Católica del Uruguay, Facultad de Cien -cias Humanas, 2005 [on line]. Available on: http://www.ucu. edu.uy/facultades/cienciashumanas/IPES/pdf/Laboratorio/ Autotextos/AT5_Serie_Estadistica_%20Indices_y_Alpha_ IPES.pdf Retrieved: June 2013.
Cronbach L.J. Coefficient Alpha and the Internal Structure of Tests. Psychometrika, volume 16 (issue 3), 1951: 297-334. Doi:10.1007/ BF02310555. ISSN0033-3123.
FreemanL.S. Centrality in Social Networks: Conceptual Clarifica -tion, Social Networks, volume 1, 1979: 215-239.
Hernández R., Fernández-Collado C., Baptista L. Metodología de la investigación, 4ta ed., México, McGraw-Hill, 2006.
Instituto Nacional de Estadística y Geografía (INEGI). Encuesta nacional sobre disponibilidad y uso de las tecnologías de la información en los hogares. (ENDUTIH), volume 1 (issue 12), 2011 [on line]. Available on: http://www.inegi.org.mx/prod_ serv/contenidos/espanol/bvinegi/productos/encuestas/espe -ciales/endutih/endutih2011.pdf Retrieved: January 2013. Méndez-Hinojosa L.M., Peña-Moreno J.A. Manual práctico para el
diseño de la escala Likert, Editorial Trillas, 2007.
Mitchell J.C. Social Networks in Urban Situations: Analyses of Perso
-nal Relationships in Central African Towns, Manchester, Univer -sity Press, 1969.
Repositorio Institucional de la Universidad de Alicante. Estadística Informática: casos y ejemplos con el SPSS, capítulo 8, Análisis Factorial [on line]. Available on: http://rua.ua.es/dspace/bits -tream/10045/12078/1/Capitulo8.pdf Retrieved: January 2013. SPSS para la investigación estadística. Consultores en estadística e
informática, Moquegua Perú, 2011.
Wellmann B. Applying Network Analysis to the Study of Support, in: Social Networks and Social Support, Gottlieb, 1981, pp. 171-200.
Citation for this article: Chicago citation style
Orantes-Jiménez, Sandra Dinora, Graciela Vázquez-Álvarez. Vali-dity and Reliability in the Assessment of the Vulnerability of Social Networks. Ingeniería Investigación y Tecnología, XV, 04 (2014): 585-592.
ISO 690 citation style
Orantes-Jiménez S.D., Vázquez-Álvarez G. Validity and Reliability in the Assessment of the Vulnerability of Social Networks. Ingenie-ría Investigación y Tecnología, volume XV (issue 4), October-De-cember 2014: 585-592.
About the authors
Sandra Dinora Orantes-Jiménez. Bachelor in Computer Science from the Universidad Centroamericana José Simeón Cañas, San Salvador, El Salvador in 1992. She obtai -ned her Master in Computer Science in 1999 from the Centro de Investigación en Computación del Instituto Politécnico Nacional, Mexico City. Her research inter -ests include: software engineering, information systems, design patterns, informa -tion management, computer audit, cloud computing, social networks and quality metrics and standards. She is a member of the Subcomité INF2 software of the Comité Técnico de Normalización Nacional de Informática de NYCE and Grupo Adhoc of the Subcomité INF2: Software. Participates in meetings of JTC1 SC7 Soft -ware and System Engeneering. She has been a programmer analyst of Banco Hipo -tecario and Seguros Desarrollo S.A. de C.V of El Salvador, and teaches at the Faculty of Engineering at Universidad Centroamericana José Simeón Cañas. She is currently professor and researcher at the Centro de Investigación en Computación del Instituto Politécnico Nacional.
Graciela Vázquez-Álvarez. Bachelor in Industrial Engineering from the Universidad Au -tónoma del Estado de Hidalgo at the Instituto de Ciencias Exactas, Pachuca, Hidal -go, Mexico. She obtained her Master of Science in Systems Engineering from the Escuela Superior de Ingeniería Mecánica y Eléctrica del Instituto Politécnico Nacio -nal. She is a Doctor in Computer Science by the Centro de Investigación en Com -putación del Instituto Politécnico Nacional. Currently she is researcher professor at the Section of Graduate Studies and Research of the Escuela Superior de Ingeniería Mecánica y Eléctrica del Instituto Politécnico Nacional (ESIME-Zacatenco-IPN).