nova página do texto(beta)
Inglês (pdf)
Artigo em XML
Referências do artigo
Tradução automática
Enviar este artigo por email
Citado por SciELO 
Similares em
SciELO 
Permalink
1. Introduction
One of the obligations of the Government in realizing good governance is to provide quality public services, easily accessible and meet the interests of the community. This is important because public services have become the needs of today's society. However, based on the Markplus survey (2017), people's perceptions of public services in Indonesia are still relatively poor (Markplus Consulting, 2017). Some issues that often arise are related to the poor quality of public services including bribery practices, unclear standards and procedures, complicated bureaucratic flow and low government integrity. One fundamental problem that has not been resolved is related to the lack of public participation starting from the process of making government policy to the implementation of public services themselves (Puspitosari, Khalikussabir, & Kurniawan, 2011). In these phases, the public not only provides their opinions or aspirations but also makes criticisms or complaints about perceived services as a form of evaluation of government performance (Cohen & Norman, 1980). This legitimacy is important for the government to be able to embrace various interests in society. When legitimacy increases, the implementation of government policies will be easier because it does not generate resistance within the community. The lack of public participation will lead to the bad performance of public services (Puspitosari et al., 2011). From a public perspective, the implementation of public participation provides various benefits in the form of hearing their aspirations, reducing the risk of a policy on the livelihoods of people, create good relationships between public and government (Usman, 2019). Thus public participation is a very important factor in the development process which is certainly not free from conflict and problems. Therefore cooperation between the government and the community is needed to resolve existing conflicts.
Information technology (IT) has become an enabler that is able to eliminate the communication gap between government and the public. IT can be used as a channel for the public so that they can easily channel their participatory rights to the government. Electronic participation or e-participation enables a faster bureaucratic pathway, time efficiency and costs incurred because citizens do not have to come to the office physically (Slaviero, Cristina, Garcia, & Maciel, 2012). The model of public participation in the context of e-participation in Indonesia is quite a lot including "e-Musrembang" which is used for medium and long-term development planning, "e-Voting" has been used by the community for “Pilkadus”, "e-Lapor" which is an electronic-based national aspirations and complaints service, and many more government initiatives to facilitate public participation and dialogue in the formulation of state policies. The similarity of e-participation has the characteristic of participatory bottom-up process (Islam, 2008).
E-Lapor is a national scale complaint information system that can be used by the public to report aspirations, complaints or any information to the government electronically. E-Report aims to make it easy for the public to interact with the government and actively participate in overseeing the implementation of public services. According to Mahendra (2017), there are at least three benefits of using e-Lapor. First, e-Lapor has been connected to many government agencies, ministries, institutions and also local governments. Second, e-Lapor provided opportunities for the government to increase their transparency and accountability in the citizens who are able to track their reports which are always documented and published on the website openly. Third, e-Lapor empowered citizens to make reports more easily without going through a complicated bureaucratic process, by utilizing various technologies such as website (https://www.lapor.go.id), mobile applications, Twitter (@ LAPOR1708) and SMS number 1708. Despite the many benefits provided, however, it turns out that the national e-Lapor has not been widely used by the public (Hasby, 2018). Some of the factors that cause are people who are not familiar with e-report applications or are reluctant to use them because they have felt neglected. This is in line with our preliminary research that the local government, especially at the district/city level, has had a local complaint application that is known to the public rather than e-Lapor. Even local applications developed by local governments have not been integrated with e-Lapor. Furthermore, the successful use of the e-Lapor application has not been reported.
Based on Delon and McLean (2003), the success of a technology can be predicted from the acceptance of its users (DeLone & McLean, 2003). The user acceptance level will affect the adoption level of the technology. Therefore many researchers and organizations are increasingly interested in conducting research in the factors that influence their acceptance. Best practice models related to user acceptance studies in the IS/IT field that are widely known are Delon & McLean IS/IT Success Model, Technology Readiness Index (TRI) Model, TAM (Technology Acceptance Model) and UTAUT (The Unified Theory of Model Acceptance and Use of Technology) (
(Davis, 1989; DeLone & McLean, 2003; Parasuraman, 2000; Venkatesh, Morris, Davis, & Davis, 2003). The best practice model is the most influential model in the technology acceptance domain. Unfortunately, most of these models have been conducted in business organizations. Whereas user acceptance studies in the context of e-participation, especially the application of national complaints services, are still rarely found in IS / IT literature. Therefore, the study of factors that contribute to user acceptance in the e-participation context is important.
In this study, we developed an instrument to measure user acceptance of e-participation especially e-Lapor applications. Since this study reuses the items and constructs in a context that is quite different from their previous application, using existing instruments may not ensure that items are still valid. As such, we re-establish the validity in the e-participation context to ensure that the items represent the construct and that each individual item measures what it is intended to measure.
The content validity is an important indicator to determine the items on instrument that are relevant or essential to cover research content (Albert & Ludwick, 2011). According to Straub (2004), content validity means “the degree to which items in an instrument reflect the content universe to which the instrument will be generalized” (Straub & Gefen, 2004). At the initial stage of an instrument development, conducting content validity become the primary concern that must be assessed immediately after the items have been developed (Schriesheim, Powers, Scandura, Gardiner, & Lankau, 1993). This could avoid problems occurred associated with incomplete or biased measures which may result in researchers drawing conclusion or interpretation of the research (McKenzie, Wood, Kotecki, Clark, & Brey, 1999). Thus, it is critical to investigate content validity prior to examining other types of validity to ensure the construct are measured accurately (Lynn, 1986).
However, the number of e-participation studies using survey instruments increase while there is a little evidence that the content validity assessment of the instruments has been undertaken. For example, previous e-participation studies such (Ju, Liu, & Feng, 2019; Naranjo-Zolotov, Oliveira, Casteleyn, & Irani, 2019; Pirannejad, Janssen, & Rezaei, 2019) have developed instruments based on reviewing literature and conducted a survey without reporting any results for content validity assessment. Thus the issues of content validity is stressed on the IS/IT literature but only a few studies have been reported. This study also contributed to fill the gap of literature on content validity assessment by providing an opportunity for more IS/IT researchers to conduct content validity assessment for their instrument development.
2. Materials and methods
This research will develop instruments related to user acceptance factors in the domain of e-participation in Indonesia, especially the national complaints information system. A number of items were taken from the relevant literature comprehensively related to user acceptance in the IS/IT field. However, we modified the items adopted from the study of information systems to fit the context of e-participation especially e-Lapor application. First, we adopt Delon & McLean's IS Success Model (1992), a well-known theory related to user acceptance framework (DeLone & McLean, 2003). The items obtained are related to quality information, system quality and service quality construct. The second model that is relevant to the e-participation context is the Technology Readiness Index (TRI) proposed by Parasuraman (2000). TRI model is a theory that has already been deployed to determine the readiness of users to adopt a technology. User readiness is an important factor that influences user acceptance (Parasuraman, 2000). Based on Parasuraman (2000), user readiness is determined based on personality traits or psychology aspects such as optimism, innovation, discomfort and insecurity. User readiness is not measured by the user's ability to adopt technology. Third, we adopted the model of UTAUT-2 (The Unified Theory of Acceptance and Use of Technology) which is an extended version of UTAUT-1 (Venkatesh, Thong, & Xu, 2012). UTAUT-2 is formulated in order to better adapt it to the user acceptance framework. UTAUT-1 was unified framework that synthesizes of several theories and models: Theory of Reasoned Action (TRA), Technology Acceptance Model (TAM), the Motivational Model, TPB, the Decomposed Theory of Planned Behavior, the Model of PC Utilization, Innovation Diffusion Theory, and Social Cognitive Theory.
TAM proposed by Davis (1989) is a simple and robust theory to predict user acceptance in the IS/IT environment (Davis, 1989). Davis (1989) formulated that user acceptance was influenced by two main constructs, perceived usefulness and ease of use. TAM had been combined with other theories into UTAUT-1 framework. UTAUT-2 introduced three new variables: hedonic motivation, price value and habit (Venkatesh et al., 2012). In this study, the construct of price value is not used or relevant because the use of e-participation does not represent any monetary cost for citizens as users. Thus, the items are obtained from other constructs such Performance expectancy, Effort expectancy, Social influence, Facilitating conditions, Hedonic motivation and Habit.
Fourth, we also adapted Self-Determination Theory (SDT) that proposed motivation including intrinsic motivation and external motivation (Ryan & Deci, 2000). SDT discussed three psychological needs: autonomy, competence and relatedness. Recent studies confirmed that experiences of competence, autonomy, and relatedness were major contributors to technology enjoyment. This will increase adoption and engagement of people with technology (Karimi & Nickpayam, 2017). Thus, the proposed instrument in this study consist of 18 constructs and 75 items.
As said before, this study reuses these items and constructs in a context that is may different from their previous research and we don’t know whether the instruments still accurately represented the constructs when used in the e-participation context. Therefore, we make content validity a major concern when developing instruments to ensure that the re-ordered items can match in this study.
Thus, after we generated the initial items then we conducted the content validity. There are some approaches widely used to test content validity, such as Lawshe (1975) with famous CVR (Content Validity Approach) proposed a quantitative assessment of content validity, which involves a panel of SMEs to rate the essential of represents the objective or domain. In this approach, the SMEs are asked to evaluate whether the item is essential, important (but not essential) and not relevant (Lawshe, 1975). The CVR is calculated based on formula as follows:
Where CVR = content validity ratio; Ne = number of SMEs indicating “essential” and N = total number of SMEs. The outcome of CVR could explained by conditions below:
When all experts say “essential”, the CVR is 1.00 (100% agreement)
When the number of experts saying “essential” is more than half (>50%) then the CVR is ranging between zero to 0.99
When less than half (<50%) of experts saying “essential”, the CVR is negative (< 0.00)
In other words, high value of content validity occurred when most of SMEs hold the same opinion that an item is essential. The items retained were good test items that possessed high degrees of content validity if items with negative values of CVR or not reached significant standards have been deleted (Yang, 2011).
The content validity could be measured with another approach namely Aiken (1980, 1985), through computing validity coefficient namely V value (Aiken, 1980; 1985). Aiken (1980, 1985) also proposed a statistical method called the homogeneity reliability coefficient in order to understand whether or not the experts share the same opinions (i.e. the reliability of expert opinions). The homogeneity reliability is conducted by computing to reliability coefficient expressly H value to serve as consistency indicators for testing the significance (Yang, 2011).
The content validity is only applicable for sequential evaluation data such as the likert rating scale of instrument. The formula to compute V value of Aiken for an item j (Vj) is calculated with n expert as (Aiken, 1980, 1985):
Where c = the highest value of assessment; n = total number of SME; Sj = score given by SME; Vj = content validity of an item. The above Vj has value between 0 and 1 while a higher value indicates higher content validity of item j. Based on Aiken’s table, we could determine whether the item have significant content validity (V) or not. After obtaining V value, then we could calculate H Value of Aiken for an item k (Hj) from the formulas as follows:
Where k is the dummy variable; if n is an even number then k=0; when n is an odd number then k=1. Hj showed the level of reliability for each item evaluations ranging from 0 to 1. A greater value indicates higher homogeneity reliability coefficient (Aiken, 1980, 1985). Based on Aiken’s table, we could also determine Hj values of item have reached significant standard value.
According to (Yang, 2011), there is a problem with CVR approach proposed by Lawshe (1975) . CVR allows negative scores to be obtained if a small number of experts do not consider the essential of an item and expert judgment is very narrow because only three choices are given. Besides, our instrument in this study used five likert scale (“1=not very important” until “5=very important”) while Lawshe (1975) proposes to evaluate the measure using a scale 1-3: not relevant, important (but no essential) and essential (Lawshe, 1975). Therefore, in this study we conducted the content validity by adopting Aiken validity that also suggested likert rating scale that comply with proposed instrument (Aiken, 1980, 1985).
3. Results and discussions
In this part, the study had conducted the content validity of instrument development. Lynn (1986) suggests that the content validity can be established through two-step process: development and judgment (Lynn, 1986). The first stage (development) involves identifying the domain, generating the items based on existing instruments or produced by researchers’ understanding of the concept suggested from theory, and formulating the instruments (Carmines & Zeller, 1979). Identifying the domain is an initial step to conceptually define the construct by reviewing the literature, which is followed by generating a set of items that are later arranged in an instrument. The second stage is judgment based on expert opinion, involving asking several experts to evaluate the validity of individual items and the whole instrument. As a whole, this process could help researchers possess set of items which can be used to measure the domain of research (Grant & Davis, 1997).
Subject matter experts (SMEs) have up-to-date knowledge of their domain area because they are professionally involved in growing, maintaining and distributing their knowledge. Therefore, the involvement of SMEs in the content validity assessment may grant helpful understanding and important judgment to determine the appropriateness of individual item on instruments.
According to Saunders, Lewis, and Thornhill (2009) before pilot testing conducted, experts could give suggestion and opinion to provide content validity in order to make improvement (Saunders, Lewis, & Thornhill, 2009). In this study, there are 10 SMEs selected from the pool of reviewer based on inclusion criteria: SMEs should have an expertise in the field of information system/information technology (IS/IT); SMEs also have at least ten years of working experiences in the field of IS/IT; SMEs have published papers in the field of IS/T related to e-Government. Therefore, the panel of experts (SMEs) involved in this study have academic background (university or research institution) and they already hold a doctoral degree in IT/IS field.
3.1 Instrument development
Based on best practice models by reviewing previous literature in the domain of e-participation, we defined about 18 constructs in the proposed user acceptance instrument, described in Table 1 below. This instrument could be used to evaluate the citizen participation in Indonesia. Those constructs respectively are Social Influence (SI) has 5 items, Facilitating Conditions (FC) 6 items, Trust (TR) 4 items, Competence (CP) 4 items, Relatedness (RE) 4 items, Autonomy (AU) 3 items, Optimism (OP) 3 items, Innovativeness (IN) 4 items, Discomfort (DC) 3 items, Insecurity (IS) 4 items, Hedonic Motiation (HM) 2 items, Habit (HB) 2 items, Information Quality (IQ) 7 items, System Quality (SQ) 6 items, Service Quality (SR) 3 items, Perceived Usefulness (PU) 6 items, Perceived Ease of Use (PE) 6 items and User Acceptance (UA) has 4 items in the model proposed.
Table 1 User Acceptance Constructs Proposed.
| No | Construct | Definitions & Source |
| 1. | Social Influence (SI) | The extent to which consumers perceive that important others (e.g., family and friends) believe they should use a particular technology (Venkatesh et al., 2003, 2012) |
| 2. | Facilitating Conditions (FC) | Refer to consumers’ perceptions of the resources and support available to perform a behavior (Venkatesh et al., 2003, 2012) |
| 3. | Trust (TR) | Cohesive prominently to behavior intension (Witarsyah, Sjafrizal, Fudzee, & Salamat 2017) |
| 4. | Competence (CP) | Perceived performance ability for a specific activity or judgments of how well one can execute courses of action required to deal with prospective situations. Individuals’ inherent desire to feel effective in interacting with the environment (Ryan & Deci, 2000) |
| 5. | Relatedness (RE) | Individuals’ inherent propensity to feel connected to others, that is, to be a member of a group, to love and care and be loved and cared for (Ryan & Deci, 2000) |
| 6. | Autonomy (AU) | Individuals’ inherent desire to feel volitional and to experience a sense of choice and psychological freedom when carrying out an activity (Ryan & Deci, 2000) |
| 7. | Optimism (OP) | Individual’s tendency to believe that technology would bring good results in life and business (Parasuraman, 2000) |
| 8. | Innovativeness (IN) | Individual's lead about technological products. The person with high innovativeness is happy to try experiments with new technology (Parasuraman, 2000) |
| 9. | Discomfort (DC) | Represents consumer's anxiety in technical terms. the person who discomfort think that the system was not suitable for them (Parasuraman, 2000) |
| 10. | Insecurity (IS) | Individual's trust to technology security or privacy. The person with high insecurity feels doubt about the capability of new technology to fulfill the complete transaction (Parasuraman, 2000) |
| 11. | Hedonic Motivation (HM) | Fun or pleasure derived from using a technology, and it has been shown to play an important role in determining technology acceptance and use (Venkatesh et al., 2012) |
| 12. | Habit (HB) | The extent to which people tend to perform behaviors automatically because of learning (Venkatesh et al., 2012) |
| 13. | Information Quality (IQ) | The degree of excellence of the information produced by the software or system, which focuses on issues related to the timeliness, accuracy, relevance, and format of the information produced by the system (DeLone & McLean, 2003) |
| 14. | System Quality (SQ) | The degree of excellence of the software or systemand focuses on user interface consistency, ease of use, system response levels, system documentation and quality, ease of maintaining the programming code, and whether the system is free of bugs (DeLone & McLean, 2003) |
| 15. | Service Quality (SR) | Quality of the resulting system whether the user is willing or not and to what extent the system can assist users in generating jobs (DeLone & McLean, 2003) |
| 16. | Perceived usefulness (PU) | The degree to which using a technology will provide benefits to consumers in performing certain activities (Venkatesh et al., 2003, 2012) |
| 17. | Perceived ease of use (PE) | The degree of ease associated with consumers’ use of technology (Venkatesh et al., 2003, 2012) |
| 18. | User Acceptance (UA) | The degree of level acceptance of user (Davis, 1989) |
3.2 Content validity analysis
Based on Table 1, we conducted judgment process based on expert opinion, involving asking several experts to evaluate the validity of individual items and the whole instrument. As a whole, this process could help researchers retain the best items are believed to adequately measure a desired content domain. In this study, we had tested content validity by calculating the coefficient of validity (V) and homogeneity reliability (H) proposed by Aiken (1980, 1985) based on data tabulation of expert judgment (Aiken, 1980, 1985). The degree of agreement among the experts regarding the importance of the item was quantified into one coefficient V. After testing the significance, the items served as indicators for determining whether or not the items developed possessed content validity. In addition, the degree of consistency of the items evaluated was quantified into coefficient H, which also served as a reliability indicator after testing the significance of the items in order to test the reliability of the scale content and whether or not the expert opinions were consistent.
According to Aiken (1980, 1985), the significance of items could be known by checking the minimum value of validity coefficient (V) and homogeneity reliability (H) based on Aiken Table (Aiken, 1980, 1985). If an item reach the significant standard based on table, it means the item already is valid and reliable. Based on Aiken table, the value of V and H depend on number of experts (raters) and number of rating categories. Thus, minimum coefficient value of V that is considered significant is 0.70 and minimum coefficient value of H that is considered significant is 0.51 for ten (10) experts (raters) and five (5) rating categories (likert) based on Aiken Table. The result of content validity (V) and homogeneity reliability (H) are presented in Table 2 as follows:
Table 2 Content Validity of the E-participation Instrument.
| No | Item | Vj | Hj | Expert Feedback |
| 1 | SI1 | 0.62 | 0.75 | - |
| 2 | SI2 | 0.75 | 0.70 | - |
| 3 | SI3 | 0.70 | 0.72 | - |
| 4 | SI4 | 0.80 | 0.68 | - |
| 5 | SI5 | 0.77 | 0.69 | - |
| 6 | FC1 | 0.85 | 0.66 | - |
| 7 | FC2 | 0.80 | 0.68 | - |
| 8 | FC3 | 0.75 | 0.70 | - |
| 9 | FC4 | 0.82 | 0.67 | - |
| 10 | FC5 | 0.72 | 0.71 | - |
| 11 | FC6 | 0.60 | 0.76 | Explain clearly “Help for what”? E-Lapor system? |
| 12 | TR1 | 0.87 | 0.65 | - |
| 13 | TR2 | 0.80 | 0.68 | - |
| 14 | TR3 | 0.82 | 0.67 | - |
| 15 | TR4 | 0.80 | 0.68 | - |
| 16 | CP1 | 0.80 | 0.68 | - |
| 17 | CP2 | 0.87 | 0.65 | - |
| 18 | CP3 | 0.87 | 0.65 | - |
| 19 | CP4 | 0.77 | 0.69 | Add in the last sentence “in using e-Lapor” |
| 20 | RE1 | 0.70 | 0.72 | - |
| 21 | RE2 | 0.70 | 0.72 | - |
| 22 | RE3 | 0.75 | 0.70 | - |
| 23 | RE4 | 0.80 | 0.68 | - |
| 24 | AU1 | 0.65 | 0.74 | - |
| 25 | AU2 | 0.75 | 0.70 | - |
| 26 | AU3 | 0.75 | 0.70 | - |
| 27 | OP1 | 0.80 | 0.68 | - |
| 28 | OP2 | 0.75 | 0.70 | - |
| 29 | OP3 | 0.82 | 0.67 | - |
| 30 | IN1 | 0.57 | 0.77 | - |
| 31 | IN2 | 0.62 | 0.75 | Change “circle of friends” with “community” or “society” |
| 32 | IN3 | 0.80 | 0.68 | - |
| 33 | IN4 | 0.72 | 0.71 | - |
| 34 | DC1 | 075 | 0.77 | Change “Technical Support” with “Officer” |
| 35 | DC2 | 0.70 | 0.80 | - |
| 36 | DC3 | 0.77 | 0.82 | - |
| 37 | IS1 | 0.45 | 0.82 | - |
| 38 | IS2 | 0.47 | 0.81 | - |
| 39 | IS3 | 0.55 | 0.78 | - |
| 40 | IS4 | 0.50 | 0.80 | - |
| 41 | HM1 | 0.77 | 0.69 | - |
| 42 | HM2 | 0.77 | 0.69 | - |
| 43 | HB1 | 0.67 | 0.73 | - |
| 44 | HB2 | 0.72 | 0.71 | - |
| 45 | IQ1 | 0.80 | 0.68 | - |
| 46 | IQ2 | 0.77 | 0.69 | - |
| 47 | IQ3 | 0.85 | 0.66 | - |
| 48 | IQ4 | 0.80 | 0.68 | - |
| 49 | IQ5 | 0.85 | 0.66 | - |
| 50 | IQ6 | 0.77 | 0.69 | - |
| 51 | IQ7 | 0.80 | 0.68 | - |
| 52 | SQ1 | 0.70 | 0.72 | - |
| 53 | SQ2 | 0.70 | 0.72 | - |
| 54 | SQ3 | 0.82 | 0.67 | - |
| 55 | SQ4 | 0.67 | 0.73 | - |
| 56 | SQ5 | 0.82 | 0.67 | - |
| 57 | SQ6 | 0.80 | 0.68 | - |
| 58 | SR1 | 0.70 | 0.72 | - |
| 59 | SR2 | 0.72 | 0.71 | - |
| 60 | SR3 | 0.77 | 0.69 | - |
| 61 | PU1 | 0.77 | 0.69 | - |
| 62 | PU2 | 0.72 | 0.71 | - |
| 63 | PU3 | 0.82 | 0.67 | - |
| 64 | PU4 | 0.75 | 0.70 | - |
| 65 | PU5 | 0.77 | 0.69 | - |
| 66 | PU6 | 0.72 | 0.71 | - |
| 67 | PE1 | 0.87 | 0.65 | - |
| 68 | PE2 | 0.85 | 0.66 | - |
| 69 | PE3 | 0.80 | 0.68 | - |
| 70 | PE4 | 0.85 | 0.66 | - |
| 71 | PE5 | 0.85 | 0.66 | - |
| 72 | UA1 | 0.75 | 0.70 | - |
| 73 | UA2 | 0.75 | 0.70 | - |
| 74 | UA3 | 0.75 | 0.70 | - |
| 75 | UA4 | 0.80 | 0.68 | - |
Based on Table 2, the descriptive statistic calculation of content validity (V) and homogeneity reliability (H) are conducted in this study. Thus, the quantitative values for every individual items of instrument are obtained and reviewed whether or not the items developed possessed content validity and homogeneity reliability. Table 2 also indicated the qualitative comments from experts according to items. Every expert were asked to give valuable comment of every items proposed and could add a new item or construct based on expert opinion. However, no new constructs added by experts in this survey.
As shown in Table 2, the items that do not reach minimum significant value are marked in bold yellow. Thus, the result of content validity (V) showed there are total 11 items that has not to be greater than 0.70: SI1, FC6, AU1, IN1, IN2, IS1-IS4, HB1 and SQ4. Therefore, those items should be deleted to reach significant standard. Thus, they are remained only 64 items developed are valid and coefficient of V has value range between 0.70 and 0.97 indicated the instrument proposed has good content validity. The calculation result of homogeneity reliability (H) after item deleted indicated the expert opinions were consistent because H coefficient has range value between 0.65 and 0.75 (greater than 0.51).
3.3 Discussion
The objective of this study is to test the content validity of an user acceptance instrument being developed in the content domain of e-participation, especially the E-Lapor application. When developing a new instrument and also when putting on existing scales to examine a new object, it is suggested by The IS/IT literature to conduct content validity (Straub & Gefen, 2004). IS/IT researchers have emphasized the importance of content validation of instrument but apparently rarely reported in previous IS/IT research (MacKenzie, Podsakoff, & Podsakoff, 2011; Straub & Gefen, 2004; Boudreau, Gefen, & Straub, 2001) indicated that only about 23% of the paper the sampled examined content validity based on their research of instrument-validation practices in IS/IT field (Boudreau, et al., 2001). Thus, IS/IT literature lack of content validity assessment, more over in e-participation studies. In line with previous researches in the context of e-participation we found that their survey did not report any procedures or results of content validity assessment (Ju et al., 2019; Naranjo-Zolotov et al., 2019; Pirannejad et al., 2019). According to Lynn (1986) , it is critical to investigate content validity prior to examining other types of validity to ensure the construct are measured accurately especially when they conduct a survey (Lynn, 1986). Therefore, for research to result in a valid understanding, it is crucial that the instruments used are content valid. Thus, this study contributed to add literature on content validity research by providing an opportunity for more IS/IT researchers to assess the validity assessment.
Based on the result of content validity assessment, we conducted expert judgment using quantitative and qualitative approaches. For quantitative review, we asked the panel of experts to evaluate the importance of items in the domain as described in operational definition and calculating the coefficient of validity (V) and the coefficient of reliability (H) based on their formulas. We also invited the panel to review the items by providing comments or feedbacks regarding the items in the space provided for qualitative assessment. However, many researchers suggested to use a quantitative approach to testing content validity in MIS field (Haynes, Richard, & Kubany, 1995; MacKenzie et al., 2011), since the quantitative way is more systematic, structured and not difficult to reproduce. According to Nor’ashikin Ali, Tretiakov, and Whiddett (2014), since qualitative way frequently involve a a large number of items, thus makes it difficult to interpet and obtain the result rather than quantitative one. (Ali et al., 2014). Meanwhile, Allahyari, Rangi, Khosravi, and Zayeri (2009) reported that choosing quantitative analysis for content validity assessment is better way (Allahyari et al., 2009). In this study, we used both quantitative and qualitative assessment simultaneously as (Ali et al., 2014) suggested.
Table 2 indicated statistical analysis that informed final decisions about whether or not to retain the items based on their coefficient value. The significant findings showed that items deleted (IS1, IS2, IS3 and IS4) which are related to the the insecurity domain in the e-participation context. Insecurity tells about individual's trust to technology security or privacy especially people’s doubt and feeling insecure about the capability of technology to complete transactions (Parasuraman, 2000). Insecurity is not relevant in the context of e-participation according to the result. The e-Lapor version 3.0 already has new features such as anonymous and confidentiality as shown in figure 1. These are an optional use for public to use them. In normal use, the identity and report of citizens will be published openly that anybody could see it (Kantor Staf Presiden, 2019).
Anonymous feature is available for reporters to keep their identities confidential, while confidential feature can be used to restrict access to reports only to reporters and reported institutions so that personal reports cannot be seen by the public. Both of these features could be used for reporting sensitive and very private issues. Thus, the public or citizen did not feel insecure to report their problems, aspirations or any information through e-Lapor.
3. Conclusions
This study has successfully demonstrated a quantitative analysis in establishing content validity assessment for developing the proposed instrument. A user acceptance instrument in the e-participation context was developed based on reviewing previous literature and its content was validated through a judgment process by involving 10 SMEs (subject matter experts). The study contributes to the literature on user acceptance factors in the e-participation context because studies on evaluating citizen participation in Indonesia are still very limited, thus the research could fill that gap in the literature. Furthermore, this study is the first research to build a scale that measures user acceptance and the adoption for public participation evaluation in Indonesia through E-Lapor. The study also contributes to providing an opportunity for more IS/IT researchers to consider a quantitative approach for their content validity assessment in their instrument development process.
