Customer Satisfaction in E-Commerce

In Proceedings of the 17th IEE UK Teletraffic Symposium, Dublin, Ireland, May 16-18, 2001 QUANTIFYING node SATISFACTION WITH E-COMMERCE WEBSITES Hubert Graja and Jennifer McManis1 Abstract E-commerce is an progressively solid part of the global economy. Users of E-commerce sack up settles often have high expectations for the lineament of service, and if those expectations are non met, the succeeding(a) pose is only a click international. A tote up of instruction execution problems have been observed for E-commerce web sets, and much take has g integrity into characterising the mathematical process of wind vane servers and light up income applications.However, the guests of E-commerce network land aims are slight well studied. In this work, we discuss a look of assessing felicity for different guest types with a wind vane localise according to assorted different parameters. Individual measures may be scurfy for simple comparison, and combined to giv e an over both cheer rating. This methodology is applied to collar Irish E-Commerce net come outs. 1) Introduction The beingness Wide meshwork is one of the most most-valuable Internet function, and has been largely responsible for the phenomenal growth of the Internet in recent years.An increasingly popular and serious weather vane-based activity is ECommerce, in which various types of financial proceedings are carried out or facilitated using the meshwork. It is widely expected that E-Commerce activity ordain continue to grow and that it will be a signifi potfult component of the global economy in the near future. A number of dischargeance problems in E-Commerce systems have been observed, mainly due to heavier-thananticipated loads and the nonessential inability to satisfy client requirements. This has resulted in a lot of work attempting to characterise the performance of nett servers and Internet applications e. . 1? 4. However the guests of these E-Commerce systems are less well studied. any(prenominal) surveys show considerable dis delight with current E-Commerce and wind vane servers for example, it has been describe that as galore(postnominal) as 60% of ingestionrs typi chattery bednot prevail the knowledge they are looking for in a Web invest, point though the knowledge is present 5. In an area such as ECommerce, guests pick up a high quality of the service they receive, since it is easy to move away to another turn up if they perceive the current one to be unsatis agenty. An important come to the fore in designing E-Commerce systems is to characterise the ustomers requirements for satisfactory service. Parameters which affect a nodes pleasure with an E-Commerce system allow in the response time, number of clicks needed to find what they exigency, amount of information they are required to give, and predictability of the service received. This track downs to the idea of guest clanification, where nodes in t he same class would value parameters in a alike fashion. client classification may be performed either based on how they judge their delight with an E-Commerce system, or on some(a) other way (e. . large/medium/sm exclusively budget type/speed of Internet connection the customer has to the server frequent/previous/new customer). here we briefly present a methodology for measuring the satisfaction of customer classes. This methodology is applied to a test case consisting of three Irish E-Commerce Web sites in the telecommunications sector. We are able to demonstrate different levels of customer satisfaction among the Web sites, and as well different levels of satisfaction with various parameters for for to each one one individual Web site. 2) MethodologyIn our methodology, we identify customer classes reflecting groups of customers with different conductal characteristics, and Web site parameters relating to features of the Web site which will potentially affect customer satisf action. We whence seek to measure customer satisfaction with the various parameters in a consistent and quantifiable way. This methodology is summarised below a more detailed discussion of the methodology may be found in 6. 2. 1) Customer yrification Customers may be classified in various ways, such as their behaviour or according to how they measure satisfaction with a Web site.However this classification is made, a representation of the customer class must then be made. This representation has twain components first, customer behaviour and second, customer satisfaction measures 1 Performance Engineering Laboratory http//www. eeng. dcu. ie/pel School of Electronic Engineering, Dublin City University, Dublin 9, Ireland email&clxprotected dcu. ie, email&160protected dcu. ie for various Web site parameters. We define customer behaviour in terms of the interaction with the Web site. A shade behaviour is outlined as the series of clicks and other information that the customer excha nges with the site.Typically, behaviour for a customer class is outlined as one or more traces. For a customer class, a weighting may be classd with the traces indicating how likely it is for the customer to perform that particular trace behaviour. That is, some behaviour may be divulgeed more frequently by a drug user in a class, and this behaviour should be granted high weighting. 2. 2) Customer gaiety Measures The factors which big business organizationman affect customer satisfaction with a Web site are contained in a parameter list.It is important that for each parameter in the list satisfaction should be quantifiable. Some quantification measures are easily defined. For instance, if the parameter is the number of clicks, the quantification may be defined as an integer value. Other parameters may have more subjective quantifications. For instance, how does one quantify the quality of information available at a Web site? In order to compare the satisfaction metrical fo r different parameters, the quantifications must be mapped to a fixed scale. For instance, all measures could be mapped to a scale of 0 to 10.This use is what allows us to represent customer valuation of the same parameters. For instance, some customers will tolerate delay founder than others. This may lead to one customer mapping a transfer time of 5 seconds to 10 and another mapping a transfer time of 5 seconds to 0. Studies such as 7 indicate that this mapping trick be complex and context babelike. 2. 3) Analysis of Customer joy for a Web website Using the above, for each trace it is possible to associate a satisfaction value with every parameter.The trace weightings may then be used to arrive at a leaden add up of the satisfaction determine associated with the parameters. This gives a measure of how satisfied a disposed class of customers is with a given parameter. Finally a weighting of parameters can be defined, allowing for an boilers suit satisfaction measure of a class for the Web site. By varying this weighting, we can select how different parameters affect customer satisfaction. 3) Test Results The most difficult part of this exercise is in relating customer trace behaviour to the satisfaction vector. How parameter satisfaction is careful nd how it is mapped onto a fixed scale must be addressed on a case-by-case basis, although experience using the methodology may lead to the definition of some standard cases. Also, since multiple executions of the same trace may lead to different values, some statistical analysis may be required. We have applied our methodology to three Irish E-Commerce Web sites in the telecommunications sector (designated here as Web sites A, B, and C). 3. 1) Customer Classification Customers for the three Web sites we examined have been divided into two distinct classes cliquish and phone line. pull ins are associated with searching for specific information that the customers might be interested in. sestet custom er tasks are identified in tabularize 1 and for each Web site a trace is devised to perform the task. For the sake of convenience, we call all traces associated with a given task by the same name, even soing though the trace is obviously specific to the Web site. Data services is split into T4a and T4b because Web site B provided different pages depending on whether the customer was private or business.Trace T1 T2 T3 T4a T4b T5 T6 Task Where to buy a phone reporting Tariffs WAP Data Services Data Services for duty Roaming List Business Tariffs Table 1 Tasks The semiprivate and Business customer classes are defined as a collection of the above tasks, and an associated weighting is given which is declarative of the relative likelihood of customers of a given class seeking to perform that task. Trace weightings for the Private and Business classes are given in Table 2. The interpretation is that for a group of Private users roughly half might want to know where to buy a phone, 3 0% might want to know about tariffs, 10% ight want to know about insurance coverage and 10% might want to know about WAP services. The Business users exhibit different behaviour with 30% wanting to know about coverage, 30% being interested in the roaming list, 20% being interested in data services and 20% being interested in business tariffs. Customer Class Private Trace T1 T2 T3 T4a T2 T4a, T4b T5 T6 Trace Weighting 0. 5 0. 1 0. 3 0. 1 0. 3 0. 2 0. 3 0. 2 Business Table 2 Trace weightings for different customer classes 3. 2) Satisfaction Measures Three parameters were identified complexity, succession, and step.Complexity was measured as the number of clicks to reach the destination. quantify was measures as total download time in seconds. persona was a subjective measure of the quality of the information contained in the site (could the information be found, and how easy was it to find? ). select was measured using a small-scale user survey where the users were asked to exa mine the end page for each task and rate their satisfaction with the information they found in that respect on a scale of 0-100%. A scale of 0-10 (with 0 being blister and 10 best) was chosen for a uniform comparison of satisfaction values.The measured satisfaction values were mapped onto the 0-10 scale as follows Complexity beat timber 10(20-(n-1)/10), where n is the number of clicks 10(10-t/60), where t is the trace download time in seconds x/10, where x is the average value of user satisfaction with the quality of the page For Quality a straightforward linear mapping was applied. to a greater extent complex mappings were employed for Complexity and Time, and are shown in Figure 1. Examining the Time mapping we see that 60 seconds is regarded as an unacceptable download time, and even 30 seconds leads to a fairly poor rating.Similarly, for Complexity, 10 clicks is regarded as unacceptable, and even 5 clicks is fairly poor. Note that we have chosen one among many possible mappi ngs. It is up to the tester to decide how to choose a mapping that best reflects customer preferences. Also note that, in this case, all customers use the same mappings, and thus are seen to perceive the parameters in a standardised fashion. It is an easy extension to attach different scale mappings to different customer classes or to different traces. Figure 1 mapping time and complexity measures to a 0-10 scale 3. ) Satisfaction Measurement for Web Sites Once the satisfaction measures are determined, it remains to test the Web sites and compare results. Data was gathered using the Web Performance Trainer 2. 1 tool 8 to black market each of the traces on the Web site in question. This was necessary wholly to take time data, and was carried out on a weekday. The other two satisfaction values can be determined by an tryout of the Web sites. Tables 3, 4, and 5 summarise the satisfaction measures for the three Web sites respectively. Web Site A Customer Class Trace Complexity nat uralSatisfaction Measures Time raw 37. 6 34. 0 34. 7 28. 6 34. 7 46. 9 28. 6 38. 7 scaled 2. 4 2. 7 2. 6 3. 3 2. 6 2. 6 1. 7 3. 3 2. 3 2. 4 Quality raw 80 72 67 68 61 69 66 64 scaled 8. 0 7. 2 6. 7 6. 8 7. 5 6. 1 6. 9 6. 6 6. 4 6. 5 scaled 4. 1 3. 0 4. 1 4. 1 3. 8 4. 1 3. 0 4. 1 4. 1 3. 8 Private Business T1 T3 T2 T4a weighted avg. T2 T5 T4a T6 weighted avg. 4 5 4 4 4 5 4 4 Table 3 Customer Satisfaction for Web Site A Web Site B Customer Class Trace Complexity raw scaled 4. 1 7. 4 5. 5 5. 5 5. 4 5. 5 4. 1 4. 1 7. 4 5. 2 Satisfaction Measures Time raw 16. 7 11. 2 17. 1 13. 9 17. 1 14. 39. 7 12. 3 scaled 5. 3 6. 5 5. 2 5. 9 5. 7 5. 2 5. 7 2. 2 6. 2 4. 9 Quality scaled 8. 6 7. 6 7. 6 7. 4 8. 1 7. 3 7. 5 6. 4 7. 6 7. 2 raw 86 76 76 74 73 75 64 76 Private Business T1 T3 T2 T4a weighted avg. T2 T5 T4b T6 weighted avg. 4 2 3 3 3 4 4 2 Table 4 Customer Satisfaction for Web Site B Web Site C Customer Class Trace Satisfaction Measures Complexity Time raw scaled 4. 1 5. 5 7. 4 5. 5 5. 0 7. 4 7 . 4 5. 5 7. 4 7. 0 raw 14. 0 13. 0 11. 1 12. 4 11. 1 10. 2 12. 4 10. 9 scaled 5. 8 6. 1 6. 5 6. 2 6. 0 6. 5 6. 8 6. 2 6. 6 6. 5 Quality scaled 8. 1 6. 8 6. 8 5. 8 7. 4 6. 1 5. 3 6. 5. 3 5. 7 raw 81 68 68 58 61 53 60 53 Private Business T1 T3 T2 T4a weighted avg. T2 T5 T4a T6 weighted avg. 4 3 2 3 2 2 3 2 Table 5 Customer Satisfaction for Web Site C The overall satisfaction measures are summarised in Table 6. Some raise conclusions can be drawn from these measures. Firstly, for all Web sites and all parameters, there was a variation in satisfaction levels between the customer classes. Thus, not all users find the Web sites equally good. This is most noticeable for the Quality parameter Private users rated Quality higher than Business users in all cases.If Business customers are considered valuable, this gap is not desirable. There is also a large difference in satisfaction ratings for the Time parameter of Web site B, again favouring Private customers over Business customers. Second ly, for all users and all measures, there are a range of values across the Web sites. For instance, the Time satisfaction for Business users varies from 6. 5 for Web site C down to 2. 4 for Web site A. This indicates that Web site C might have an edge in attracting Business customers. Finally, for a given user class and Web site, different satisfaction levels are observed.For example, Private users of Web site A have a Time satisfaction value of 2. 6 and a Quality satisfaction value of 7. 5. The accurate interpretation of this is difficult, since the different parameter satisfaction values are dependent on the mapping of the raw data, which of necessity, differs for each parameter. However, it does perhaps indicate a favouring of form over efficiency. Customer Class Satisfaction Customer Web Site Class Web site A Private Web site B Web site C Web site A Business Web site B Web site C Satisfaction Measures Complexity Time Quality 3. 8 5. 4 5. 0 3. 8 5. 2 7. 0 2. 6 5. 7 6. 2. 4 4. 9 6 . 5 7. 5 8. 1 7. 4 6. 5 7. 2 5. 7 Table 6 Customer Class Satisfaction for Web sites A, B, and C Finally, an overall assessment of customer satisfaction may be found by weighting the various parameters. Table 7 displays the overall satisfaction results under several different weighting schemes Weighting 1 gives all parameters equal weighting Weighting 2 gives Time and Complexity equal weighting and Quality zero weighting Weighting 3 considers Time only (zero weighting for Quality and Complexity). These weightings reflect possible values the tester places on the various parameters.We can see that for all the weightings, Business users have a clear order of preference, ranking Web site C highest, then Web site B, and finally Web site A. The order of preference for Private users varies according to the weighting used, although Web site A is worst under all three weightings. Customer Class Satisfaction Customer Web Site Class Web site A Private Web site B Web site C Web site A Business W eb site B Web site C Satisfaction Measures Weighting 1 Weighting 2 Weighting 3 4. 6 6. 4 6. 1 4. 2 5. 8 6. 4 3. 2 5. 6 5. 5 3. 1 5. 1 6. 8 2. 6 5. 7 6. 0 2. 4 4. 9 6. 5 Table 7 Customer satisfaction with a Web site ) Conclusions manikin customer satisfaction with Web and E-commerce sites is not as well studied as Web server modelling, but ascertain whether and how the customers of these sites are satisfied with their interactions is becoming increasingly important as the Web matures. We have proposed a methodology for estimating how satisfied defined classes of customers are with a Web site. Our approach recognises that customer satisfaction is a complex issue and includes factors which are not easily measured. We have applied our methodology to the study of three Irish E-Commerce Web sites.These sites were chosen for representative purposes only and the results do not necessarily generalise to other Web sites. Choices for the tester include not only what customer categories and w hat Web site parameters to examine, but also how to interpret the measured data such as download time. The tractability of the methodology means that it will be necessary for the tester to carefully consider all of their options. The next step is to investigate whether generic categories of users can be defined, and/or whether they care about generic Web site parameters (e. . it seems download time will always be a factor in user satisfaction). Given a specific Web site, we will explore methods for mapping these generic user types and satisfaction parameters into the sites content. If an analysis of the resulting satisfaction measures shows that there is a disparity in the satisfaction of different user types, we will study how the Web site designer or decision maker should take this into account, and whether their reaction can be determined dynamically mend the user is interacting with the site.References 1. 2. 3. 4. 5. 6. 7. 8. Nakamura et al, ENMA the WWW Server Performance M easurement System via big bucks Monitoring, INET99. Cottrell et al, Tutorial on Internet Monitoring and PingER at SLAC available from http//www. slac. stanford. edu/comp/net/wan-mon/tutorial. html Kalidindi and Zekauskas, Surveyor An Infrastructure for Internet Performance Measurements, INET99. Hava and Murphy, Performance Measurement of military personnel Wide Web Servers Proc. f 16th UK Teletraffic Symposium, May 2000. http//www. ecai. ie/usability_online. htm Graja and McManis, Modelling User Interactions with E-Commerce Services, to be presented at ICN01, Colmar, France, July 2001. Bouch, Kuchinsky, and Bhatti, Quality is in the affection of the Beholder Meeting Users Requirements for Internet Quality of Service, HP proficient report HPL-2000-4, http//www. hpl. hp. com/techreports/2000/HPL-2000-4. html Web Performance Incorporated, http//www. Webperfcenter. com