Performance is a crucial aspect in the quality of service of e-commerce sites. Although many factors may affect performance, in this paper we focus on the role computer servers play in the performance of e-commerce sites. The paper presents an experimental study into how system factors (software and hardware), as well the external authentication service influence the throughput provided by e-commerce servers. The study was carried out using the TPC-W benchmark as a synthetic workload to test the servers. The results obtained from the study highlight certain behavioral aspects of servers which may provide valuable insight useful for the performance studies required to set up e-commerce sites.
Benchmarking, TPC-W, E-commerce Servers.
The TPC-W e-commerce benchmark [1, 2] has client-server architecture. Figure 1 shows the elements which make up the architecture of the benchmark. The SUT (System Under Test) includes all the components that constitute a typical e-commerce server. The RBEs (Remote Browser Emulators) emulate the workload that real customers would generate. The PGE (Payment Gateway Emulator) emulates the entity which authenticates the users and authorizes the payments. The SUT, the PGE and the RBEs communicate through a dedicated network.
In TPC-W the Workload is modeled as a mix of web interactions. Examples of these interactions are search and browse products, add products to the shopping cart, and buy products. Different mixes of web interactions are used to model the behavior of different types of clients. These mixes are referred to as user profiles. In TPC-W three user profiles are defined: Browsing, Ordering and Shopping.
In addition to the user profile, another crucial configuration parameter of TPC-W is the number of items to sell. Five values can be selected: 103, 104, 105, 106 and 107.
The metric reported by TPC-W is the number of Web Interactions Per Second (WIPS) measured using the shopping user profile as the workload.
The experimental work presented in this paper was developed using an Intel Pentium-III tetra-processor running the Linux operating system. The database used was ORACLE and the HTTP server was Apache, connected by the PHP tool. In all the experiments, the benchmark was configured to use the shopping user profile as the workload and 105 items to sell in the database.
The system factors, shown in Table 1, can be classified in two broad groups: hardware factors and software factors. The hardware factors are mainly ordered quantitative factors, whose levels are expressed numerically in increasing order. The hardware factors analyzed are the number of CPUs and the amount of RAM.
The software factors are mainly unordered qualitative factors, mainly associated to the version or release of each software component of the system or regarding their configuration parameters.
| Sys Factor | Intel SC450NX Platform | Levels |
| CPU | Pentium III-Xeon 450 MHz | 1, 2, 3, 4 |
| RAM | EDO ECC 133 MHz | 256 ... 1536 |
| DISK | SCSI Seagate Cheetah 18 GBytes (10,000 rpm 5.4 ms seek time) | -- |
| Op. System | Linux, RedHat V6.2, kernel 2.2.18 | -- |
| Http server | Apache V1.3.14 | -- |
| Data Base | Oracle 8i V8.1.7 dedicated servers | -- |
| CGI | PHP V4.0.4 permanent connections | -- |
To evaluate the influence of hardware system factors on WIPS, a set of experiments with the number of processors varying between 1 and 4 and the memory installed varying between 128 MB and 1,5 GB was carried out. Figure 2 shows the results of the experiments.
When the minimum amount of memory which allows the database to operate is installed (curve of 128 MB), the number of processors used has no influence on WIPS. In other words, under the maximum memory contention conditions, the scalability of the e-commerce server is null.
With a sufficiently high amount of memory (curve of 512 MB), the addition of processors always generates an increment of WIPS, although not in a linear manner. On this curve the server works under minimum or unnoticeable memory contention conditions. With more memory installed (curve of 1536 MB), there is a total absence of memory contention.
The evaluation of the influence of software system factors involves many configuration parameters of the operating system, web server and database software. However, the default installation values for these parameters generally provide nearly optimum performance. Not so with the data base queries. The optimization of the queries is obtained by indexing the database, which is carried out with the following rule: when a field of a table is referenced in one or more SQL clauses, a single index is created using the field. The results presented in Figure 2 were obtained without optimizing the queries. Figure 3 shows the equivalent evolution of the WIPS metric when single indexes were added to the tables of the database: single indexing of tables allows an increment of WIPS of up to 300%.
The authentication service is provided by the Payment Gateway Emulator (PGE). To evaluate if the PGE represents a brake in performance, WIPS were measured with and without PGE under two optimization levels of the database, indexed and not indexed. The results are shown in Figure 4.
When the database is not indexed, the WIPS, with and without the PGE, are the same. They are represented by the two nearly identical curves in the lower part of Figure 4. However, when the database is indexed, the authentication services produce a reduction of 10% in the WIPS.
This work was supported by the Spanish Research, Development, and Innovation Program under project TIC2001-1374-C03-03.