Figure 1. Screenshot of the revision guide, showing a matched environment for a visual learner
The evolution of web sites from static information repositories to complex dynamic applications delivered via web browser has occurred at an unprecedented rate over the last few years. Web applications are now ubiquitous, affecting numerous aspects of Internet use, such as shopping, email, distance learning, online gaming and communication via wikis and discussion boards.
With this widespread use of web applications, it is no wonder that Internet users may feel as though they are lost amongst the masses, with little attention paid to individual users or their preferences and requirements. Any form of personalisation that suggests the user is being recognised and welcomed as an individual is a powerful tool, and one that is growing increasingly common amongst web applications [41]. It suggests to the consumer that they are important and that they matter to the company in question; that their individual desires will be catered for. In short, it is a magnet for drawing in and retaining consumers who have no shortage of choice, and it is a vital weapon for web-based customer services [36]. Nor is this phenomenon only of corporate interest: the UK government's e-strategy "Harnessing Technology: Transforming learning and children's services" [17] describes how personalisation should be used in all areas of education through interactive and digital technologies in the next few years. With an ever-increasing interest in personalisation for web applications, it is of utmost importance that we start to consider just how this can be achieved.
This paper examines how cognitive styles
can be used in web applications in order to create
individualised experiences for the user. Cognitive styles have
already been used in web applications created by large
corporations; for example, there is a digital learning tool
created by EDS [39]. We introduce a case study of an
educational application, written in XML and XSLT with a
PHP/mySQL front-end, which utilises a visual-verbal cognitive
style for content adaptation. We also report on the results
gleaned from user trials, that suggest that cognitive styles
may not be as useful as first thought.
Whilst most web sites can be modelled according to either informational or navigational concepts [4], the idea of cognitive styles can be used to inform either of these concepts. Cognitive style is a psychological construct relating to how individuals process information. It can include categories such as field-dependence/independence and reflexivity/impulsivity [45].
Information in web applications can be represented in a number of different ways, such as static or dynamic text, images, animations, video and audio. Likewise this information can be structured in varying manners, such as sequential or hyperlinked data; specific details or summaries; or a hybrid of these architectures. Both these overarching concepts can be modelled and presented differently according to a user's cognitive style.
In an educational web application, the use
of cognitive style is often referred to as 'learning style'.
Keefe [30] states that learning styles are "characteristic
cognitive, affective and psychological behaviours that serve as
relatively stable indicators of how learners perceive, interact
with and respond to the learning environment". There are many
published variants on this definition, and there is also some
debate on whether learning styles are contextual or not. For
the purposes of this paper, we intend to use Keefe's definition
and are assuming that learning styles are non-contextual.
Adaptive web-based
educational systems (AWBES) are a fast-growing area of research
and development, with applications in areas such as e-commerce
and education. AWBES research seeks to address the issue of
users being overwhelmed by the massive amount of information
and links that is commonly experienced within a hypermedia
system. The goal of AWBES is to personalise the experience for
the user. In order to do this, the system must construct user
models. These require data about the user, which can be
gathered by a number of methods, either explicitly from the
user (via forms/questionnaires, etc.) or implicitly by the
system (by monitoring the user's actions, etc.) [8].
Models of web users are often based on various different characteristics. Brusilovsky defines these features as a "users' goals, knowledge, background, hyperspace experience and preferences" [8].
Users' goals relate to what the user is aiming to achieve, such as information retrieval or a problem-based learning activity. A user's knowledge is defined as their existing knowledge within a specific domain, and many AWBES (such as AHA! [16], WHURLE-HM [50], and MOT [14]) use this criterion to perform adaptation, within educational settings.
Background is a much broader distinction than knowledge, and includes any relevant experiences outside of the specific knowledge domain, whilst experience is a measure of familiarity with hypertext structure and navigation.
Lastly, user preferences may be profiled
in order to provide explicit criteria for adaptation. These
preferences have to be expressed by the user rather than by the
system, and are most heavily used by information retrieval
systems.
Many hypermedia systems are employed for educational purposes, especially with the advent of distance and distributed learning [15]. One of the main tenets of education is that students are different from one other, and these differences may have an impact on how they learn. This strongly suggests that any material used for pedagogical purposes should be changeable or adaptive, in order to cater for these differences. There are several AWBES developed for educational purposes, such as AHA! [16], CHEOPS [23], WHURLE [33], and Interbook [9]. User models in these systems are based largely on prior knowledge, and adaptation occurs at both the content and link level.
New methods of adaptation are being trialled in an attempt to
increase the sophistication and pedagogical validity of these
user models. Many researchers are investigating how learning
style theory may be used to create the user profile and
subsequent matched content. Early trials indicated that this
profiling may be more beneficial to learners than models that
are based simply on domain knowledge [3, 5, 11].
Several web-based learning environments
have been created, that employ learning styles as a means of
personalisation. The table below summarises these systems,
together with the learning style preferences that they utilise
and the research upon which they are based:
Table 1. AWBES that incorporate learning styles
| System: | Learning style preferences used: |
Based on research from: |
|---|---|---|
| AES-CS [46] | Field dependence (FD) and field independence (FI) | Witkin and Goodenough [48] |
| Arthur [25] | Audio, visual, tactile and text |
Sarasin [40] |
| ILASH [2] | Summarising, questioning | Hsiao [29] |
| iWeaver [49] | Global, analytical, impulsive, reflective, visual, auditory, kinaesthetic | Dunn and Dunn [18] |
| MANIC [44] | Abstract, concrete, graphic, text |
Stern et al [43] |
| MOT [42] | Diverger, converger | Kolb [31] |
| AHA! 42] | Activists, pragmatists, reflectors, theorists | Honey and Mumford [28] |
| INSPIRE [26] | Reflector, activist | |
| CS-383 [11] | Global, sequential, sensing, intuitive, visual, verbal, active, reflective | Felder and Silverman1 [20] |
| LSAS [3] | Global, sequential | |
| Tangow [35] | Sensing, intuitive |
It is clear from Table 1 that a multitude of web systems exists, that represent a diverse assortment of learning styles, some based on work by psychologists such as Witkin, Dunn and Dunn, Kolb, and Honey and Mumford. It seems that, provided a psychological learning theory can be modelled computationally, there are always researchers working in the field of computer science who will attempt to incorporate it into an AWBES. This is not necessarily a good thing, as discussed later in this paper.
The variety of learning style preferences that have been modelled can be somewhat overwhelming for the uninitiated, although many of the learning style models incorporate similar dimensions [12, 37]. Several theories include visual/verbal, or imager/verbaliser classifications (such as Sarasin [40]; Dunn and Dunn [18]; Stern and Woolf [43]; and Felder and Silverman [20]). These classifications relate to how individuals prefer to process information that they are engaged in learning: visual learners/imagers favour graphically-represented data whilst verbal learners tend to choose textual information (either as audio or written text) [37]. Another commonly-used dimension is how information is organised for presentation to the learner. Global (or wholist/holist) information is arranged so the student can gain an overview of the subject before studying the finer detail, to gain a broad conceptual view. Conversely, sequential (or analytic/serialist) information emphasises a structured, linear approach to learning, with the student looking at one topic at a time [38].
Field dependent learners are those students who rely on an external frame of reference in order to make sense of their surroundings, i.e. they prefer well-structured, analysed information to be presented to them. Field-independent learners tend to analyse the components of a situation in a way that is separate from its background, and are much better at imposing their own structure on a body of information [27]. Those who are field-independent enjoy using a hypothesis approach to finding new information, whilst the field-dependent student prefers a more passive, observing approach [24]. The summarising and questioning approaches used in ILASH [2] relate strongly to these dimensions.
Reflector/reflective and activist/active/impulsive learning preferences, used in 4 of the systems used in Table 1, are relatively straightforward in their meaning. Reflective students prefer to think through new information and contemplate these ideas, whilst active/impulsive students prefer a more 'hands-on' approach and active experimentation [38].
Convergers and divergers are another classification, where convergers are problem-solvers and pragmatists, who prefer to deal with facts, technical tasks and the application of theories. Divergers tend to be more imaginative and observational, and are good at brainstorming and viewing situations from different perspectives [31]. These correlate well with sensing and intuitive learning preferences, with sensing learners displaying diverging tendencies, and intuitive learners with converging tendencies. Convergers have also resulted in the development of the 'pragmatist' stereotype, coined by Honey and Mumford [28], (after work by Kolb [31],) and the 'theorist' label, reflecting the diverger learning style. Likewise, abstract and concrete are additional labels that can be associated with divergers and convergers respectively.
Lastly, tactile/kinaesthetic
learning styles relate to how students prefer to be able to
manipulate objects to help them learn, such as through the use
of interactive multimedia (for example, drag-and-drop
animations) [49].
Whilst journals abound with research demonstrating how cognitive models may be modelled mathematically in computer systems (especially adaptive ones), there are still some issues that remain unaddressed. The emphasis of most researchers is placed on the pragmatics of such a task: how can you go about creating a quantitative model of what can be fairly complex psychological constructs? Or, once the user model has been created, how can it be used within an existing architecture? Additionally, the field has progressed into investigations such as the authoring AWBES for learning styles, in particular with respect to authoring for either multiple styles or multiple systems, such as the work of Stash et al [42]. Whilst we do not deny that these are important and worthwhile issues to be exploring, there has been a fairly large oversight that has yet to be addressed: just because we can use learning styles in AWBES, does this mean that we should?
There have been several publications in recent years that suggest that matching students to their preferred learning style in an adaptive web-based system does in fact benefit them (for example, see work by Bajraktarevic et al [2, 3], Barker et al [5] and Carver and Hill [10]). However, much of this evidence is, at best, not tested on a very large number of users and, at worst, is merely anecdotal or reporting trends. An influential report published by Coffield et al in 2004 suggests that teaching to accommodate learning styles is fairly ineffective in actually helping students learn [13].
This paper describes how visual-verbal learning styles have been investigated within an existing AWBES, to see if there were any significant differences in academic performance between different experimental groups. The system in question, WHURLE (Web-based Hierarchical Universal Reactive Learning Environment), had previously employed a user model based on the students' prior knowledge [51], however it had never utilised any form of learning style adaptation.
The experimental design for this research
was developed in conjunction with a psychologist and measured
the academic performance of both matched and deliberately
mismatched students.
WHURLE [6, 50] is an XML-based integrated learning environment that is designed to deliver adaptive hypermedia content over the web. It does not, in itself, contain any particular user model; rather it is a framework in which any user model may be implemented. WHURLE is a server-side system, implemented largely in XSLT that is currently rendered into HTML using the Apache Cocoon web development framework (http://cocoon.apache.org).
The information contained in a lesson that learners experience using WHURLE comprises a lesson plan and a number of chunks. Chunks are created by the content author, and it is they that contain the actual academic content. Chunks constitute the atomic units of the WHURLE system, consisting of the smallest unit of content that makes sense as a conceptually self-contained entity. A chunk therefore consists of either a single media item (such as a paragraph of text) or several closely related media items (such as a captioned image) together with all of the meta-information that pertains to that media. Lesson plans are created by the teachers who implement the learning experience (these may or may not be the same people as the content creators). Lesson plans consist of one or more hierarchical page definitions - each page containing either another page, or any number of chunks. Where chunks are specified in a page, the specification also contains metadata that defines circumstances under which the chunk is to be presented as a part of that page. Page definitions are thus a list of potential chunks, some or all of which may be presented to the end-user according to the rules specified in the user model. In this way, the narrative experience is adapted to the needs of the user according to the contents of their user profile [6]. The existence of chunks is completely transparent to the end-user - what students experience is an apparent docuverse that is created by WHURLE.
The mechanics of this are that a node-tree
is generated from the lesson plan using XInclude [32]. This is
initially processed by an adaptation filter - an XSLT
stylesheet that removes any chunks not required according to
the current user model, and the user profile[52] and it is
this filter that implements the user model. The next stage in
the XML pipeline is the display engine - an XSLT stylesheet
that adds a navigational overlay (using information derived
from the position of nodes in the lesson plan), and a user
interface (that is defined in a separate skin). The
implementation of the visual-verbal user model therefore has
required a custom adaptation filter to be written, together
with the concomitant addition of attributes in the lesson plan
and fields in the user profile.
Upon using WHURLE for the first time, users fill in a questionnaire; in WHURLE-HM (Hybrid Model) this rates their prior knowledge within that domain and the resulting information is stored in an SQL database [51]. However, in this new version of WHURLE, the user's learning style is determined, and thus the student completes a version of the Felder-Solomon Inventory of Learning Styles (ILS) questionnaire[2] during their initial registration with the system. The logging-in and registration of the system is controlled via a PHP front-end, whilst the user information is stored in an SQL database. When users access the adaptive part of the system, functionality is carried out via the underlying XML/XSLT architecture. Chunks within WHURLE contain attributes relating to the visual or verbal properties of that chunk. What the user actually sees is adapted content based on the chunks that correspond to their stored data [33], via conditional transclusion of unnecessary chunks.
Thus content is adaptively presented to
suit either visual or verbal learners, according to their
preferred learning style.
In the WHURLE system, learning style was measured according to the ILS assessment tool [19]. In its complete form, this is a 44-item questionnaire, that assesses users on 4 axes: visual/verbal; global/sequential; sensing/intuitive and active/reflective. WHURLE content that was personalised to learning styles differed only in its mode of presentation (i.e. visual or verbal), hence students were only asked to fill in the 11 questions relating to the visual/verbal axis, rather than the entire questionnaire. This shortened version of the questionnaire was filled in when the students first accessed the system; they were subsequently scored on a scale between 11 (highly visual) and ‑11 (highly verbal). Those scoring between 4 and 11 were classed as 'visual' learners, whilst those scoring between ‑4 and ‑11 were classed as 'verbal' learners. Students scoring between 3 and ‑3 were classified as having no preference and thus labelled 'bimodal'.
There were a number of reasons why the Felder-Solomon ILS instrument was chosen for use in this system. There were a number of criteria that needed to be fulfilled by any potential method for modelling learning styles:
Several learning style models were researched. It was felt that the Felder-Solomon tool had already been used successfully by researchers in other AWBES [3, 11] and evaluation data indicated that the instrument shows a good level of validity and reliability [22, 53]. Since it used multiple representations as one of its aspects, these representations could easily be rendered by different media.
Due to these reasons, the authors felt it
would be the best model to use to investigate visual/verbal
personalisation within an AWBES.
User trials were conducted with a number of undergraduate and taught postgraduate students (n=221) studying for a module offered by the School of Computer Science & IT at the University of Nottingham. They were given the opportunity to use a web-based revision guide as part of the teaching support for the module. This revision guide was an adapted system that took into account the students' visual/verbal learning styles and placed them randomly into either a matched, neutral or mismatched group.
Matched students were given content that matched their particular style, whilst mismatched students were given content that was contrary to their learning style (e.g. a visual learner would be given verbal content). Neutral students were given a mix of visual and verbal content, irrespective of their learning style. Screenshots from the system can be seen in Figures 1 and 2, showing examples of differently represented information, that could be matched and mismatched respectively, for a visual learner.
Assessment and tracking of the students was carried out using a variety of methods. Access logs of all the pages in the revision guide were collected (giving information about which pages/units were visited by each student), and also data relating to academic performance. Students' marks for module coursework, exam and overall module mark were collected, together with the mark gained from a Multiple-Choice Quiz that formed part of the revision guide.
Figure 1. Screenshot of the revision guide, showing a matched environment for a visual learner
Figure 2. Screenshot of the same section of the revision guide, but in a mismatched environment for a visual leaner (i.e. a verbal representation)
A thorough analysis of the data was carried out to explore whether there was sufficient evidence to support any of the following hypotheses:
H0 - there will be no difference in the learning experience between matched, neutral and non-matched users
H1 - students who learn in a matched environment will learn significantly better than those who are in mismatched or neutral environment
H2 - students who learn in a mismatched environment will learn significantly worse than those who learn in a matched or neutral environment
H3 - one particular type of learning style is better for students in terms of performance
H4 - one particular type of learning environment is better for students in terms of performance
The
initial hypotheses of H1 and H2 were
expanded somewhat into the next two hypotheses; it was felt
that a thorough investigation should explore the possibility of
a significant effect of either a particular learning style or
particular type of learning environment (i.e. how the material
was presented). We also wanted to explore the effect of
students using the revision guide; this was a resource that had
not previously been provided for this module and hence any data
that could be collected about it would prove to be a useful
formative assessment.
Extensive data was collected over a 2-week period during which students had full, 24-hour access to the web-based revision guide. Using SPSS, statistical analyses of the data were carried out to see whether there were any significant differences relating to each hypothesis. Academic performance was measured by module exam mark and the score achieved by the student if they chose to answer a 20-question Multiple-Choice Quiz used in the revision guide.
Out of 234 students taking the module, 221 decided to log into the system at least once, and consequently had their learning style assessed and recorded. Out of those, 105 were assessed as visual, 105 were bimodal (no preference) and only 11 classed as verbal. This small number of verbal learners compared to visual learners is consistent with findings of other studies carried out with electrical engineering students and those studying for other scientific degree programmes [22]. Although we intended to carry out 3-way group analyses for some hypotheses, due to the extremely small and uneven group sizes, we have had to exclude these verbal users from the statistical analysis. Hence the following results are based on those users who were either visual or bimodal. In addition, we also excluded those students who had not appeared to use the system very much, since they may have contributed some erroneous data to this quasi-experimental situation. 'Use' of the system was defined as those students who had viewed 6 or more separate pages from the system since this was the number of pages in the smallest topic of study.
Each of the following hypotheses is now
studied in turn, with accompanying statistical data.
H1 - students who learn in a matched environment will learn significantly better than those who are in mismatched or neutral environment
This hypothesis examined closely the module exam mark awarded to students who were placed in matched, mismatched or neutral groups. A one-way between-groups multivariate analysis of variance (MANOVA) was carried out to study the significance of any difference between performances of these groups. Two dependent variables were used: module exam mark and revision guide MCQ score. The independent variable was group (match, mismatched or neutral).
The information contained in Tables 2 and 3 shows mean scores from each of the groups for each dependent variable. Statistical analysis indicates that there was no significant difference between the groups: F(4,210)=0.66, p=0.62, Wilks' Lambda=0.98, partial eta squared=0.1.
Thus it can be concluded that matched
students did not perform any better than students from the
mismatched or neutral groups in either the exam performance or
in the revision guide MCQ. Thus there is no evidence to support
hypothesis 1, and it can be rejected.
Table 2. Details of mean exam mark for students in matched, mismatched and neutral groups
| Student type | N | Mean exam mark (%) | Standard deviation |
|---|---|---|---|
| Matched | 101 | 60.06 | 13.588 |
| Mismatched | 24 | 59.33 | 17.591 |
| Neutral | 28 | 62.0 | 10.364 |
| Total: | 153 | 60.3 | 13.715 |
Table 3. Details of mean MCQ mark for students in matched, mismatched and neutral groups
| Student type | N | Mean MCQ mark (%) | Standard deviation |
|---|---|---|---|
| Matched | 74 | 62.3 | 16.201 |
| Mismatched | 14 | 67.5 | 11.561 |
| Neutral | 21 | 66.9 | 16.461 |
| Total: | 109 | 63.85 | 15.239 |
H2 - students who learn in a mismatched environment will learn significantly worse than those who learn in a matched or neutral environment
In this analysis, students were likewise assessed as per the previous hypothesis. This time, mismatched students were compared with students who were not mismatched (i.e. those placed in neutral or matched environment): the scores are presented in Tables 2 and 3 as before.
The academic performances of mismatched students were very
similar to those who were matched or in the neutral
environment. With the same statistical values as for hypothesis
1, it is clear that this hypothesis must likewise be rejected
since there is no evidence to support it.
H3 - one particular type of learning style is better for students in terms of performance
The idea behind this hypothesis was to
explore the main effect of learning style: was there any
benefit to having one particular learning style over another?
If so, predictions could be made that certain student types
would do better than others, regardless of the type of
presentation that they were given.
Table 4. Details of mean exam mark for students categorised visual or bimodal
| Student type | N | Mean exam mark (%) | Standard deviation |
|---|---|---|---|
| Visual | 53 | 60.49 | 14.101 |
| Bimodal | 56 | 61.71 | 13.755 |
| Total | 109 | 61.12 | 13.873 |
Table 5. Details of mean MCQ mark for students categorised visual or bimodal
| Student type | N | Mean MCQ mark (%) | Standard deviation |
|---|---|---|---|
| Visual | 53 | 64.72 | 13.778 |
| Bimodal | 56 | 63.04 | 16.588 |
| Total | 109 | 63.85 | 15.239 |
Analysis was carried out again using a one-way MANOVA. The dependent variables were the same as for the previous hypotheses (i.e. exam score and MCQ score) but the independent variable was learning style (visual vs bimodal). The mean exam and MCQ scores are shown in Tables 4 and 5.
From our analysis, it seems that there was no significant difference between students using different learning styles and their exam or MCQ scores: F(2,106)=0.46, p=0.63, Wilks' Lambda=0.99, partial eta squared=0.01.
We conclude from these results that there
is no evidence to suggest that one learning style is better
than others for in terms of academic performance, and can thus
reject our H3 hypothesis.
H4 - one particular type of learning environment is better for students in terms of performance
Instead of investigating if one learning style might lead to better academic results over another learning style, this hypothesis tested the environment that the student was presented with. We wanted to see if a particular type of multimedia representation resulted in a better academic performance, regardless of the students' learning styles.
Data relating to this hypothesis is shown
in Tables 6 and 7. Independent variables used for this MANOVA
were environment type: visual, neutral or verbal. Dependent
variables were exam score and MCQ score, as used
previously.
Table 6. Details of mean exam mark for students using visual, neutral or verbal environments:
| Environment type | N | Mean exam mark (%) | Standard deviation |
|---|---|---|---|
| Visual | 18 | 57.89 | 13.936 |
| Neutral | 77 | 61.55 | 12.953 |
| Verbal | 14 | 62.93 | 18.574 |
| Total | 109 | 61.12 | 13.873 |
Table 7. Details of mean MCQ mark for students using visual, neutral or verbal environments:
| Environment type | N | Mean MCQ mark (%) | Standard deviation |
|---|---|---|---|
| Visual | 18 | 60.0 | 15.146 |
| Neutral | 77 | 64.09 | 15.806 |
| Verbal | 14 | 67.5 | 11.561 |
| Total | 109 | 63.85 | 15.239 |
Whilst some of the mean scores shown in Tables 6 and 7 suggest that there are differences between groups, upon testing these statistically, there is actually no significance between them: F(4,210)=0.59, p=0.67, Wilks' Lambda=0.98, partial eta squared=0.01.
It can therefore be concluded that the type of multimedia representations made to the students does not have an effect on their academic performance, and the H4 hypothesis can be rejected.
Two further hypotheses were also investigated, relating to the use of the revision guide, although these are not reported on fully in this publication. However, similar conclusions were drawn, namely that there did not seem to be any significant difference between the academic performance of those students who used the revision guide, and those who did not use it (using the same definition of 'use' as stated previously). Nor was there a correlation between the amount of use of the revision guide and a student's academic performance (i.e. students who used the revision guide more did not necessarily get correspondingly better marks). Qualitative feedback from the students suggest that they found it an enjoyable and useful resource and that it helped provide a novel way to revise that gave them a change from the more traditional use of textbooks.
The evidence produced from this study overall indicates support for the H0 hypothesis (that there will be no difference in the learning experience between matched, neutral and non-matched users) and thus the other hypotheses are summarily disproved. However it is important to note that the verbal learner sub-group could not be included for statistical testing, and it cannot be assumed that these conclusions are necessarily true for those particular students.
From the extensive data collected in this user trial, it seems as though the use of a visual-verbal learning style model to provide matched or mismatched content to university students is unlikely to enhance learning in a statistically significant way. It did not seem to matter whether a student was a visual or bimodal learner, nor if they were presented with visual, verbal or mixed representations of data. However, there were many variables present in this study that could not be controlled for, which may have had an impact on the results (such as students' prior academic performances - although there was a positive correlation between performance measured in the study when compared with previous academic performance of the same students). It is also possible that, if there was any significant difference to be found, they were so small so as to be obscured by the coarse-grain measures used to assess academic performance in this study.
This paper has introduced the concept of using cognitive styles as a user model for adaptive web-based educational systems (AWBES), and has also presented the findings of a user trial that investigated their use within an existing AWBES. The conclusions of this study - namely that using matched or mismatched learning materials did not significantly benefit nor disadvantage the students taking part - now throw into doubt the extent of their effectiveness in a learning situation.
There are several possible reasons why these results occurred. It could be that the students used in the study have already been unintentionally pre-selected on the basis of their academic ability: they are all studying in higher education for a degree. This being the case, it is not unreasonable to assume that these students can already learn effectively, even when presented with less-optimal opportunities (i.e. a mismatched environment).
It may be that, since learning styles are not static [47], that assessing the styles once when students first use the system is not sufficient. A variable that is susceptible to change should thus be catered for in a truly adaptive system. This user trial investigated an adapted system that did not support changes in the user model and was thus not as functional as it might have been. The resultant lack of adaptability might have resulted in both the system and the user model being overly simplistic. In addition, the visual and verbal representations may not have been correctly designed; even if they were suitable for use in this system, the ways in which multiple representations affect learning have still not been fully explored [1]. Paivio's dual coding theory states that human cognition deals with visual and verbal processing simultaneously [34], which suggests that AWBES should provide for both of these types of representation if learning is to be effective. Another factor to take into account was the way in which students were assessed, using the ILS model that was simplified down to a single axis. Though necessary in the context of this work, it may not have been appropriate, since that axis is actually a component of the model and not the whole instrument. It is thus possible that students were not assessed as effectively as they might have been.
Of course, it may be that cognitive styles in themselves are not an effective means of personalising the learning experience, although it is possible that they have an effect on attitudes to learning rather than academic performance per se [5]. An extremely comprehensive and influential report published in 2004 is very critical of many learning style models and their instruments. It suggests that any beneficial effect gained through using them to make learning more effective, is only very small [12, 13], if it can be measured at all. Given the large amount of time taken to create and maintain an AWBES and associated multimedia, is this amount of effort justified when the adaptation mechanism seems potentially ineffectual?
However, just because the results of this user trial did not show any benefits to using cognitive styles as means of adaptation in an AWBES, does not imply that they are totally without merit. The preferences studied in this experiment were constrained to visual and verbal perspectives; other models incorporate many other aspects of cognition and behaviour and these have not been investigated to the same extent by the authors. A study by Bajraktarevic et al [3] certainly showed some interesting results when matching and mismatching learners with global or sequential preferences. They demonstrated that matched students achieved significantly higher post-test scores than non-matched. It is possible that global-sequential adaptation is more appropriate for hypermedia systems than visual-verbal adaptation, certainly when taking into account the variance in browsing habits of web users. There is some published evidence to suggest that systems such as these may be effective under certain conditions, but we (and others, such as Brusilovsky [7]) would argue that they have not been tested with enough users, or with sufficient scientific rigour. Likewise there does not seem to be any particular evidence to invalidate this area of research, and any work carried out by others should not be dismissed out of hand; however it does seem that personalisation with a statistically significant benefit in educational systems is a lot harder to create than the authors first envisaged.
Until more evidence is acquired (e.g. from
more extensive user trials), it is difficult to draw firm
conclusions about the efficacy and validity of using cognitive
styles as means of adaptation in adaptive web-based education
systems. The lack of any kind of correlation seen in these user
trials might be a particular characteristic of the study,
although it is possible (maybe even probable) that it is
indicative of a universal pattern. It is also possible that
different results might be achieved with students from younger
ages, who may not have learned how to effectively navigate
visual and verbal learning material, and that is a potentially
fertile area for further investigation. However, we can say
with some confidence that there is currently no positive
evidence of any educational benefit whatsoever in using
cognitive styles as a basis for user modelling. It is important
to remember that just because one can build an adaptive
system based upon cognitive styles does not necessarily mean
that one should.
Many thanks to Dr Shaaron
Ainsworth, James Goulding and members of the Web Technology Lab
at the University of Nottingham for their support and for many
useful discussions. The authors are also grateful to those
students who willingly participated in the user trials. This
research is supported by a PhD scholarship from the School of
Computer Science & IT at the University of Nottingham.
[1] Ainsworth, S.E. DeFT: A conceptual framework for learning with multiple representations. Learning and Instruction, (in press), to appear.
[2] Bajraktarevic, N., Hall, W. and Fullick, P., ILASH: Incorporating Learning Strategies in Hypermedia. Proceedings of the Fourteenth Conference on Hypertext and Hypermedia, (Nottingham, 2003), 2003.
[3] Bajraktarevic, N., Hall, W. and Fullick, P., Incorporating learning styles in hypermedia environment: Empirical evaluation. Proceedings of the Fourteenth Conference on Hypertext and Hypermedia, (Nottingham, 2003), 2003.
[4] Baresi, L., Garzotto, F. and Paolini, P. From Web Sites to Web Applications: New Issues for Conceptual Modeling. Lecture Notes in Computer Science, 1921, 89, 2000.
[5] Barker, T., Jones, S., Britton, C. and Messer, D., The user of the verbaliser-imager cognitive style as a descriptor in a student model of learner characteristics in a multimedia application. Proceedings of the 5th Annual Conference of the European Learning Styles Information Network (ELSIN), (University of Hertfordshire, UK, 2000), 2000.
[6] Brailsford, T., Stewart, C., Zakaria, M. and Moore, A., Autonavigation, links and narrative in an adaptive web-based integrated learning environment. Proceedings of the Eleventh World Wide Web Conference 2002, (Honolulu, Hawaii, USA, 2002), 2002.
[7] Brusilovsky, P. Adaptive hypermedia. User Modeling and User Adapted Interaction. Ten Year Anniversary Issue (Alfred Kobsa, ed.), 11 (1/2), 87-110, 2001.
[8] Brusilovsky, P. Methods and techniques of adaptive hypermedia. User Modeling and User Adapted Interaction, 6 (2-3), 87-129, 1996.
[9] Brusilovsky, P., Eklund, J. and Schwarz, E. Web-based education for all: a tool for developing adaptive courseware. Computer Networks and ISDN Systems, 30 (1-7), 291-300, 1998.
[10] Carver, C.A. and Hill, J.M.D., Adaptive hypermedia systems. Proceedings of the Universal Web Accessibility Symposium 2000 (Conference), (San Antonio, 2000), 2000.
[11] Carver, C.A., Howard, R.A. and Lane, W.D. Addressing different learning styles through course hypermedia. IEEE Transactions on Education, 42 (1), 33-38, 1999.
[12] Coffield, F., Moseley, D., Hall, E. and Ecclestone, K. Learning styles and pedagogy in post-16 learning: A systematic and critical review, Learning & Skills Research Centre, 2004.
[13] Coffield, F., Moseley, D., Hall, E. and Ecclestone, K. Should we be using learning styles? What research has to say to practice, Learning & Skills Research Centre, 2004.
[14] Cristea, A.I. and de Mooij, A., Adaptive Course Authoring: My Online Teacher. Proceedings of ICT'03, (Papeete, French Polynesia, 2003), 2003.
[15] Davies, P., The Virtual School of Biodiversity: Towards a Model for Quality Assured Distributed Learning. Proceedings of the Fifth Hong Kong Web Symposium (4-6 October 1999), (Hong Kong, 1999), 1999.
[16] De Bra, P., Aerts, A., Berden, B., de Lange, B., Rousseau, B., Santic, T., Smits, D. and Stash, N., AHA! The Adaptive Hypermedia Architecture. Proceedings of the Fourteenth ACM Conference on Hypertext and Hypermedia (HT03), (Nottingham, 2003), 2003.
[17] Department for Education and Skills (DfES). Harnessing Technology: Transforming Learning and Children's Services [e-Strategy document], 2005.
[18] Dunn, R. and Dunn, K. Teaching Students Through Their Individual Learning Styles: A Practical Approach. Reston Publishing, Virginia, 1978.
[19] Felder, R.M. Learning styles http://www.ncsu.edu/felder-public/Learning_Styles.html
[20] Felder, R.M. and Silverman, L.K. Learning and teaching styles in engineering education. Engineering Education, 78 (7), 674-681, 1988.
[21] Felder, R.M. and Soloman, B.A. Index of Learning Styles http://www.ncsu.edu/felder-public/ILSpage.html
[22] Felder, R.M. and Spurlin, J. Applications, Reliability, and Validity of the Index of Learning Styles. International Journal of Engineering Education, 21 (1), 103-112, 2005.
[23] Ferrandino, S., Negro, A. and Scarano, V., CHEOPS: adaptive hypermedia on the World Wide Web. Proceedings of the European Workshop on Interactive Distributed Multimedia Systems and Telecommunications Services (IDMS'97), (1997), 1997.
[24] Ford, N., Wilson, T.D., Foster, A., Ellis, D. and Spink, A. Information seeking and mediated searching. Part 4. Cognitive styles in information seeking. Journal of the American Society for Information Science and Technology, 53 (9), 728 - 735, 2002.
[25] Gilbert, J.E. and Han, C.Y. Adapting instruction in search of "a significant difference". Journal of Network and Computer Applications, 22 (3), 149-160, 1999.
[26] Grigoriadou, M., Papanikolaou, K., Kornilakis, H. and Magoulas, G., INSPIRE: An INtelligent System for Personalized Instruction in a Remote Environment. Proceedings of the Eight International Conference on User Modeling (UM2001), (Sonthofen, Germany, 2001), 2001.
[27] Hall, J.K. Field Dependence-Independence and Computer-based Instruction in Geography, Doctoral thesis, Faculty of the Virginia Polytechnic Institute and State University, 2000.
[28] Honey, P. and Mumford, A. The Manual of Learning Styles. Peter Honey Publications, Maidenhead, 1992.
[29] Hsiao, Y.-P. The effects of cognitive styles and learning strategies in a hypermedia environment: a review of literature http://web.archive.org/web/20040514071116/www.edb.utexas.edu/mmresearch/Students97/Hsiao/
[30] Keefe, J.W. Learning style: an overview. In J.W. Keefe, editor, Student learning styles: diagnosing and prescribing programs. NASSP, 1979.
[31] Kolb, D.A. Experiential learning: experience as the source of learning and development. Prentice Hall, New Jersey, 1984.
[32] Marsh, J. and Orchard, D. XML Inclusions (XInclude) Version 1.0. W3C Working Draft 10 http://www.w3.org/TR/xinclude/
[33] Moore, A., Brailsford, T.J. and Stewart, C.D., Personally tailored teaching in WHURLE using conditional transclusion. Proceedings of the Twelfth ACM Conference on Hypertext and Hypermedia, (Aarhus, Denmark, 2001), 2001.
[34] Paivio, A. Mental Representations. Oxford University Press, New York, 1986.
[35] Paredes, P. and Rodriguez, P., A mixed approach to modelling learning styles in adaptive educational hypermedia. Proceedings of the Third IASTED Conference on Web-Based Education (WBE2004), (Innsbruck, Austria, 2004), 2004.
[36] Pramataris, K., Vrechopoulos, A., Mylonopoulos, N., Papmichail, G. and Poylymenakou, A., Personalised Services and Promotions in Internet Retailing. Proceedings of the eBusiness and eWork 2000 Conference,(Madrid, Spain, 2000), 2000.
[37] Riding, R. and Rayner, S. Cognitive Styles and Learning Strategies: Understanding Style Differences in Learning and Behaviour. David Fulton, London, 1998.
[38] Riding, R.J. and Cheema, I. Cognitive Styles - An Overview And Integration. Educational Psychology, 11 (3&4), 193-215, 1991.
[39] Riegelhaupt, B.J. and Peck, P.G. viewpoint: Digital learning for consumer-driven healthcare http://www.eds.com/services/whitepapers/downloads/digital_learning_cdhc.pdf
[40] Sarasin, L.C. Learning Style Perspectives: Impact in the Classroom. Madison, WI: Atwood Publishing, 1998.
[41] Searby, S. Personalisation - An Overview of its Use and Potential. BT Technology Journal, 21 (1), 13-19, 2003.
[42] Stash, N., Cristea, A. and De Bra, P., Authoring of learning styles in adaptive hypermedia: problems and solutions. Proceedings of the Thirteenth World Wide Web Conference 2004, (New York, USA., 2004), 2004.
[43] Stern, M.K., Beck, J.E. and Woolf, B.P., Using a Naive Bayes Classifier for User Modeling. Proceedings of the Conference on User Modeling, (1999), 1999.
[44] Stern, M.K. and Woolf, B.P., Adaptive content in an online lecture system. Proceedings of the International Conference on Adaptive Hypermedia and Adaptive Web-based Systems, (Trento, Italy, 2000), 291-300, 2000.
[45] Summerville, J. Role of awareness of cognitive style in hypermedia. International Journal of Educational Technology, 1 (1), 1999.
[46] Triantafillou, E., Pomportsis, A. and Georgiadou, E., AES-CS: Adaptive Educational System based on Cognitive Styles. Proceedings of AH2002 Workshop, Second International Conference on Adaptive Hypermedia and Adaptive Web-based Systems, (University of Malaga, Spain, 2002), 2002.
[47] Valley, K. Learning styles and courseware design. ALT-J, 5 (2), 42-51, 1997.
[48] Witkin, H.A. and Goodenough, D.R. Cognitive styles - essence and origins: Field dependence and field independence. International Universities, New York, 1981.
[49] Wolf, C., iWeaver: towards "learning style"-based e-learning in computer science education. Proceedings of the Fifth Australasian Computing Education Conference, (Adelaide, Australia, 2003), 273 - 279, 2003.
[50] Zakaria, M.R. and Brailsford, T.J. User modeling and adaptive educational hypermedia frameworks for education. New Review of Hypermedia and Multimedia (NRHM 2002), 8, 83-97, 2002.
[51] Zakaria, M.R., Moore, A., Ashman, H., Stewart, C. and Brailsford, T., The hybrid model for adaptive educational hypermedia. Proceedings of AH2002, (Malaga, Spain., 2002), 2002.
[52] Zakaria, M.R., Moore, A., Stewart, C. and Brailsford, T.J., "Pluggable" user models for adaptive hypermedia in education. Proceedings of the Fourteenth ACM Conference on Hypertext and Hypermedia (HT03), (Nottingham, UK, 2003), 2003.
[53] Zywno, M.S., A contribution to validation of score meaning for Felder-Soloman's Index of Learning Styles. Proceedings of 2003 ASEE Annual Conference and Exposition, (Nashville, Tennessee (June 23-25), 2003), 2003.
[1] There is much confusion over the naming of this model. It is important to note that the theory is based on the work of Felder and Silverman [20], although the instrument itself (the questionnaire) is based on work by Felder and Solomon [21].
[2] such as that available online at http://www.engr.ncsu.edu/learningstyles/ilsweb.html