Final Report of the Working Group on
General Survey Questionnaires and Nomination Methods Concerning Bullying

Contents

1. Introduction
2. General Survey Questionnaires
   1. What are the contents of general survey questionnaires?
   2. Do they have any definition of the problem? What criteria are used?
   3. How many questions do they have?
   4. What is the age range of these instruments?
   5. Conclusions about the use of general survey questionnaires
3. Nomination Methods
   1. What are the contents of the nomination methods?
   2. How many items do they have?
   3. What is the age range of these instruments?
   4. Conclusions about the use of nomination methods
4. Proposal of core questions for a general survey questionnaire
5. The pilot study
6. Comments after piloting
7. Appendix
8. References

1. Introduction

The "Nature and Prevention of Bullying" project has among its objectives the description and explanation of bullying, and the proposal of models of intervention to eliminate this problem as a habitual behaviour in peer groups. To this end, it has been necessary to work on the definition of the phenomenon, trying to collect and analyse the different words and meanings that this problem can have in different cultures. Also, it has been necessary to order the different methodological traditions that have been used by researchers and educators in the study of bullying.

This working group has reviewed the main general survey questionnaires developed so far. Some previous reports have been presented and discussed in the meetings held in Florence (November, 1998) and Braga (May, 1999), and the result of those discussions is the final report that we present now.

The group has also reviewed the nomination methods, procedures which have been considered complementary to the questionnaires because they allow us to identify the involved students. If anything has become clear after this analysis it is that the individual detection of students involved in bullying must be a carefully completed process where, depending on the objectives to be covered, we will have to select the most suitable instruments.

With these aims, this report has been organised into three main sections. In the first section, we tackle the general survey questionnaires, paying special attention to the contents that are included in the different instruments, as well as the definitions which appear inside them. We also analysed the formal aspects: age range and number of questions. In the second section, we studied the nomination instruments, in all their forms (self, peer and teacher nomination). Contained in these evaluated contents are the roles that the students can assume in the problem, and the type of aggressive behaviours.

In the third section we expound the proposal of a general survey questionnaire developed at the heart of this working group, initially presented in the Braga meeting. The version discussed here covers the comments made in that meeting, as well as the changes that were considered convenient after its piloting with a sample of Andalucian students. This proposal is also accompanied by comments about its application and analysis, as well as a comparative study with the previous questionnaires. This piloting could not have been developed successfully without the collaboration of Professor Fernando Justicia and D. Juan Luis Benítez of the Universidad de Granada.

2. General Survey Questionnaires

One of the biggest problems in studying bullying is that it usually grows and stays hidden in the world of relations that are established between peers. The aggressors are able to hide behaviour which they know to be undesirable, especially in front of adults, while the victims, ashamed of the situation to which they are habitually subjected, also fail to openly record their experiences.

To confront these difficulties the researchers have usually made use of general survey questionnaires. There are various reasons which have facilitated this circumstance. Normally, they are anonymous measurements which improve the validity of the responses; although that means that we cannot directly identify the students implicated in the bullying problem (a reason which encourages us to use methods of nomination, which are described in the second part of this report). Besides, the questionnaires can be applied to large samples of students, with a proportionally large quantity of information over the nature of the phenomenom and its extent, in a short space of time.

There are many questionnaires to assess bullying, but the most used and adapted has been the Olweus' questionnaire (1989; revised in 1991). There are some very important instruments based on it, but with significant changes (number of questions, and mainly in the contents; see appendix):

• Sheffield version (Smith et al., 1991).
• Spanish version (Ortega, 1992)
• Italian version (Genta et al., 1996).
• London version (Smith et al., 1997).

All of them have been applied to large samples of students in many different countries (Belgium, Canada, Germany, Japan, Ireland, Italy, the Netherlands, Norway, Spain, Sweden, or UK).

There are also other survey questionnaires designed to explore this phenomenon with other features:

• Portuguese questionnaire (Pereira & Almeida, 1994)
• Spanish questionnaire ("Cuestionario sobre intimidación y maltrato entre iguales", Ortega, Mora-Merchán & Mora, 1995).
• Questionnaire about bullying (Smith & Levan, 1995). For children 6 and 7 years old.
• Bullying and Friendship Patterns (Wolke). Initially intended to be applied through interviews. It is not anonymous.
• "Defensor del Pueblo" [Spanish Ombudsman's] questionnaire (Del Barrio et al., 2001 ; Defensor del Pueblo 1999, 2000). Applied across a national Spanish sample.

In the next part we analyse these instruments in order to know their characteristics and potential in the assessment of the bullying problem.

^ to Top

1. What are the contents of the general survey questionnaires?
   
   When we analyse the contents of questionnaires concerning bullying we must make a distinction between the topics that appear in the majority of them, and those that are not common, but still interesting. For this analysis, in order to organise this information, we will show all the topics, the questionnaires in which the topics appear, and the questions related to them.
   
   The next table shows the information that each questionnaire provides. Although in the majority of them two parallel forms for primary and secondary school exist, we have only distinguished those forms in the Portuguese questionnaire, because in that case the organization of the contents are quite different.
   
   Description of the used categories:
   
   

   Identification victims:	Self-identification, number of victims in the class/school, frequency of victimization events.
   Identification bullies:	Self-identification, number of bullies in the class/school, frequency of bullying events.
   Types of bullying:	Identification of ways to bully others (physical, verbal, indirect).
   Places of risk:	Identification of the places where bullying occurs.
   Structure of the group of bullies:	Number, gender and grade (age) of bullies.
   Attitudes to bullying:	Thoughts about bullies, bullying events, and hypothetical participation in bullying events.
   Family:	Sociodemographic aspects, well-being, relationships.
   Social relationships:	Number of friends, peer relationships, feelings of loneliness.
   School:	Attitudes to breaktime, school environment, relation to teachers, well-being.
   Bullying outside of school:	Bullying events on the way to/from school, bullying events by others.
   Intervention:	Intervention carried out by teachers, peers or family.
   Communication about victimization:	Communication about have been bullied with teachers, peers or family.
   Communication about bullying others:	Communication about bullying others with teachers, peers or family.
   Feelings regarding bullying:	Feelings of the students about bullying events.
   Bullying teachers:	Number of students who bully teachers in the class/school, self-identification, frequency of bullying events.
   Coping strategies of victims:	Responses to the bullying events.
   Causes of bullying:	Attribution of reasons, motivation to bullying events.
   Concept of bullying:	Definition of the problem, selection of examples of bullying behaviours.
   Duration of victimization:	Period of how long victims have been bullied.

^ to Top

1. Do they have any definition of the problem? What criteria are used?
   
   The general survey questionnaires usually provide students with a definition of the bullying problem to increase the validity of the instrument. However, there are differences among the definitions found in the questionnaires. The table below shows the criteria used in the different questionnaires.
   
   

   	Olweus	Sheffield	Spanish version	Italian version	London version	Portuguese (primary)	Portuguese (secondary)	Spanish Questionnaire	Questionnaire about bullying	Defensor del Pueblo	Bullying and friendship
   Physical bullying
   Verbal bullying
   Indirect bullying
   Bullying teachers
   Intention
   Frequency
   Imbalance of power
   Individual bullying
   Group bullying
   Effects of bullying
   Pictures
   What isn't bullying

   
   
   The "Questionnaire about bullying" (Smith & Levan, 1995) and the "Bullying and Friendship Patterns" are an exception in this group, because they don't have any definition. However, the first instrument asks about the concept of bullying which the children have (see previous part about contents).
   
   Two different and interesting features appear in some of the questionnaires. First, the use of an exclusion clause to mark what is not bullying. This type of information is very useful to establish the limits of the concept. Second, the use of pictures to make easier the understanding of the bullying events.
   

   ^ to Top

   
   
2. How many questions do they have?
   
   The number of questions is a formal issue in the questionnaires, but it is relevant in this report because long instruments with a lot of items decrease the attention levels of the students and make it more difficult to maintain consistency in the answers (especially with younger students). Previous experience with these instruments suggests that it is better if the questionnaire does not have more than 30 questions.
   
   

   	Junior (primary School)	Senior (secondary school)
   Olweus	40	41
   Sheffield	26 (there is a follow-up version with 28, JRG5)	26
   Spanish version	27
   Italian version	27	28
   London version	24
   Portuguese	32	30
   Spanish questionnaire	30	30
   Smith & Levan (1995)	20
   Defensor del Pueblo		27
   Bullying and Friendship	24

   
   

   ^ to Top

   
   
3. What is the age range of these instruments?
   
   The general survey questionnaires give us the possibility to assess students of school age. It looks possible to use these questionnaires with older students at school; however it is necessary to use adaptations for younger students (7 years old or less).
   
   

   	Junior (primary School)	Senior (secondary school)
   Olweus	8-11	11-16
   Sheffield	8-11	11-16
   Spanish version	8-16
   Italian version	8-11	11-16
   London version	8-16
   Portuguese	7-12	12-18
   Spanish questionnaire	8-11	11-16
   Smith & Levan (1995)	6-7
   Defensor del Pueblo		12-16
   Bullying and Friendship	not specified

   
   

   ^ to Top

   
   
4. Conclusions about the use of general survey questionnaires
   
   Advantages:
   1. They allow us to evaluate a large sample with low cost (in comparison to other methodologies: e.g. interviews).
   2. They provide us with first-hand information based on the student's self-reports.
   3. They are anonymous to improve the validity of the responses.
   4. They provide us with a lot of information about the nature of bullying, but also about social relationships, family or intervention strategies.
   5. They take account of the student's age.
   6. They can be applied in an easy way, even for non-expert people.
   
   So far, there is no questionnaire that includes all the relevant topics. In the third section of this report, we suggest a group of core questions that try to solve this problem.
   
   Criticisms:
   1. Maintaining consistency of responses is complicated, especially for the youngest children, because we ask them to keep their attention for a long time in questions with different formats. It is not unusual that the students forget what they have answered before.
      
      
   2. This type of instrument allows us just a superficial approach to the problem, because their limited number of questions doesn't give us the opportunity to go into some topics (e.g., family influence, antecedents and effects of bullying events). We would need to design long instruments, or to use other qualitative methods like interviews to solve it.
      
      
   3. We cannot get information without the influence of the sociocultural context, because very often we have to use cultural explanations to make the problem understandable for the students
      
      
   4. The questionnaires, like all the methods based on retrospective measurements, need precise information about past events, but it is not always possible to get this with accuracy.

^ to Top

3. Nomination Methods

Nomination methods are based on the identification of the people who fit one or more features. To do it, the person who answers has to name, or select from a list those people who he/she thinks fit to those specific characteristics.

Nomination methods can be divided into three main categories: self-, peer-, and teacher-nomination. These collect information from the different people who are near to the bullying. The main feature of these instruments is that they provide us with information about the students involved in the bullying. Therefore, nomination methods give us the opportunity of focusing our attention on the students very much affected who need a direct intervention. At the same time, it is very useful to know more about the nature of the bullying through the knowledge of the students involved in it.

Here, we describe seventeen different instruments, but this description is not exhaustive. However, we think that these instruments are the most relevant to achieve our objectives (nominations methods regarding bullying). We have distributed the instruments into the three categories; sometimes, this means that the same scale appear in more than one place.

Self-nomination:
• Life in School (Arora & Thompson, 1994)
• Peer Relations Questionnaire (Slee & Rigby, 1994)
• Kochenderfer & Ladd (PPSS,1996)
• Perry, Kusel & Perry (1988)
• Bjorkqvist, Lagerspetz & Osterman (DIAS, 1992)
• Bjorkqvist & Osterman (PECOBE, 1995)
• Participant Role Questionnaire (Salmivalli, 1993,1995R short version)
• PVS (Austin & Joseph, 1996)
• BBS (Austin & Joseph, 1996)
• Multidimensional Victimization Scale (Mynard & Joseph, 1998)
• Ortega & Ortega (1999)
Peer-nomination
• Perry & Bussey (PNI modified, 1977)
• Schwartz, Dodge, Pettit & Bates (1997)
• Bjorkqvist, Lagerspetz & Osterman (DIAS, 1992)
• Boulton (1995)
• Bjorkqvist & Osterman (PECOBE, 1995)
• Crick (1995)
• Participant Role Questionnaire (Salmivalli, 1993,1995R short version)
• Ortega & Ortega (1999)
Teacher nomination
• Perry & Bussey (PNI modified, 1977)
• Bjorkqvist, Lagerspetz & Osterman (DIAS, 1992)
• Bjorkqvist & Osterman (PECOBE, 1995)
• Ortega, Del Rey & Ortega (1999)



^ to Top



1. What are the contents of the nomination methods?
   
   In describing the contents of the nomination scales we first consider the roles distinguished in each instrument. This is very important for selecting the students in order to go deeper into the nature of bullying from the different aspects of the problem. Second, we describe the aggressive behaviour categories that appear in the scales, and the number of questions related with each.
   
   Role in the bullying problem

   	Bully	Victim	Prosocial Scale	Defender	Assistant	Reinforcer	Outsider
   Life in School
   Peer Relations Questionnaire
   Kochenderfer & Ladd (in PPSS)
   Perry, Kusel & Perry
   DIAS
   PECOBE
   Participant Role Questionnaire
   PVS
   BBS
   Multidimensional Victim Scale
   PNI modified
   Schwartz, Dodge, Pettit & Bates
   Boulton
   Crick
   Ortega & Ortega
   Ortega, Del Rey & Ortega
   Bullying Nomination Scale

   
   
   Actually, the prosocial scale (fourth column in the table) is not a specific role. In fact, prosocial behaviours could appear in more than one role. However, we think that this information could be interesting to appreciate the participation of the students in this respect.
   
   Aggressive Behaviour Categories

   	Physical	Verbal	Indirect / relational	Attacks on property	Unspecific
   Life in School	7	5	5	3	0
   Peer Relations Questionnaire	2	2	2	0	5
   Kochenderfer & Ladd (in PPSS)	1	1	1	0	1
   Perry, Kusel & Perry	1	1	0	0	1
   DIAS	7	5	12	0	0
   PECOBE	1	1	1	0	0
   Participant Role Questionnaire	1	1	1	0	0
   PVS	1	1	1	0	3
   BBS	1	1	1	0	3
   Multidimensional Victim Scale	4	4	4	4	0
   PNI modified	5	4	0	0	5
   Schwartz, Dodge, Pettit & Bates	2	1	0	0	3
   Boulton	0	0	0	0	2
   Crick	3	2	5	0	0
   Ortega & Ortega	6 (male)	0	6 (fem.)	0	0
   Ortega, Del Rey & Ortega	0	0	0	0	5
   Bullying Nomibation Scale	2	2	2	2	0
   TOTAL	44	31	41	9	28

   
   
   Description of the aggressive behaviour categories:
   
   

   Physical:	Hit, kick, trip, push, pull, shove, fight, threaten
   Verbal:	Insult, call names, yell, make fun of somebody
   Indirect/relational:	Shut the other out of the group, ignore, tell bad things behind the back, tell false stories, tell secrets to a third person, try to get the others to dislike the person, criticize the other one's hair or clothing
   Attacks on property:	Take things, steal, break something of other, deliberated damage on properties of others
   Unspecific:	Bully others, pick on, tease, be victimized

   
   

   ^ to Top

   
   
2. How many items do they have?
   
   The numbers of items which appear in this table don't have to be the same as given in the previous table, because some of these scales use filler items which we haven't considered.
   
   

   	Number of Items
   Life in School	40
   Peer Relations Questionnaire	20
   Kochenderfer & Ladd (in PPSS)	4
   Perry, Kusel & Perry	8
   DIAS	24
   PECOBE	7
   Participant Role Questionnaire	48 (22 in short version)
   PVS	6
   BBS	6
   Multidimensional Victim Scale	16
   PNI modified	26
   Schwartz, Dodge, Pettit & Bates	6
   Boulton	2
   Crick	14
   Ortega & Ortega	6
   Ortega, Del Rey & Ortega	5
   Bullying Nomination Scale	5

   
   

   ^ to Top

   
   
3. What is the age range of these instruments?
   
   Although the authors have used these scales with different samples (kindergarten, primary or secondary school), an analysis of their contents tells us that we can use all of them for a wide range of ages. To get better information we just have to check if the roles which appear in the scale are suitable for our objectives, and if the aggressive behaviour categories which they assess are the most relevant for the features of our sample (gender and age differences).
   

   ^ to Top

   
   
4. Conclusions about the use of nomination methods
   
   Advantages:
   1. 1. These methods usually have an easy and quick application, although there are exceptions (e.g., long scales, individual applications during interviews).
   2. They provide us with a safe identification of the children involved in the bullying process based on the agreement among informants.
   3. These methods identify particular children who are involved, and this can be important can be important for further research and/or intervention.
   
   
   Criticisms:
   1. These instruments don't give us a lot of information. They are weak when we try to assess aspects like feelings, causes, or contexts concerning bullying.
   2. Overall in Self- and Teacher- reports, the validity of the answers is not sure for different reasons:
      1. Self-reports: the students sometimes look for the desirability in their answers or protect themselves.
      2. Teacher-reports: the teachers are not always aware about what is happening between the students.

^ to Top

4. Proposal of Core Questions for a General Survey Questionnaire

The proposal of core questions for a new general survey questionnaire were made by the working group, at a meeting held in Seville (12th - 13th of February, 1999), after checking all the questionnaires analysed in this report. This instrument has been piloted previously with a sample of Andalucian students. Resulting from this pilot study some small variations were introduced to the initially designed questions, we will specify these changes after the questionnaire.

Comments on core questions:
1. The period of time that we are going to explore is the last three months.
2. After the definition, three cartoons should be included to show the students each type of bullying: physical, verbal and indirect.
3. There are some questions where we could include more answer options, in order to adjust the questionnaire to the country's characteristics: 6, 10, 19
4. The questionnaire could be completed with more questions, depending on the aims of each team. Anyway, it looks like a good idea if the size of the questionnaire is between 25 and 30 questions.

The assessment of the response consistency in the questionnaire:
We can distinguish between major inconsistencies, when a child is clearly either deliberately not answering properly, and/or misunderstanding key questions; and minor inconsistencies, where one question might be inconsistent, but in an "understandable" way. The major inconsistencies must be discarded, but we should not take out questionnaires where one or two explicable errors were made. The proposed major inconsistencies, used in other instruments and in the piloting of this questionnaire would be:

*D or E in q3, and A elsewhere (q4-q14)
*D or E in q18 and A elsewhere (q19-q22)
*More than 1 response for q3 or q18

* In case of incompatible responses in the questions where the students can select more than one option, remove the question not the questionnaire.

Comparison with other questionnaires:

In this section we show, regarding the proposal that we are presenting here, the same analysis of contents and definition that have been used on the previously analysed instruments, with the aim of facilitating possible comparisons between them.

'TMR' questionnaire contents (number of questions)



Identification victims (frequency)	3	Intervention	12, 13, 14
Identification bullies (frequency)	18	Communication about victimization	11
Types of bullying	4, 20	Communication about bullying others	22
Places of risk	6	Feelings regarding bullying	9, 15, 21
Structure of the group of bullies	7, 8, 19	Bullying teachers
Attitudes to bullying	16	Coping strategies of victims	10
Family		Causes of bullying	17
Social relationships	2	Concept of bullying
School		Duration of victimization	5
Bullying outside of school



Definition included in 'TMR' questionnaire

Physical bullying
Verbal bullying
Indirect bullying
Bullying teachers
Intention
Frequency
Imbalance of power
Individual bullying
Group bullying
Effects of bullying
Pictures
What is not bullying



The questionnaire is not reproduced on this website. Most of the (core) questions, including the definition of bullying and the specific questions measuring 'being bullied' and 'bullying others', are taken from The Revised Olweus Bully/Victim Questionnaire [Olweus, D. (1996) The Revised Olweus Bully/VIctim Questionnaire. Mimeo. HEMIL, University of Bergen, N- 5015 Bergen, Norway]. Use of these questions, in research or otherwise, requires written permission from Dan Olweus (Research Center for Health Promotion, Christies gate 13, N-5015 Bergen, Norway), except in Japan. Those interested in using the 'TMR' questionnaire should first [unless they already have permission from Dan Olweus] contact him at the above address, or at olweus@uib.no



The pilot study


With the aim of checking the functioning of the questionnaire we completed a pilot study with a sample of 423 students from the provinces of Seville and Granada, with ages between 9 and 16. The students were from four schools, two from each province, two of them being primary schools and two of them secondary.



Comments after piloting


The results of the pilot study were clearly satisfactory. Here are those aspects that have turned out to be the most relevant.

• The application time was between 15 and 30 minutes.



• We noticed the utility of holding a discussion with the group (approximately 15 minutes long) about the concept of bullying which appears in the questionnaire, before applying it. The number of questions asked by the students during the administration decreased considerably.



• The number of questionnaires with serious problems of inconsistency amongst their answers, using the previously mentioned system, has been small (6%). Therefore, It seems convenient to use this procedure.



• It appears convenient to include the age in the initial data of the questionnaire.



• We have also included changes to some of the questions:
• Question 15: We have included the option "I can't stand it".
• Question 17: We have included the options: "to feel superior" and "because of envy".
• Question 17: We have given the opportunity to choose more than one option.


Appendix 4
References


^ to Top