Wednesday, 21 October 2015

Final Post (with comments)


Reflection about different ways of combining different methods in order to answer complex research questions

After taking this course I can see a lot of the mistakes I made writing my bachelor’s thesis, and I have gained much knowledge about how papers can/should be written both by reading many different ones and by reading about and thinking through what the different parts of research really are. I used qualitative methods for my bachelor thesis, and thought that it was best if we only chose one kind of method, but now I’ve learnt that the study could have used a combination of qualitative and quantitative methods as long as it would have been beneficial to it.

Case Studies

One subject I have read about and reflected on is case studies. Case studies are a great way of conducting research while combining different methods to answer complex research questions.
I would say that a case study is not a research method in the same sense as qualitative and quantitative methods but rather a method to gather data about the case at hand. The data then has to be analysed by quantitative and/or qualitative research methods for the researchers to be able to formulate a theory about the subject.

Examples of combining qualitative and quantitative methods

Are case studies the only research studies where different methods are used in conjunction to answer research questions? No, far from it; a great deal of research conducted use a combination of methods in pursuit of knowledge and answers. For example, when designing a study it can be wise to use qualitative methods to find out ways the study could be improved, and quantitative methods can be used to find out if the study yields satisfactory data. The study itself can then use any combination of qualitative and quantitative methods, which I will discuss next.
There are many examples of how qualitative and quantitative methods can be combined. One could use qualitative research methods such as open-ended questions in conjunction with quantitative methods to catch information that would otherwise be missed in a purely quantitative research—you can for example find out that some participants in your study did something (that you hadn’t thought about beforehand) that invalidates the data gathered from them.
Another example of how quantitative and qualitative methods can be combined is to gather more data than could be gathered if only one of them was used. When conducting interviews one could gather quantifiable data by measuring all kind of body data (movement frequency/amplitude, amount of sweat, body temperature, frequency of blinking etc.), and analysing it with quantitative methods, while also gathering data which can be analysed by qualitative methods (the answers to the questions etc.) By doing this new knowledge can be synthesised from the combination of the separate results. This can possibly take more time than just analysing the data using one method, but that would yield different knowledge.

Design Research

Design research is another area where a combination of different methods can help answer complex research questions.
As I reflected on during theme 5 the design of prototypes can be used in research to answer research questions and gain knowledge. The design process itself can be the empirical data of a research (how the prototype is changed, because of what it changes, what works and what does not work in this specific context etc). Usage of a prototype or how it provokes things in its surroundings can also be the empirical data of a design research.
The design research method of gaining empirical data can be combined with a case study for example where a specific group tests each iteration of the design.
To make a research contribution, as the lecturer Anders Lundström said, the empirical data gained from the design work must be analysed using a combination of qualitative and quantitative methods.

Freedom

To be able to make a substantial knowledge contribution on does not have to follow strict guidelines and frameworks which tells you what methods to use and how they can be combined.
I think what Paul Feyerabend said about that anything goes when conducting research was true. If you follow guidelines or frameworks too strictly there is a great deal of alternate approaches which could yield better results that will go unused, and we’re then left with faulty models of the world because the methods that could result in a paradigm shift are left unused.
A paradigm shift is what happens when an old theory is disproved by a new one and we have to formulate new theories based on the new one, thus creating a different model of the world.
An example of this that has been brought up many times during this course is the Copernican revolution where the research of Copernicus disproved the geocentric model of the universe and replaced it with the heliocentric model which we have today. If Copernicus would have used the same methods that his fellow researchers used he might have not come up with the model that led us to where we are today, and instead continued the search for complex mathematical models that explained how the other celestial bodies revolved around Earth.

With this in mind I think that researchers are free to use methods and combinations of methods that are not widely used or haven’t really been used or thought of yet in order to see if the results will lead to progress in our understanding of the world. No praxis in the research community should keep researchers from trying to make revolutionary findings.
Nothing can be seen as objectively true and so we have to keep on prodding the world to find out more and keep on creating improved models of it.

That’s it for me.
Thanks for now; have a good time reading my comments below!


Theme 6: Comments

Great reflection on what defines a case study and what it can be used for. I like that you bring up that we don't have to follow a specific method; I believe that if you do follow specific methods your research can become too constrained and the knowledge gained won't be of as great value as it could be if you were to "think outside the box".


I think it's great that you've gained new insights about case studies, and by reading your reflection I can see some things more clearly now than I did before.
Ilias' example about the person with a rare disease is a very good example. I would like to add that when doing a case study one does not have to follow any particular guidelines since it may not be any knowledge about the case beforehand and too strict guidelines can be inhibiting on the research. Of course you will have to gather data and analyse it in a way that is verifiable etc, but since—as you say—it's often an investigation of a new, interesting and different case than what has been researched before I think "anything goes".

Good job!



Hello!
I love what you say about the number of participants in a study not having a determining effect on whether it can be qualitative or quantitiative; too many have preconceptions about this.
Regarding case studies, I don't think you have to draw a line between case studies and qualitative methods. Since "case study" isn't a method in itself you will have to use either qualitative, quantitative or some other methods when analysing the data gathered in the case study. I agree on the other points you bring up, though.

Keep up the good work!



Hello!
Feyerabend's thoughts stuck with me after the seminar as well. I really liked his points and how it ties to case studies where no previous frameworks may exist etc, but the research methods you use may restrict the research if not used with care.
Great summary and reflection on what we learned about case studies this week, I feel like we had a similar experience for this theme.



Hi!
I enjoyed reading your reflection. You explanation, in your pre-reflection, of what a case study is was good, but I agree that it's easier to get the gist of case studies with the explanation provided here.
The example with Sephora seems like a great example of what a case study can be like, and sounds interesting.

Friday, 16 October 2015

Theme 6: Post theme post

This week I read two papers—Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media
 and A value-oriented and culturally informed approach to the design of interactive systems
—the first of which was a qualitative research using semi structured focus group interviews about using mobile devices to complement formal higher education. The other paper was also conducted using qualitative methods, but this one involved a case study about the creation/design of a social network for teachers of students in need of specialised education.

At the seminar we discussed what we had read and what we had learnt about qualitative methods and case studies from reading our papers.
First of all we agreed to having a hard time finding papers using qualitative methods and case studies, at least in relation to finding the papers for the preceding seminars.
The qualitative methods that were used in the papers we discussed were semi structured interviews, focus groups, auto driving, and one of us read a cross case study which used structured interviews + some quantitative method.
From these the most interesting to me was the “auto driving” method because I hadn’t heard about it before. In this case the method was that the participants were given a voice recorder and were supposed to give a tour of their home and say what their thoughts were about the things they saw, so in a way it’s really an observation method.

When we discussed case studies we agreed on a definition that was very similar to what I had written in my pre-theme post. We also said that you can mix qualitative and quantitative methods when conducting a case study since the case study itself isn’t research method but more like a tool to gain knowledge about something you know nothing or very little about before. The data gathered through your case study must then be analysed and discussed for us to be able to formulate a theory about the subject.

At the seminar I also learned about Paul Feyerabend’s assertion “anything goes”, which means that you don’t have to follow guidelines or “recipes” when conducting research.
Case studies is an example of Feyerabend’s assertion; you don’t have to know a lot in advance of your study, and there may be no guidelines if the case’s context is completely new. This means that you don’t get stuck in the old ways when conducting your research and can in that way gain new knowledge. An example of scientific progress made when not following the rules that were set by other theories is the Copernican revolution which was brought up earlier in this course.

So don’t follow the old way of conducting research just because it’s considered the correct way; see if you can go outside the frames of our knowledge and make a real contribution.

Thursday, 15 October 2015

Theme 5: Comments

I think it's interesting that you focused on ideas, went more into detail about what ideas are than was done in the first lecture and that you discussed ideas in a general sense as well as how it relates to academic research.
About the second lecture, I also think that an important part of it was that design in itself is not a knowledge contribution, but when the data gathered during the design process is analysed we get a knowledge contribution.



Great summary of the week's theme.
I think a large part about coming up with great ideas is having a lot of knowledge that you then can see connections between. This knowledge can be gained through for example the intake of earlier research or by prototyping.
I agree with you, and the lecturer, that prototypes are used as something that provokes to gain more knowledge, and that this directs the research process. This design process is then analysed and evaluated so that you are able to make a knowledge contribution.



Hello!
You summarised the week very well, and I enjoyed reading your reflections on the subjects brought up.
The part about securing funding through a proof of concept and pitching it the right way was interesting to me as well. It's good to have that in mind as early as possible, since you won't get any resources when doing something completely new and innovative if you can't show those who have the resources why it's so great.
It's great that you bring up that the process in design research can be considered the empirical data.



Hello!
I completely agree with your sentiment that it would have been great to pilot our research designs for our bachelor theses, since so much small errors really get in your way when conducting "poorly" designed research. I piloted my questionnaire, but not some of the other integral parts of my research which in hindsight wasn't perfect.
As others has said your distinction between the use of prototypes in industry and research doesn't really agree with my view. The use of prototypes can, in my opinion, be the same between the two, but I guess that it's more common in research to use prototypes just as a provocation to gain new knowledge and it doesn't have to solve any problems.



Great summary of some of the key concepts from the week's lectures, as well as some insightful reflections!
I agree with you on the point that all research shouldn't be motivated by how much money there is to make from providing a solution to a problem. Industrial research kind of has to be, but academic research shouldn't. Another part of Haibo's lecture that involved money was that you have to get funding for your research, and to get it you can for example produce a proof of concept and—probably most importantly—"sell" it in a good way by doing as the entrepreneur he spoke about.
I like that you bring up the way design research process steers the researcher towards the real problem.

Monday, 12 October 2015

Theme 4: Comments

Interesting reflection!
I agree with what you say in the last paragraph; the context is important to have in mind since, for example, the time the data was gathered or analysed can have an impact, among many different factors.
Your example from the seminar about quantitative vs qualitative methods and tastes in food was great, but I'm not sure i fully agree with you about the need to use qualitative methods as soon as opinions are of interest. However, I get your point.




I like that you came to the conclusion that quantitative and qualitative methods are complimentary, and that you can use qualitative methods while designing your research.
When you brought up your example with the picture I came to think about how the data you analyse isn't 'true' since the 'picture' will differ depending on many variables such as time, who took the picture, what was used to take it etc.
To reflect on your example, I think you still have to analyse the objective data you extract from the image when using quantitative methods. What you say about extracting deeper data about a specific part is a great explanation of qualitative methods, in my opinion.




I like the different examples you bring up of when to use one method or the other.
It really is good to have knowledge of different methods and how they fit for different questions and cases, so you don’t refrain from using one just because ju have a preferance for the other. The importance of designing methods so your participants don’t alter their behaviour, or at the very least having the possible change of behaviour in mind when analysing data, is great to have in mind.
Great summarisation and reflection!




I think it's great that you read "Methods in empirical research of communication", did they have anything interesting to say that wasn't discussed in class?
I like that you bring up that qualitative research is not more subjective than quantitative since I believe it's common for people to think it is.
After reading all the papers in this course, going to the lectures and discussing at the seminars I feel I have much more knowledge on how to write a good research paper than I had when writing my bachelor's thesis, it would be interesting to hear what everyone else in this course feel. Can you see some of the mistakes you made before or what you could have done better now?




Hello!
I found your reflection interesting, especially that you discussed what you had done in your bachelor studies during the seminar.
I was also familiar with quantitative and qualitative research methods before this week, but I feel that there is alway something new to learn and our knowledge of the different research methods can alwlays be improved.

Friday, 9 October 2015

Theme 5: Post theme post

This week I read the three papers and thought they were very interesting; Haibo’s paper was easy to read, follow and understand, the other two were also easy to read but didn’t capture me the same way as the first one.

Haibo Li

Haibo Li’s lecture was mainly about tips and tricks on how to conduct (design) research, all the way from how to come up with an idea to the way you should communicate it. I learned that you should spend most of your time on defining the problem and then, when you have a well defined real problem, you spend the rest of the time solving it. If a problem seems unsolvable you might just have to re-state it to find the real, solvable, problem.
He also told us how to filter ideas if we have to many, and only go for the ones who lead to a breakthrough if we don’t want to waste our time.
Validation of an idea using proof of concept was briefly touched upon, but I thought this was something we ourselves thought about when writing our pre-post for this week.

Anders Lundström

In this lecture I learned that Proof of concept and Evaluation was mostly used in the industry and not as much in research. Since research is about gaining knowledge Anders told us that a prototype used in this context doesn’t have to solve anything but it should provoke something that helps you answer the research question to gain knowledge.

He talked about the Fernaeus & Tholander paper and that the process—how and why the prototypes were changed—was the empirical data of their research. When they then analysed that data in their “Lessons learned” part they made a research contribution, but the process itself was not a contribution.

He then discussed what design was and said that it is to arrange/reconfigure something into something else (e.g. an unwanted reality into a wanted reality). Then he said that design can be about finding the real problem; “Is this the problem we should be looking on? Maybe we should be looking at something else?”

So what I learned during this week was how to conduct design research, what design can be, ways of validating and evaluating an idea, the difference between industry and research design and that there are different ways of conducting design research since it is a relatively new research method.

Theme 6: Qualitative and case study research

It was very hard for me to find a paper which used qualitative methods in the journals I searched through. But when I thought about in what areas qualitative methods are frequently used, according to what I had read beforehand, I remembered that educational research often used them. So I browsed some journals with focus on media technology and education, and finally found a paper!

Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media

The goal of this paper was to present students' in-depth perspectives of experiences with implementing mobile computing devices.

The qualitative method used in this paper is focus group interviews using Skype video calls which were recorded, transcribed (additional non-verbal behaviours identified were noted such as head nodding, smiles, frowns or signs of boredom) and then analysed. Guidelines were used to provide structure to the interviews and a semi-structured interview protocol was used.
The interview transcripts were verified by each of the participants who could request edits and additions.

Benefits of using this method is that you can get a lot of extra information from the non-verbal behaviours that you wouldn’t get from a questionnaire, and  by having a focus group interview the researchers could get the real thoughts of the students about their perspectives (getting the information straight from the source).
If they instead would have gone for an approach where they observed the students, a lot of relevant information could have been lost and the results wouldn’t represent the students’ real perspectives.
As stated in the paper: “The qualitative approach allowed for the representation of reality through the eyes of the individuals interviewed in order to share their stories and hear their voices.”

By using Skype video calls instead of face-to-face interviews could mean that they missed some non-verbal behaviours, and by having the interviews be conducted in groups some of the students’ answers could have been changed because of the other students’ answers or just because the other students were present. Though, the students did have a chance to request edits and additions after the interviews had been transcripted, which was a good part of the method used.

From reading the paper I learned that you have to think through how you will code and analyse the data for it to be valid.
Using previous theories and frameworks when you analyse the data and prepare the empirical data for analysis is something I will have in mind for future research.

As they say in their conclusion: “[...] the participants who volunteered to share their experiences did so because they felt the mobile devices did impact their learning — another story would have emerged if participants who did not see the benefits of the devices were captured.” I think they should have had some participants who didn’t think so alike be part of the study, since the result and discussion is very one-sided because of this.

A value-oriented and culturally informed approach to the design of interactive systems

Briefly explain to a first year university student what a case study is.

A case study is an in-depth study of a specific subject and its context; e.g. a study of a family using a new product in a specific environment during a specific time. This study can then be generalized from.

Strengths and weaknesses of the paper

The researchers are very clear with their research question, and gives the readers a lot of information regarding the area of study before going into the case study, which gives knowledge of why this kind of design-approach is needed.

Their participating teachers were all trying to spread knowledge about specialized education services for students who needed this specialized education, and were from different regions of Brazil. There were also researchers in Education and researchers in Computer Science and HCI.

They do make comparisons to similar literature, but at the same time they don’t compare with conflicting literature which is a weakness.
Within-case analysis is made, but cross-case pattern search isn’t.

They do not use multiple data collection methods which is a weakness for the paper, and only qualitative data is used which is another weakness.
There are multiple investigators, and that’s great because they therefore had different perspectives and complementary insights when doing the research.

There is an overlap of data collection and analysis when they go through the case study step by step, and they include some field notes.


References

Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media
Available at:
http://www.sciencedirect.com/science/article/pii/S1096751613000262

A value-oriented and culturally informed approach to the design of interactive systems
Available at:
http://www.sciencedirect.com/science/article/pii/S1071581915000592