Peer Review Workshop

For this week I’d like to comment on our workshop on how to do a peer review. I learned that peer reviews are an important way to protect the quality of research that is published. It was also interesting to discover that the standards of what constitutes “good quality” can differ between different journals. From the peer review tips we learned, what really struck me was the advice of Dr. Tyworth, that we should not turn a project into what we would do. For example, I can see this becoming problematic if you are in a field in which you have extensive knowledge and experience. I learned that ultimately, as a peer reviewer, I must not try to change the question. Rather, I must thoroughly understand exactly what the author is arguing or trying to explain in their article, and based on this, then evaluating whether their methods are appropriate to their research question.

As we all are submitting a peer review, I also found it very useful to hear my classmate’s suggestions on best practices and strategies to utilize in our peer reviews. For example, we could look at the general readability of the article as well as the originality of the actual research question. Perhaps an extension to the workshop could have been reviewing an article together (however, not one of the articles we are selecting from), and then commenting on areas for improvement. I understand the time constraints; we are limited in our discussion time. Still, a practical hands-on example could be beneficial. Athough many of us have never written a peer review before, hopefully we all feel a bit more confident in our ability to write professionally and constructively in our peer reviews after attending the workshop.

Peer Review Workshop Review

I thought I would post about the Peer Review Workshop, as I actually took a lot away from it (at least more than I thought I would)! It helped to drill into my head that criticisms of my work do not equal attacks. Logically I know this, but as I'm sure most of you are aware it's so easy to get emotionally attached to your own written ideas. When someone tears them apart (or at least it feels like that's what they're doing) you tend to become defensive or quite honestly embarrassed and sad (at least I do). The workshop hit home with me in that I genuinely took the comments from my SSHRC proposal in a different light. Some of these comments were harsh, but most of them were clearly designed to help me not only make my proposal better, but to make myself a better student and communicator. This should be common sense by the time one enters a Master's program, but I'm glad I had the reminder that criticism is really designed to help you, not embarrass you. I feel much more confident going into my peer review with these thoughts in mind, and hopefully I will see my own writing improve even further as the course goes on.

Peer Review thoughts...

For this assignment, I decided to review the article dealing with women entrepreneurs in Mauritius. The reason for this was that the research method the author used was focus groups, which is the method I chose for my SSHRC proposal. I was excited about focus groups because of the possibility of rich data (as both Luker and Knight state), and I was also aware of the difficulties of interpreting the data and ensuring that you, as the researcher, can actually say something about what you've discovered through the interviews. What is becoming increasingly clear to me is the possibility that the group dynamic (which is the part that attracted me to the method in the first place) might be an issue.

I've looked at some other articles about focus group research and methodology, and all of them state that it is very important to pick a group that's "just right" - not too heterogeneous, or else people might feel inhibited, but also not too homogenous, or else you might get the same opinions over and over again. In fact, one of the articles critiques the method pretty harshly and says that it provides data that is, basically, wrong. He says that individual interviews yielded much more reliable data than groups. I wonder if it's because the topic was more about individuals' decisions about products (market research, really), whereas focus group research is more conducive to exploring questions that have to do with group dynamics. In the case of the article I'm looking at, I'm not sure yet whether the question was right for the method, but it's definitely something for me to consider for the research proposal assignment: focus group use needs to be really carefully justified.

If I can get Harry Potter involved, I am game

For my Peer Review assignment I have decided to rip apart (jks) President or Dictator? A Comparison of Cuban-American Media Coverage of Cuban News by G. D. Peterson et al. The reason I choose this particular article is because I am interested in content analysis. I think that it (content analysis) will be a method that is relevant to the research that my research proposal is proposing...enter Harry Potter. 

When I was looking for online resources on content analysis I came across this: Teaching Content Analysis Through Harry Potter. I try, very deliberately, to integrate Harry Potter (sans prologue) and various other fantasy series/novel/movies into my life, and here is an opportunity for me to do it academically and relevantly, score! Adam M. Messinger (the author) presents a compact crash course on context analysis by analyzing music from the Harry Potter films. He begins by briefly defining content analysis and its processes (sampling, coding, theory developing). Messinger then uses this method to analyze the music found in Harry Potter to uncover its (the music's) message with the aid of a coding sheet. The article was not spectacular, but it did clarify a few things about the method through application. Also, because it was meant as an learning exercise for undergraduates, it was easy to understand and it related the intents and purposes of content analysis clearly. It is fun and worth reading if you want to obtain a general understanding of content analysis (which I did not get from Luker). However, be warned: you will get Hedwig's theme song stuck in your head...which is not necessarily a bad thing.   

Reflections on being an Interviewee

Last weekend I had an interesting encounter on the street. I was heading through the Philosopher's Walk, on my way to meet a friend, when a total stranger stopped me on the path and asked if he could ask me a few questions about "student life." Normally I would have said no to this - I don't like talking to people I don't know, and I don't like being put on the spot to answer questions, and I find that kind of stopping-strangers-on-the-street method of gathering information a little bit invasive. But I said yes, because I got the impression he was a student and I thought it might be a good opportunity to observe a research method in action.

It was a really strange experience, as an interviewee, to begin answering questions with no real sense of what the questions were going to be about, and with no idea what it was the interviewer was trying to find out. All I knew before we began was that he wanted to know about "student life" -  but what he actually ended up asking me about was how I, as a student, celebrated birthdays. I was distracted, when answering his questions, by trying to anticipate what he wanted to know or what he was actually trying to find out. I don't know if it was a halo-effect kind of thing - I didn't really care about telling him what he wanted to hear, but I was trying to figure out what he was actually doing. Would it have been that hard for him to preface his request with, "I'm interested in finding out about how university students celebrate birthdays?" I feel like just that tiny bit of extra context would have made it easier for me to answer his questions.

It turned out that he wanted to find out whether or not I would find a particular product (some sort of content aggregation thingy for finding birthday gifts based on things like age & gender, from the looks of it?) useful. In the end I wasn't sure what his motives were and felt a little weird about the fact that I let myself get roped into an interview I didn't really know the whole context of. I realize this is probably not going to be how most of us conduct interviews in our research careers, but it was interesting all the same and made me wonder about how or if to prepare a respondent for an interview you're conducting.

In another class I am taking, we had an in-class workshop on conducting semi-structured interviews. When we prepared our interview questions, some of us made a point of giving our questions a little bit of context, and others did not. Is there a reason to choose in one direction or the other? Or, I guess, what is the benefit of giving your respondant no/limited context?  I feel like it would be to everyone's benefit to give a little bit of explanation - by at least letting the person your interviewing know what it is, in a general way, you are trying to explore - but maybe I'm wrong.

BB-SSHRC?

With no readings this week, I figured I’d reflect on my SSHRC proposal. During the lecture Sara brought up the importance of not underselling yourself in the proposal. As a student of bureaucratic  methodologies  and a believer that BBB (bullshit baffles brains), I may have taken that bit of advice a tad too far in my early drafts; as some of my statement induced chuckles and eye rolls from my proof-reader. This makes me wonder what the norms are for how you rhetorically promote yourself within grant work. Through my work opportunities, I’ve had the chance to work with both management teams and technical teams. When I make statements to the technical team that would pass unnoticed by the management group, the frequently engender the same laugh and eye roll that my early SSHRC drafts got. I doubt that saying different groups (technicians, managers, academics, etc.) have different rhetorical norms for self-promotion (and different thresholds for their BS meter) is controversial statement. But I wonder how an outsider (me) can quickly learn the BS threshold for their target audience.

The "President or Dictator?" article

I just read the "President or Dictator?" article that is among our choices for the peer review assignment. It alarmed me by how irrelevant the selected methodology is to the study's thesis statement. I found it surprising that the group of authors did not recognize that. Does anyone else see this or am I completely off?..

The study proposes to "examine the role the media plays in [the US policy positions towards Cuba] to determine if the Cuban-American media is reflecting these views". The study then goes on to gather quantitative data to reveal the amount of times positive and negative descriptors are used in the same sentences as "Fidel", "Raul Castro", and "Cuba" in four American newspapers. This is content analysis, which, I think, works well to determine how much more frequently the three words above are used negatively versus positively, but, unfortunately, it does not offer examination of the media role in people's views! And yet media role in people's view is what the researchers propose to examine in the beginning. Luker says in chapter 8, that when it comes to content analysis she is "skeptical that we can show anything other than a correlation." This method "can only find the distribution of a population into categories that we have defined a priori". The information collected in this study using this method is not enough to offer analysis of the Cuban-American community and its views. And yet, upon presentation of the results, the conclusion of the study attempts to offer qualitative analysis. All of a sudden, in the last paragraph it offers a new idea: "the Cuban-American population  <...> has a more negative opinion of Castro to begin with". How is this clear from the data collected? How do we know from the data examined what views the Cuban-American population had "to begin with"?

The researchers of this paper also failed to successfully justify the use of their method other than noting that it would be not too expensive...

This study is an example of how not to choose irrelevant tools to answer your research question. It is also, maybe, an example of how sometimes it is necessary to completely change your research question after you have collected the data. I found this article to be a troubling example of a mistake that can easily happen to me and to anybody in research work. I guess I have a lot to say about this one, so I'll choose it for the assignment. Who is with me?