Wednesday, June 24, 2009

Continued Problems with Intercoder Reliability

We still don't have publishable intercoder reliability. Once something is identified as important, there is decent agreement on the characteristics of what was said. The problem is in identifying what's important, in unitizing the data. With 20% done, I have to say that it doesn't look like this method is going to work. Grrr. So, Plans B and C.

All along, we've been planning to move from the Primetime "breadth" project to one or two shows in "depth." We planned to get DVDs or online versions of at least a season of one or two shows and look at our topics in the context of interactions rather than isolated from the rest of the season. With the show that we picked, I think that what I am studying will be more straight forward, so I'm hopeful that I can get useable data from this part.

Meanwhile, I am planning to go through all of the Primetime data and unitize it myself. I will record the show, disk, and time of each datum. Then, if there's time in the summer, I'll have everyone code the characteristics of that conversation. If there's not time, I'll save it for a directed studies project down the road. Frustrating, but it's such a cool project that I can't scrap it.

No comments: