Today I am at the Student Social Media Showcase (#SSMS2014) and the Mixed Methodologies Seminar, both precursors to the European Conference on Social Media (#ECSM2014) which I will be at until Friday. I’ll try to liveblog most of the conference days but today I’ll be posting notes as this is a loosely structured day. The Showcase, being Storified here, brings together both students and academic delegates of the conference and, for the student social media showcase, over 100 local school children as well as local businesses and apprenticeship schemes operating in Sussex. Both the conference and today’s event’s have been organised by the Brighton Business School, at University of Brighton.

This morning, while the kids have been experimenting in the creativity suite, I have met the organiser of ECSM2015 (which will be in Portugal), and we have been hearing about the DV8 Sussex Apprenticeship scheme which has been placing students, aged 16 to 23, in businesses from very small cafes to big social media agencies, on specific digital media and social media apprenticeships. They spend four days a week at their employer, and one day a week at college taking a number of social media, digital media, and marketing modules. It sounds like a really interesting scheme and the two students we met this morning seemed like great representatives of the scheme – they will be running hands on experiments in running mini campaigns for the students.

Introductions

Asher, one of the main organisers, is talking about social media and how central it is in business and marketing, and the business school’s recognition of the centrality of social media in our day to day lives. Today the focus is on what social media means for us, for the kids in the audience, and for jobs. And Asher is also talking about some work on “what is it students get out of studying?”, we think that the most important thing is learning how to learn… if we give you a seminar on Snapchat, it will be out of date in 6 months time, so the important thing to learn is how to research this stuff, how to learn about it, and how to think about what social media can do in business, in media, in the arts.  And as you look at the displays around the building you will see work by students that demonstrates that.

Sue: When we knew we would be hosting this event we went out looking for partners from the local community. We knew that the research conference would bring in people from across the world, but we also wanted to pull in local graduates and near graduates, but also local employers, and schools. We want to see how this all works, and we plan to do it again and again every year. We should have lots of spontaneous conversations… talk to anyone, see what they do, what they use… And there will be stuff every hour in this theatre – and we have five students you can talk to right away…

Tom English: I will be talking about Snapchat and ASOS, and how Asos could use Snapchat to sell their clothes

Cecilia: I’ll talk about Zara and how they use Facebook, Twitter and Pinterest to communicate with customers

Abiola Oduwasi: I’ll be talking about how people prepare to present themselves for the jobs market – graduates and recruiters

Sean Fitzsimons: promoting your writing and journalism through social media

Alice Britton: I did a project on how Bagelman, a local business, used social media for their business

Showcase

Running throughout the venue today are screens showing digital media presentations from students. Some nice case studies that I’ve already seen included a presentation on beauty bloggers and brands’ use of sponsored posts – where the blogger receives direct or indirect benefit from the brand for writing about them. Some examples were shown and some research suggesting that consumers find reviews useful no matter whether or not they have been paid for was quoted – an interesting finding in the blurry authenticity space that is social media. More on that in Lu, Chang and Chang 2014.

Brighton Fuse Project: Why Social Media needs all your skills – Dr Jonathan Sapsed

It will be good to talk to you today about this project, the Brighton Fuse Project, a research project looking at new media, digital media, and creative industries in Brighton. There is real clustering of these industries in Brighton – you see it in Shoreditch, in Bristol and Bath to some extent, Salford, etc. There is no one big company in Brighton drawing people in – unlike the BBC in Salford – so we wanted to see what was drawing them to Brighton, what attracts them. And we also saw that these companies need people like you (the teens in the audience), and all your skills.

This was a £1.5 million project with University of Brighton, University of Sussex, Natioanl Centre for Universities and Skills, BBC Academy, etc. involved. And Ed Vaizey welcomed this report and it’s findings on the Brighton CDIT. We’ve had a lot of interest because we looked at how the creative industries and skills really intersect with business. And we’ve also seen a huge investment made in Brighton to encourage these industries, to improve infrastructure and the quality office space for these high growth creative businesses. These sorts of things can be exposed through this kind of research, and you can then talk about how to address this.

So what is “fusion”? Well the combination of creative design skills and digital technology skills, the mix of artists, programmers, and business skills. One of our participants from Plug-in Media talked about how important the relationship between creativity and tech is. And we’ve known that idea, that concept of fused content, is important for a long tie for converging platforms – games, tv, mobile, online, etc.  But we didn’t know the extent to which this fusion was needed in sectors like social media. So lots of these digital media companies who have been running since the 1990s are increasingly adding design skills, social media skills, it’s about working out what the company desires, what they will want next, how a campaign can engage people more, to sell more. So you need those sensibilities of the analytical, segments, and patterns of search but also the creative skills and sensibilities for this space.

We looked at entrepreneurs… those who did their first degree in Arts and Humanities or Design are about 48% of the entrepreneurs. That was a bit of a surprise. And those with more degrees, with PhDs, their businesses often were doing even better. And whilst STEM and Computing folks were also doing well, it was equally as well as those from Arts and Humanities backgrounds.

But we also found that some firms are more fused than others. Some – about a third – are specialist so only really employ developers, or only really employ designers. About a third have some mix, and then we have the “super fused” who are dependent on having a tightly integrated mix of these skills. In terms of what types of companies are represented here… the Digital Agencies are more likely to be super fused, as are design services. And the least fused were arts organisations – but that’s probably a good thing, they need to be specialists in my opinion. On the whole fused businesses correlated positively with innovation and turnover growth. The super fused firms grow three times faster than unfused companies. That mix is very important.

So, looking at business models, the firm iCrossing, probably the second biggest digital agency in terms of employees in Brighton, do lots of work as “creative technologists” for various big firms, including Rolls Royce. Now they have a small customer base, they are happy with sales levels, but they want their brand to be more popular…  [brief break as kids leave] So Rolls Royce is an example of a company not looking at sales as a measure. But they had 14 measures of engagement in social media – really playing into the geeky side of what they do, the craftsmanship is shared via YouTube videos and shares of those… so it’s about good creative skills, how to make that interesting and enticing engagement, that is needed. So those 14 measures also get used for triggering payments to iCrossing. Each time they meet a target there, they get paid. So iCrossing employs programmers, journalists, copywriters, graphic designers, tim makers. They are looking for “Creative Technologies” job roles.

And an iCrossing campaign – which I can show now the kids are gone – was for Ann Summers and around paid search (YouTube: Ann Summers: Sexy Paid Search). So this was about using high interest news related web searches that hijack that news story by triggering related ads – for the budget, the BA Strike in particular – and got a good reception and impact for clients – click throughs, media coverage, a huge boost in profile etc. So for that client they have that client on a retainer – giving space for creative ideas, something thought of on the fly. That’s a particularly useful space for experimentation, for lateral thinking, for trying stuff out that is clever rather than high tech, trendy stuff perhaps. Counter intuitive stuff.

We found high levels of innovation in the cluster… and we used the types of innovation used in the European Innovation Survey… usually they find 60-65% innovation but for this cluster in Brighton  99% innovation. And more innovation in super fused companies. And 37% of firms allow time for personal projects – and that allows space for unexpected products and services for the firms.

Fusion is linked to innovation but… it’s not new to the world technology, traditional R&D, or protected by patents. Instead it’s service-oriented, continuously attending to user-experience and design. The value is hard to capture, in spire of £231m revenues across the 500 companies we looked at.

In terms of location… these organisations work for some local firms 40% ish of the companies do local, often business to business work for each other. A good 56% work for clients in london. And about a quarter work for international clients. And these firms are relatively young… the average respondent is 41.7 years old, two thirds of respondents are in their 30s and 7.8% in their 20s. And there are real cross overs of backgrounds… some have STEM backgrounds (22.89%) but many are from Arts and Humanities, Design, Business Management or Economics… but some have, say, stage management degrees… and they bring that creative background to bear on their work.

And the people working in these companies… only 8.4% always lived in Brighton. Many moved to Brighton for the lifestyle (e.g. one of the most successful web company CEO’s cited Britain’s only Vegetarian Shoe Shop as a reason he moved to Brighton!), many for personal reasons. Rarely do they move to find a job, for professional reasons… we think that is starting to change… there’s a kind of second wave here… many of these companies started in the 90s and they need people like you guys to be part of that next wave… And Ian Elwick, Founder-Manager of Brighton Media Centre and The Werks cite the support, the peer communities, these physical co-working spaces, those types of aspects as being important to these communities [we are now watching video – findable on the AHRC website along with the report – on these types of spaces, how they foster knowledge sharing and “being a good corporate citizen in the modern world”].

There are a lot of different styles of network events… there are cheese and wine events… but those are not so much about help, collaboration, contracting in a business sense… and those engaging in those benefit in material terms… So, a good example. Black Rock Studio, a big developer which was acquired by Disney. They did so well for 10 years they were brought by Disney… something happened… probably a failure of marketing for two big games… closed in 2011… made all of their 279 staff redundant… but a whole group of “black pebbles”, companies started by former employees, set up… and they create apps, small games, smaller scale stuff… some work for hire… some brought out by big Shoreditch company… they meet up, they help each other out, they use social networks online and offline, supportive culture there that is so important to clusters. Though fusion tends to be weak at community level, strong at a business and project level.

But it’s not all perfect news… some risks and barriers facing these companies. Fused firms face skills barriers, they find it hard to find the right skilled candidates. Easy in Brighton to recruit good design hirees, but paid search, product managers, etc. are not skills easily found. Sometimes they have to hire more technical roles through London. That limits growth. They find it hard to find the right people with the right skills… and larger firms perceive artistic community as a barrier… perhaps too laid back, too bohemian according to some. The recession and skills barriers were the main issues facing these firms at the time of the report.

But a key conclusion for us is that arts and humanities is key to interdisciplinary interaction and innovation and economic growth… but the HE system can be suite set again interdisciplinarity, often fields of study are quite separate and that’s not a good fit for creating these fused individuals. And this is a really organic cluster in Brighton, it’s hard to create that sort of effect artificially… policy makers often want to support a wide geographic range of locations but we think they should fund succeeding clusters more, to stimulate growth there…. to let that growth be organic…

Q&A

Q: You didn’t mention Brighton SEO… are you aware of any other conferences or similar happening that cement Brighton as a digital hub…

A: There are lots of those but tend to be very segmented and just known to that sector. In September Reasons to be Creative… and another which Warren Ellis is involved in, Deconstruct,… lots of these things… Twitter is the place to look for these things… a lot more smaller meet ups, in pubs, etc. and a great way to meet and make connections and find jobs, etc. That stuff leads to pub chat… I know one guy, now a senior manager for Electronic Arts in Montreal, who got the leads that led to that job through a pub chat…

Q: If you were designing a module or similar what would you include to address gaps… stuff to support such clusters in future…

A: We’ve talked a lot about this… but a lot of the message that comes from businesses themselves is that comfort with technical and creative sides is essential. And knowing how to manage a project, to be organised, to show leadership, also key. And we’ve thought about ways to best deliver that… practitioners say that graduates aren’t industry ready… and you ask them to help and to get involved in course design… and they are too busy to help… But the bureaucracy of developing courses, and the existence of disciplinary silos, can be the enemy of those sorts of skills…

Asher: if you are a graduate and you have experience of creative writing but never done SEO… or vice versa… what are the first steps to being part of this fused economy?

A: A lot of these skills are very much self-taught… a lot of people learn in that way. A lot of people hire someone they know with those skills and pay them for a morning to teach them on an ad hoc basis – as courses often exist that help with that. And they learn through others…

Information Visualization for Knowledge Discovery: Big Insights from Big Data – Ben Shneicerman, Professor of Computer Science at University of Maryland

One of the fun things here I think is the breadth of types of people involved in these spaces, as we heard before in Jonathan’s talk. Steve Jobs used to talk about his work being at the intersection of technology and the liberal arts. I am based at the Human Computer Interaction Laboratory, an interdisciplinary research community of Computer Science, Information Studies, but also Psychology, Sociology, Education, Journalism, and the wonderful Maryland Instute of Technologies for Humanities. Now many of you may know me from the book Designing the User Interface. Now the stuff you will be talking about at this conference was a real driver for the most recent update, in 2010, to that text. More than 5bn people have mobile phones now and they are changing the world, the way that we interact around health, around community. We have mobile, desktop, web, cloud. We have diverse users, diverse applications… so many opportunities to explore the world around us…

Now today I am going to talk about “Big Data”. In 2012 a release from Obama, announcing a Big Data initiative and talking about visualisation, talks about developing scalable algorithms for processing imperfect data in distributed data stores, and creating effective human-computer interaction tools. So we need to be teaching the key skills of visual reasoning, which we don’t usually teach… In 1999 we published a collection of papers on information visualisation. That area has now massively grown so no longer possible to capture in a book – the web gathers that whole world of papers that is emerging. But we do get some new directions… Jim Thomas and Kristin Cook wrote about the concept of Visual Analytics, Illuminating the Path, in 2004 (online for free). And in Europe Daniel Kein wrote on visual analytics (also available for free).

Now… one of our graduates set up an information visualisation company called Spotfire, growing a business out of their research work. For instance a visualisation showing Retinol’s role in embryos in vision – a rare example of a single image acting as an important research finding. That’s a rare occasion… but that tool became well known for genomic, biomedical, oil and gas discovery, etc. So…. increasingly visual tools are being used… we see a move to large display walls (10M to 100M pixels) helping productivity… Bloomsburg uses arrays of 8 screens with very fixed windows having huge value… we see radiology workstations with multiple displays to see a brain scan… some with 16 displays showing last weeks as well as this week’s scans… these sorts of workspaces are becoming common – multiple people sharing, collaborating, around multiple screens.

We are also seeing small screens (1M pixels and less) having a real impact… mobile screens with data such as Google’s expansive transportation interfaces through their maps, and historical data on that… There is a huge amount of data, our job as designers is to organise that, to understand data needed to make decisions…

So, the information visualisation mantra (and I once wrote this a dozen times in a paper – now cited over 27k times!):

• Overview – the full range of items
• Zoom and Filter – let the user do that, find what they want…
• Details-on-demand – let the user drill into the data

The most compelling part here is the centrality of the human user. It’s not just about the algorithm…

And if we think about the last 50 years of Scientific visualisation in 1D Linear (Document Lens; SeeSoft, Info Mural), 2D Map (GIS, ArcView, PageMaker; Medica Imagery) and 3D World (CAD, Medical, Molecules, Architecture) forms… and they have a great future. And we now have the new area of Information visualisation… often about muti-variable data (Spotfire, Tableau, Qliktech, Visual Insight), Temporal (LifeLines, TimeSearcher, Palantir, DataMontage); Tree (Cone/Cam/Hyperbolic/SpaceTree/Treemap); Network (Pajek, UCINext, NodeXL, Gephi, Tom Sawyer). Loads of blogs here that are worth a read: Flowing Data; Perceptual Ledge; Etc.

So, let me go to the first demo… traditionally we often look at temporal data… for instance Stock Market Data. So… overview first… so looking at a year… February has a lot of uncertainty. Now you (an audience member) mentioned a “spike”… is that a spike upwards? Or downwards? We have the wrong language for visual reasoning yet! Now we can zoom into this data… look through this data…. seek patterns… Information visualisation allows you to see new patterns, new changes, to ask new questions. So with this [demo] visualisation you can create a pattern and look for that in your data set… but people were interested in how one might do the opposite – make a pattern and explore by inverses of that pattern… that’s thought patterns you can’t explore on paper and you can do it rapidly, and readjust them on a screen… You can try out and test hypotheses easily with these tools – and you can try this out, look for “TimeSearcher”. TimeSearcher was designed to do time series for stocks, wealth, genes, and to work with large data sets and allow the user to really shape interactions.

Now another tool we built was LifeLines, an attempt to create a visualisation for Patient Histories – with the overview acting as routes into that medical history, to understand changes, medications, interactions… And one of the nice things I like is that visualisations can also show you what isn’t there… harder to do algorithmically… but you can see gaps that might be concerns, questions, it’s a starting point…. we thought one patient was good, but a million patients would be better… so we worked with some data from the Pediatric Trauma Centre in Washington DC and using a tool we built called EventFlow (also free to download). The hospital (via video recordings then transcribed) record initial checks – airway, breath sounds, distol and central pulse in the first few minutes… and then you get longer for the secondary checks… Looking over a large set of data (216 patients) you can get a sense of how quickly secondary checks occurred… And you can spot anomalies in how staff conducted checks – not dangerous perhaps but not the hospitals protocol…. And you can see all the ways that these patients have been seen, how they vary… the most common variance was starting the disability check before secondary checks… there are some repetitions… some took ages to get their checks done.

So talking about Treemaps… that was our work… for instance SmartMoney Stock Data… looking at a terrible day you see a single blip of good activity – a real clear contrast… often you see patterns that are more subtle… but that visual training happens when data is spatially fixed, when you can spot change…

Treemap: Newsmap (work by Marcos Weskamp) looks at global news items and the number of online articles on a given topic… you can compare countries’ coverage directly… again, a free to use/explore visualisation.

And we did some work with the Hive Group on tree maps for Nutritional Analysis. SpotFire added tree maps in 2007, Tableau now has it. the New York times have used tree maps now. And a German researcher developed the idea of Voronoi tree maps – they look cool and organic it can be hard to read. There is a design aesthetic aspect here, these look cool but are hard to compare size of spaces.

Manual Lima has a great site called VisualComplexity.com with thousands of network visualisations…

And the work we did was in a tool called Node XL, it’s free to download and use, and it’s a network overview for discovery and exploration in Exel… designed to show interactions and connections between people… So for instance can be used to see voting in the US Senate… And you can use NodeXL to directly import from Facebook, Twitter, YouTube etc… feel free to create another importer tool… So one of our first experiments was for #WIN09 Conference back in 2009… and you could see from the 80 people in the room a kind of split between two groups of people – computer scientists and sociologists – and in the tweets you saw that clearly shown… just one cross over in a graduate student!

And that sort of connecting and cross over issue is even more compelling in political discourse… So we did this for the #GOP tweets… you could see a very cohesive densely connected group of republians. A less connected group of democrats. And a few cross over people… but they talk within their group but very little interaction between them. Cross over only via Politico. Media consumed between these groups otherwise really diverged…

But, this work kinda works…. but not a great way to visualise… using grapes for inspiration we tried to restructure around smaller clusters, separations, etc. in a more clear to view way…. for instance used in looking at #SOTU (State of the Union Address).

And… a researcher called Scott Dempwolf who looks at Innovation Networks… he took data on companies, patents, grants from government agencies… 26k edges, 11k nodes…. so he has created a beautiful visualisation for Pensylvania Innovation Networks… but hard to read…. so we tried to break this down a bit…. found a major pair of nodes who hold a lot of patents…. And you see real cluster of some of the big players in innovation…. Westinghouse Electric and the Navy being key drivers here…. So drilling down you see the big players…

We asked Scott to show us something on Maryland…. he created a visualisation for our lab…. again looking at connections and gaps… we can also look at innovation in Chicago to see how we see clusters here… You begin to see the finer grained structure more clearly when you have a visual way into the data…

Recently we published this on the Pew website – you can see Node XL Gallery for more of this sort of data – looked at Twitter network structures: polarised crowds; Tight crowds; Brand clusters; broadcast network; community clusters; and support networks… for those doing customer support via Twitter…

So, you can read more. You can find out about our Social Media Research Group. And we also want to talk about not only business but also other spheres in which these tools can help, for instance the UN Millennium Development Goals… Some progress towards their goals… Bill Gates is helping with next goals… The Gates Foundation is a big user of Node XL… in that presentation earlier we saw visualisations via Bar Charts but understanding interactions is key here.

Q&A

Q: I’m sure over the next few days we’ll see a lot of papers with statistical analysis… what would your advice be for business and finance academics to get papers more visual, and get published…

A: A good question. You do see Science and Nature moving to printed visualisations… they are static…we have a long way to go to make those interactive… by contrast the web and blogs are much more interactive and visual… and increasingly you see that supplemental stuff – video or interactive website – online. Science encourages you to have a website, data if possible, and visualisation tools with your papers. Actually  there is an annual competition around visualisation run by Science and partners…

Q: This is on errors and potential for misrepresentation… with many of these tools there is so much potential to accidentally misrepresent the data…

A: You are right of course… statistics can lie, data can lie, and visualisations can lie… you can use colour, labelling, etc. in misleading ways. But for any visualisation I think an intelligent understanding can reduce that impact. But the majority of datasets I get into my office have errors that the person whose data set it is didn’t know about it…. I was looking at emergency room admissions data recently… 8 patients in that data were 999 years old… those kinds of errors are widely found in data, or a patient admitted 14 times, but discharged only twice… And you have people using flawed data to predict sales but miss one month when their sale is on! Statistics without visualisations risk never spotting that error… visualisation provides a sort of microscope, telescope… new ways to explore and understand our data. And you need a new sort of literacy, that concept of visual reasoning. And the tools have made that possible…

Q: You talked about a lack of vocabulary… what should we be using?

A: We have a tool, not quite as polished as a shape finder, but the question is can you make a measure of the spikiness of each spike? In books you see standards about what is and is not a spike. During a discussion a student suggested something brilliant… using the angles within the spike to find sharp spikes, and also areas of fall and rise. So we have started to explore this sort of stuff… but of course volatility can be a measure… but there are interesting shapes that we ca use and explore here… you have concepts like “value line”, sizes of plateau. It’s a rich space we’ve only just started to explore in the shape finder.

Q: In terms of the methodology to create these models… I am interested in customer journeys between social media channels, capturing those touch points between platforms…

A: You have some systems, like Klout, that gives you numeric data… but we are interested in networks here…. IBM did a project with their internal networks of these things, of connections in discussion. My colleague did work with emails, to see cohesiveness of discussions… but we are only 5 or 6 or 7 years into this social media world… but it’s definitely an opportunity to do good… And again there is an effort from the National Cancer Institute to use social media to make health related opportunities, for smoking cessation, obesity reduction, etc…. to get changes through use of social media… And you see media networks evolve. Jenny Priess and I wrote a paper called “From Reader to Leader”… On Wikipedia only 1/10th of 1% ever make an effort… and only 11,000 admins…. so we need to understand the dynamics of that… how one goes through that path, what the motivations, rewards, recognition, to encourage people along that path… The sciences of the natural world have been successful for 400 years but I think the science of the made world, of social structures, etc. is the science of the next 100 years.

Q: You mentioned bar charts etc. in my presentation earlier. We have looked at new ways to present this data… info graphics etc… there are a lot for quantitative data but fewer for qualitative data…

A: Well one step back…. it’s not about visualising your data…. it’s about your goal, your question, what are you trying to answer… in your data there was clearly more there… a simple taste of what’s possible… the network structure of these community might be interesting…. so it might be a geographic relationship… but you need to know the questions first, and use that to decide what you need, what you will find in the data, how you make new opportunities happen.

 

Mixed Methodologies Seminar – Professor Dan Remenyi 

Dan Remenyi is introducing himself as an itinerant academic, who teaches research methods at various universities and also supervises PhD students.

When I completed my PhD, rather late in life, I felt the most interesting part was the research methodologies but I felt like I needed to learn more in that area, and had a lot to learn. I have supervised a lot of PhDs now and most actually use “mixed methods” but, a bit like “reflection”, you needed to do this stuff… you have to do that… these days you can’t just do it, you actually have to write about, to describe that stuff. If you use the phrase “mixed methods” about your research – and I’m going to counsel you not to necessarily do that – you have to be able to say why you did that, what that means, what the implications are…

So today we will talk about what Mixed Methods really is, and how you talk about it… You should all have had the slides in advance… I took those slides and put them into Wordle… you can see I’ll be talking about Data, about Mixed Methods, and about Synthesis… Now… as I progress down this road of talking about research methodology I’ve learned that it is so important to understand the vocabulary of the research world, how to use them appropriately…. Some are easy perhaps but some are much more tricky. You should know these… I suggest you create your own glossary where you really pin down your own understanding of these words… You need to know what they mean, you need to be able to defend your work.

Now, lets talk Mixed Methods… Well this is an expression, some call it a misnomenclature – it really doesn’t explain what it does (a bit like Life Insurance, of Jumbo Shrimp, some often refer to “military intelligence” as the same type of misnomer!). Why? Well there is almost no way that methods can be mixed. What we mean is using both qualitative and quantitative data to make a convincing argument… In the previous talk the speaker talked about charts, visualisations, and that the research question is absolutely key. And that’s the case in methods… but think slightly wider than that… in actual fact when we do research the research enables us to understand better the research question, and come up with possible answers for it…

So what is usually meant by Mixed methods is that combination of qualitative and quantitative data in research. In your research you need to be contributing to the academy, both in terms of the findings and the theoretical aspects of the field. And you have to convincingly make your case. There is still a lot of confusion about Mixed Methods. Researchers sometimes lose sight f the fact that evidence, of whatever sort, is a constituent of the argument which underpins the findings. The challenging part is bringing these different dimensions of the argument into a convincing whole.

At it’s heart Mixed Methods is a research design issue. You can adjust that plan as you go along, academia is essentially about self-improvement… your plan will always emerge and involve as you go along. A research design might start with what data you require to answer the question, then think about how you will collect it. How will you analyse it? How will you use it to establish some findings? And increasingly you are expected to interpret those findings, to talk about what the implications of your research is.

So the term Mixed Methods is being used in two senses…

• – There is an emerging school of thought, or community of practice, that argue for the use of mixed methods research design.
• – There is the research practice which has been in place for decades which have called upon researchers to use different methods at different times, stages, phases in their research. Indeed it is hard to use an entirely quantitative approach in research.

Now, not all researchers welcome the concept of Mixed Methods… some think you have to be world class and that you cannot be world class quantitively or qualitatively…. the aspiration is to be world class but I think you can be extremely competent at both. But the philosophical argument is trickier… the ontological argument is that you can either be a realist – positivist, quantitative type road – or a relativist and that that takes you down the more constructivist, analytical route. In reality we are often a combination of both in reality…

Now the key person in this area, he has made it his own, is Creswell. He says you cannot tell your story unless you can put together the numbers behind your research and to tell the stories behind those numbers. He says that numbers never speak for themselves… you have to be able to see the numbers and the facts in context. Paulos (1998) talks about statistics as being uninterpretable without context, background, their origins then they cannot be properly understood…

An example here… stats on home runs in the US Baseball league show increasing numbers of home runs… what’s happening? More matches? More training? More reporting of games? Changes in recording measures? More rewards for better players? Stand out players like Babe Ruth? But a more important reason… they banned cheating! Generally Baseball was played in the afternoons… and the light got dimmer… flood lights weren’t great… pitchers started messing with the ball, spitting on it, rubbing it in the dirt… and the batter could see the ball…. How will you know that just looking at numbers? You won’t, you need some other form of research to understand that data. (For more on these stats Dan recommends Bill Bryson’s book A Short History of Nearly Everything – a great book for PhD students to read as, essentially, a history of science. And his book One Summer: 1927 include those statistics… in that book the most important thing is Charles Lindberg flying the Atlantic….)

Now, there is another phrase you need to be aware of and that is “Multiple Methods”… If you are using multiple methods in the qualitative arena then some say you are using Multiple Methods, that Mixed Methods is exclusively for the combination of Qualitative and Quantitative Methods. You also hear Combined Methods, Hybrid Methods, and (from an audience member) Multi-Level Methods.

A few really important distinctions… At the highest level research can either be Theoretical – this is based on secondary data, data that has been previously been published, and already-established ideas and you create something new from those existing ideas. Empirical Research is about the collecting of data. Now data is a hugely contested term, there is a surprising lack of papers on data… when I questioned what data was in a statistics department they thought I was mad but data is a really tricky term, I’ll come back to this.

Now, in theoretical research is highly linked to empirical research, but always relating that back to theory, and using existing empirical data.

And then we have the two major paradigms of Positivist which is about the qualitative world, numbers (mostly), the process is deductive so there are hypotheses that you are attempting to reject (you try to reject it, if you don’t you accept it pro tem), it’s interpretation with a “little i”. And we have Interpretivist approaches… an inductive process, uses a wide range of data… and it’s about taking that data and from it attempting to form a hypothesis from that. Now the vast majority of research is deductive, a faster process. An inductive approach can take longer and require much more data… Now… Mixed Methods sits between these, straddling both positivist and interprevist perspectives. And following a side chat on mathematical methods, mathematics fits not quite anywhere into these research paradigms… The concept of Ocham’s Razor is useful here: the explanation that the idea that is simplest is best… In general we can never say we have proved something… the only thing that is certain is that we know what we don’t know… But we can say “the evidence suggest”, or “it appears from the evidence”… that can be said… much harder to say that “the evidence shows this is true”.

Now… a comment on Qualitative and Quantitative research and how they differ…

In Quant: You articulate the research question, you collect evidence, you process evidence (questionnaire) – only after you have collected data, and you produce findings…

In Qual a learning loop is involved: you articulate the research question, you collect evidence (interview), you understand the question as you process the evidence and you really have a loop, you learn as you go, and you do produce your findings.

There are alternative approaches too… Action research often takes an iterative approach for instance.

Of course Mixed Methods can be used in theoretical work… you might collect data to support a theoretical perspective. And Mixed Methods are particularly useful in interdisciplinary work. And it can also be useful in applied research, where there are blurred boundaries between topics…

So we have 12 steps in research design:

Setting the course

Remember that to do competent academic research we not only have to understanding our data and analysis of that but we also have to understand all of the arguments in the body of knowledge, and we have to be able to articulate that. And that has to feed into the research design.

There are different ways to approach Mixed Methods research…. One way is to start with qualitative data as a way to reach understanding, and to design a quantitative instrument (e.g. a questionnaire) that is then deployed and leads to findings… It’s a big deal to create a questionnaire from scratch! And in this approach each step is distinct. You take two steps… one step followed by another… the mixing is very minimal…

But there is no reason not to take a different approach… You use an established research instrument to gather data, then you conclude that stage, and you take a qualitative approach next, in order to reach your findings. That’s a perfectly respectable Mixed Methods approach.

Now you can also take what they call a “supportive mixed methods” design… here you have overlap between types of research, you can benefit from understanding the data of one type in your work collecting data of another type. Now I like metaphor… so take the buttress (flying and not)…. someone pointed out to me that the way that Cathedrals are built is fundamentally unstable… will push the walls out… and that’s why buttresses, and flying buttresses came about. And I like to think of scientific discovery as not always standing on it’s own without data from a variety of different sources. Multiple sources of validation are always welcome… they act like buttresses… (and now we have a side chat in which Dan makes  the point that doctoral students should not touch longitudinal studies… “that’s a different methodological world”).

You should know that academic research gives you a great deal of flexibility in what you do. It is based on peer review – your papers will be seen by at least two people reviewing it – but there is a lot of flexibility as to how you do it. Paul Feyerbiant wrote a famous book, a difficult book, called “Against Method”. And in that book he says the only universally accepted academic research methods, and that is “anything goes”! It doesn’t mean you can be sloppy… it means no one can tell you how you must do your research, or what you cannot do… you can do it your way as long as you can convincingly argue your case, and show that you are contributing to the academic body. As long as you can argue that your methods got you to the right answer, you have to be able to argue your methods, to justify them… I had someone come up for examination who had done 35 interviewers… a particularly tough examiner who said he needed more… but how many do you need? Well you need as many as need before you reach the point of data saturation… you have to be able to justify the number that is acceptable. As it happened this guy went out and found a whole load of papers showing that 35 could be a valid number… this is part of why you have to understood the literature… you have to have read everything that can be read about your topic… And the other thing about academic research is that you have a lot of flexibility but you have to use the language consistently, and to understand the meaning of those words… we had a chat before about what it means to be longitudinal… it means an extended period of time… is that 3 months? 3 years? 3 weeks? For anthropologists they conduct ethnography, they talk about a lived experience… how many of us in the business or management world truly have a live experience… Ethnography is, as a word, taking liberties there… but we can talk about being “ethnographically informed”, by the same token we could talk about “a longitudinal type study”. Teet was talking about interviews over a few months as being not a snapshot… but argued appropriately you could use some of that language of longitudinal language… Because, as we’ve said, we have to be clear of making a clear and justifiable case for your choice of methods… We have so many methods but you have to be clever about how you put your argument together…

So… back to a third model for Mixed Methods… this is a parallel or converging Mixed Model… Where you undertake quantitative and qualitative research in parallel… now I have gone light on talking about “triangulation” here… some people love that term, some hate it… to be precise the word is borrowed from land surveyors who use various tools to map particular features, measuring from different angles… social scientists have borrowed that term to talk about different perspectives… now when I did my research 25 years ago I was told triangulation was a way to resolve conflicts and contradictions in the data… that is nonsense… by being able to look at things through different perspectives, different lens, different data, different people… you get a richer understanding of the question, of the issues involved. Now some say the term “triangulation” is too positivist, that something like corroboration is better…. I don’t really mind… more perspectives is usually better. BUT…. it is tempting to believe that the more panoramic the view, the better… and that may often be the case, but is not always true….  Sometimes putting all this extremely rich view into a cohesive whole can be really problematic… Research does not seek complexity for it’s own sake… If you have a credible answer to the research question from one or two data sources then the job is probably done… Answering the research question is the paramount issue.

So in this third approach, the parallel or converging mixed method design… we will get two sets of data, from two different sources, and bring them together into an argument… and we will draw on both sets of data to draw our conclusions… There’s no other sense in which we want to mix it… Now in the literature you will see some discussion of putting numbers into words and vice versa but I am not convinced by that. Some critical issues… were the two different data collection strategies driven by the same research question? If not, then why to? Was the same research logic used for both – i.e. inductive or deductive? And are the results commensurable? They don’t have to be but you will have to argue your case well, you have to change your argument and explain any contradictory results. And again, you have to answer the research question.

Now, reflection is central to research. It has always been necessary. But it’s now really important to be able to discuss it… Reflection may be defined as a process of questioning the range of activities and thinking which have been performed by the researcher in order to surface any inadequacies or bias which may be present in research. And why you have come to the conclusions you have come to.

Reflexivity – and the piece in MIS Quarterly is worth reading – is about seeing the interrelationships between the sets of assumptions, biases and perspectives that underpin the different facets of the research undertaken. So you might ask yourself what assumptions are at play when you start your research? All research starts with assumptions that there will be an answer to the question, that that question is worth answering, and that the process of answering that research question will change you, will develop you to a higher level in the case of a doctorate for instance. Reflexivity is about understanding that, of understanding biases… nobody likes to feel that they are biased… but you can’t get away from the facts what you are… so I’m a white, British, elderly, academic… all of those mean expectations and values… I might work against those but there are always some residues there… You also want to ask yourself what values of yours affect your research? So all of us have the shared values that knowledge is important for instance, we want to learn more. As someone in academia you also have to believe there is some value in sharing, that’s part of being an academic… you could explore all of that much further of course… but that’s what we mean by reflexivity.

Some mixed methods researchers talk about integrating the qualitative and the quantitative data so that an overarching analysis can be performed… so about how and when you mix the data… now I argue that we are really talking about synthesising the arguments. And the test of an argument is whether it convinces… There are various types of evidence which include data, authority and logical inference… So in academia argument is used to support theoretical conjectures. The way we learn is influenced by the Greeks… Socrates, regarded as close to a tramp, walking around picking arguments, who developed the idea of the dialectic… and that is how academia works… you articulate a thesis… you float an idea, then someone does the “ah, but…”, they correct the idea or take the antithesis… and then you put those together, you synthesise them, and create a new idea… and that re-articulation of thesis starts a new cycle… that’s an ancient concept that still underpins academia.

Now, Teet earlier mentioned a model like an Advanced Mixed Methods Design, something which may result in a case study, experiment or action research project. But what actually determines the method? This can be influenced by your background… an engineer may not want to work in qualitative research, a humanist may not want to undertake complex equations… So it may be about the scale of the work required, the skills that you have and, in the case of doctoral students it may also be about the influence of the supervisor or culture of the institution.

And with that, we are done.

 

Share/Bookmark